The Year in AI Papers
The year/Labs/NVIDIA

Research lab

NVIDIA

Research on models, simulation, robotics, graphics, systems, and accelerated AI computing.

Papers
27
Citations
808
Official code
13

27 papers from NVIDIA

01

Research paper

Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

The paper introduces Zone of Proximal Policy Optimization (ZPPO), a post-training method for small vision-language models (VLMs) that transfers knowledge from a larger teacher without imitating its logits or injecting its responses into the policy gradient. ZPPO addresses two failure modes: distillation's brittleness in the small-student regime and RL's…

Byung-Kwan Lee, Ximing Lu, Shizhe Diao, Minki Kang, et al.
Published
Jun 2026
Citations
2
Code
Not linked
Read summary Original
02

Research paper

SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning

SpatialClaw is a training-free framework that improves spatial reasoning in vision-language models (VLMs) by using code as the action interface. It maintains a persistent Python kernel pre-loaded with input frames and perception tools, allowing a VLM-backed agent to write and execute one code cell per step, inspect intermediate results (e.g., masks, depth…

Seokju Cho, Ryo Hachiuma, Abhishek Badki, Hang Su, et al.
Published
Jun 2026
Citations
3
Code
358 stars
Read summary Original
03

Research paper

Cosmos 3: Omnimodal World Models for Physical AI

NVIDIA introduces Cosmos 3, a family of omnimodal world models that jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. It subsumes vision-language models, video generators, world simulators, and world-action models into a single framework, supporting flexible input-output…

NVIDIA, :, Aditi, Niket Agarwal, et al.
Published
Jun 2026
Citations
36
Code
Not linked
Read summary Original
04

arXiv.org

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

The paper investigates whether vision-language models (VLMs) achieve spatial reasoning through structured 3D understanding or by exploiting statistical shortcuts in natural images. The authors introduce a representation-level analysis framework using contrastive pairs to measure how spatial axes (horizontal, vertical, depth) are organized in VLM…

Cheolhong Min, Jaeyun Jung, Daeun Lee, Hyeonseong Jeon, et al.
Published
May 2026
Citations
1
Code
16 stars
Read summary Original
05

arXiv.org

Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

The paper introduces Gamma-World, a generative multi-agent world model for interactive simulation that scales beyond two players. It addresses limitations of prior work like Solaris, which uses dense attention and learned per-slot identities, by proposing two key innovations. First, Simplex Rotary Agent Encoding extends 3D RoPE with an agent axis,…

Fangfu Liu, Kai He, Tianchang Shen, Tianshi Cao, et al.
Published
May 2026
Citations
2
Code
Not linked
Read summary Original
06

arXiv.org

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

The paper introduces AXPO (Agent eXplorative Policy Optimization) to address the Thinking-Acting Gap in multimodal agentic reasoning, where tool use is under-trained compared to thinking. Under GRPO, tool use occurs in only ~30% of rollouts, and tool-using subgroups are all-wrong on ~40% of questions, suppressing learning signals. AXPO fixes the thinking…

Minki Kang, Shizhe Diao, Ryo Hachiuma, Sung Ju Hwang, et al.
Published
May 2026
Citations
2
Code
Not linked
Read summary Original
07

arXiv.org

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

LocateAnything is a unified vision-language model for visual grounding and detection that introduces Parallel Box Decoding (PBD). Unlike standard next-token prediction (NTP) which serializes bounding box coordinates into 1D token streams, PBD treats each bounding box as an atomic unit, predicting all its coordinates in a single forward pass. This…

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, et al.
Published
May 2026
Citations
7
Code
Not linked
Read summary Original
08

arXiv.org

LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

LongLive-2.0 is an NVFP4-based parallel infrastructure for long video generation, co-designing training and inference. For training, it introduces Balanced SP, a sequence-parallel autoregressive (AR) training method that pairs clean-history and noisy-target temporal chunks on each GPU, enabling a natural teacher-forcing mask and SP-aware chunked VAE…

Yukang Chen, Luozhou Wang, Wei Huang, Shuai Yang, et al.
Published
May 2026
Citations
11
Code
2.5K stars
Read summary Original
09

arXiv.org

SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer

SANA-WM is a 2.6B-parameter open-source world model for generating one-minute, 720p videos with precise 6-DoF camera control. It uses a hybrid linear diffusion transformer combining frame-wise Gated DeltaNet (GDN) and softmax attention for efficient long-context modeling, a dual-branch camera control (UCPE and Plücker mixing), a two-stage generation…

Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, et al.
Published
May 2026
Citations
13
Code
Not linked
Read summary Original
10

arXiv.org

MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models

MEMLENS is a new benchmark for evaluating multimodal long-term memory in large vision-language models (LVLMs) and memory-augmented agents. It comprises 789 questions across five memory abilities (information extraction, multi-session reasoning, temporal reasoning, knowledge update, and answer refusal) at four context lengths (32K–256K tokens). An…

Xiyu Ren, Zhaowei Wang, Yiming Du, Zhongwei Xie, et al.
Published
May 2026
Citations
0
Code
26 stars
Read summary Original
11

arXiv.org

AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation

AnyFlow is a video diffusion distillation framework that enables any-step generation by learning flow-map transitions between arbitrary time pairs, unlike consistency models that degrade with more sampling steps. It uses a two-stage pipeline: forward flow map training (with interpolated timestep conditioning, guidance-fused training, and adaptive loss…

Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, et al.
Published
May 2026
Citations
6
Code
406 stars
Read summary Original
12

arXiv.org

TriAttention: Efficient Long Reasoning with Trigonometric KV Compression

TriAttention is a KV cache compression method for long-context LLM reasoning. It exploits the observation that pre-RoPE Q/K vectors are highly concentrated around fixed non-zero centers, which remain stable across positions and contexts. This concentration causes attention to follow predictable distance preferences, which can be modeled as a trigonometric…

Weian Mao, Xi Lin, Wei Huang, Yuxin Xie, et al.
Published
Apr 2026
Citations
14
Code
838 stars
Read summary Original
13

arXiv.org

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

Nemotron-Cascade 2 is an open 30B Mixture-of-Experts (MoE) model with 3B activated parameters, achieving best-in-class reasoning and agentic capabilities. It is the second open-weight LLM to achieve Gold Medal-level performance in the 2025 IMO, IOI, and ICPC World Finals, with 20x fewer parameters than DeepSeek-V3.2-Speciale-671B-A37B. The model builds on…

Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, et al.
Published
Mar 2026
Citations
20
Code
Not linked
Read summary Original
14

arXiv.org

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

MM-Zero is a reinforcement learning framework that enables zero-data self-evolution for Vision Language Models (VLMs). It introduces a tri-role system—Proposer, Coder, and Solver—all initialized from the same base model and trained sequentially using Group Relative Policy Optimization (GRPO). The Proposer generates visual descriptions and questions, the…

Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, et al.
Published
Mar 2026
Citations
12
Code
79 stars
Read summary Original
15

arXiv.org

On Data Engineering for Scaling LLM Terminal Capabilities

This paper addresses the lack of transparency in training data for terminal-capable LLMs by introducing Terminal-Task-Gen, a synthetic task generation pipeline, and Terminal-Corpus, a large-scale open-source dataset. The authors fine-tune Qwen3 models (8B, 14B, 32B) to create Nemotron-Terminal, achieving substantial gains on Terminal-Bench 2.0: 8B improves…

Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, et al.
Published
Feb 2026
Citations
15
Code
Not linked
Read summary Original
16

arXiv.org

PhyCritic: Multimodal Critic Models for Physical AI

PhyCritic is a multimodal critic model designed for physical AI tasks, addressing the lack of physics-aware evaluation in existing judge models. It uses a two-stage RLVR pipeline: first, a physical skill warmup stage with GRPO on physical QA pairs to enhance perception and reasoning; second, self-referential critic finetuning, where the critic generates…

Tianyi Xiong, Shihao Wang, Guilin Liu, Yi Dong, et al.
Published
Feb 2026
Citations
9
Code
Not linked
Read summary Original
17

arXiv.org

Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text

The paper introduces Golden Goose, a method to synthesize unlimited Reinforcement Learning with Verifiable Rewards (RLVR) tasks from unverifiable internet text by converting them into multiple-choice fill-in-the-middle questions. Given a source text, an LLM masks a contiguous span of crucial reasoning steps (the ground-truth answer) and generates diverse,…

Ximing Lu, David Acuna, Jaehun Jung, Jian Hu, et al.
Published
Jan 2026
Citations
6
Code
Not linked
Read summary Original
18

arXiv.org

Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning

Fast-ThinkAct is an efficient reasoning framework for Vision-Language-Action (VLA) tasks that compresses lengthy chain-of-thought (CoT) reasoning into compact latent representations. It uses a teacher-student distillation approach, where a textual teacher VLM (trained with GRPO) provides reasoning traces, and a latent student VLM learns to generate…

Chi-Pin Huang, Yunze Man, Zhiding Yu, Min-Hung Chen, et al.
Published
Jan 2026
Citations
17
Code
Not linked
Read summary Original
19

arXiv.org

Motion Attribution for Video Generation

The paper introduces Motive, a gradient-based data attribution framework for video generation models that isolates motion from static appearance. It uses motion-weighted loss masks, computed via AllTracker optical flow, to focus influence scores on dynamic regions. Motive scales via single-timestep estimation, common randomness, frame-length normalization,…

Xindi Wu, Despoina Paschalidou, Jun Gao, Antonio Torralba, et al.
Published
Jan 2026
Citations
2
Code
Not linked
Read summary Original
20

arXiv.org

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization

The paper identifies a flaw in applying Group Relative Policy Optimization (GRPO) to multi-reward reinforcement learning: normalizing the summed reward causes distinct reward combinations to collapse into identical advantage values, reducing training signal resolution and causing suboptimal convergence or early failure. To address this, the authors propose…

Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, et al.
Published
Jan 2026
Citations
114
Code
495 stars
Read summary Original
21

arXiv.org

ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration

The paper introduces ToolOrchestra, a method for training small language models as orchestrators that coordinate diverse tools, including basic tools (web search, code interpreter), specialized LLMs (math, coding), and generalist LLMs (GPT-5, Claude Opus 4.1). The resulting 8B-parameter Orchestrator model is trained end-to-end with reinforcement learning…

Hongjin Su, Shizhe Diao, Ximing Lu, Mingjie Liu, et al.
Published
Nov 2025
Citations
32
Code
752 stars
Read summary Original
22

arXiv.org

TiDAR: Think in Diffusion, Talk in Autoregression

TiDAR is a sequence-level hybrid architecture that combines diffusion and autoregressive (AR) language modeling in a single forward pass using structured attention masks. It drafts tokens in parallel via diffusion (thinking) and samples final outputs autoregressively (talking), exploiting free GPU compute slots to achieve high throughput without quality…

Jingyu Liu, Xin Dong, Zhifan Ye, Rishabh Mehta, et al.
Published
Nov 2025
Citations
33
Code
Not linked
Read summary Original
23

arXiv.org

OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM

OmniVinci is an open-source omni-modal LLM from NVIDIA that jointly understands vision, audio, and text. The paper introduces three architectural innovations: OmniAlignNet, which aligns vision and audio embeddings in a shared latent space via contrastive learning; Temporal Embedding Grouping (TEG), which organizes embeddings by timestamps to capture…

Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, et al.
Published
Oct 2025
Citations
50
Code
675 stars
Read summary Original
24

arXiv.org

QeRL: Beyond Efficiency -- Quantization-enhanced Reinforcement Learning for LLMs

QeRL is a framework that combines NVFP4 4-bit quantization with Low-Rank Adaptation (LoRA) to accelerate and reduce memory usage in reinforcement learning (RL) for large language models (LLMs). The authors find that quantization noise increases policy entropy, which enhances exploration during RL, contrary to its detrimental effect in supervised…

Wei Huang, Yi Ge, Shuai Yang, Yicheng Xiao, et al.
Published
Oct 2025
Citations
16
Code
512 stars
Read summary Original
25

arXiv.org

Fast-dLLM v2: Efficient Block-Diffusion LLM

Fast-dLLM v2 is a block diffusion language model that adapts pretrained autoregressive LLMs (Qwen2.5-Instruct 1.5B and 7B) for parallel text generation. It requires only ~1B tokens of fine-tuning, a 500x reduction compared to full-attention diffusion models like Dream (580B tokens). The method uses block-wise diffusion with complementary masking and a…

Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, et al.
Published
Sep 2025
Citations
114
Code
1.1K stars
Read summary Original
26

arXiv.org

SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer

SANA-Video is a small diffusion model for efficient, high-resolution (up to 720×1280) and minute-long video generation, deployable on RTX 5090 GPUs. It uses a Linear DiT with linear attention (O(N) complexity) and a constant-memory KV cache for block linear attention, enabling long videos with fixed memory. Training cost is 12 days on 64 H100 GPUs (1% of…

Junsong Chen, Yuyang Zhao, Jincheng Yu, Ruihang Chu, et al.
Published
Sep 2025
Citations
82
Code
8.7K stars
Read summary Original
27

arXiv.org

LongLive: Real-time Interactive Long Video Generation

LONGLIVE is a frame-level autoregressive (AR) framework for real-time, interactive long video generation, addressing efficiency and quality challenges in diffusion and AR models. It introduces KV-recache to refresh cached states with new prompts for smooth, adherent prompt switches; streaming long tuning to enable train-long-test-long alignment; and…

Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, et al.
Published
Sep 2025
Citations
189
Code
Not linked
Read summary Original