The Year in AI Papers
The year/Topics/Video and spatial AI

Topic area

Video and spatial AI

Every collection across video and spatial ai.

Papers
164
Research labs
4
Official code
135

51–100 of 164 papers in this topic area

51

Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers

Video Analysis and Generation via a Semantic Progress Function

This paper introduces the Semantic Progress Function (SPF), a one-dimensional representation that captures how semantic meaning evolves over time in video sequences. The SPF is computed by measuring pairwise semantic distances between frames using a pretrained embedder (SigLIP) and fitting a smooth curve via regularized least squares. Departures from a…

Gal Metzer, Sagi Polaczek, Ali Mahdavi-Amiri, Raja Giryes, et al.
Published
Apr 2026
Citations
1
Code
38 stars
Read summary Original
52

arXiv.org

CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generation

CoInteract is an end-to-end framework for speech-driven human-object interaction (HOI) video synthesis, conditioned on a person reference image, a product reference image, text prompts, and speech audio. It addresses two common failures in diffusion models: structural instability in hands and faces, and physically implausible contact (e.g., hand-object…

Xiangyang Luo, Xiaozhe Xin, Tao Feng, Xu Guo, et al.
Published
Apr 2026
Citations
0
Code
165 stars
Read summary Original
53

arXiv.org

Seedance 2.0: Advancing Video Generation for World Complexity

Seedance 2.0, released by ByteDance in early February 2026, is a native multimodal audio-video generation model that supports text, image, audio, and video inputs. It generates 4-15 second clips at 480p/720p, with a Fast version for low latency. The model excels in real-world complexity, multimodal reference and editing, high-fidelity binaural audio, and…

Team Seedance, De Chen, Liyang Chen, Xin Chen, et al.
Published
Apr 2026
Citations
78
Code
Not linked
Read summary Original
54

arXiv.org

HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

HY-World 2.0 is an open-source multi-modal world model framework that unifies 3D world generation and reconstruction. For sparse inputs (text or single-view images), it generates navigable 3D Gaussian Splatting (3DGS) scenes via a four-stage pipeline: panorama generation (HY-Pano 2.0), trajectory planning (WorldNav), world expansion (WorldStereo 2.0), and…

Team HY-World, Chenjie Cao, Xuhui Zuo, Zhenwei Wang, et al.
Published
Apr 2026
Citations
19
Code
2.5K stars
Read summary Original
55

arXiv.org

SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments

SpatialEvo introduces a self-evolving framework for 3D spatial reasoning that replaces model consensus with deterministic geometric feedback. The core contribution is the Deterministic Geometric Environment (DGE), which defines validation rules for 16 spatial reasoning task categories and computes exact ground truth from point clouds and camera poses,…

Dinging Li, Yingxiu Zhao, Xinrui Cheng, Kangheng Lin, et al.
Published
Apr 2026
Citations
3
Code
80 stars
Read summary Original
56

arXiv.org

OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

OmniShow is an end-to-end framework for Human-Object Interaction Video Generation (HOIVG), synthesizing videos conditioned on text, reference images, audio, and pose. It introduces Unified Channel-wise Conditioning to inject reference images and pose via channel concatenation with pseudo-frame tokens, and Gated Local-Context Attention for precise…

Donghao Zhou, Guisheng Liu, Hao Yang, Jiatong Li, et al.
Published
Apr 2026
Citations
3
Code
464 stars
Read summary Original
57

ACM Transactions on Graphics

Strips as Tokens: Artist Mesh Generation with Native UV Segmentation

The paper introduces Strips as Tokens (SATO), a framework for generating artist-quality 3D meshes with native UV segmentation using autoregressive transformers. SATO uses a strip-based tokenization strategy that serializes meshes as connected chains of faces, preserving edge flow and structural regularity. This unified representation allows the same token…

Rui Xu, Dafei Qin, Kaichun Qiao, Qiujie Dong, et al.
Published
Apr 2026
Citations
2
Code
181 stars
Read summary Original
58

arXiv.org

When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models

The paper introduces NUMINA, a training-free framework for improving numerical alignment in text-to-video diffusion models, which often fail to generate the correct number of objects specified in prompts. NUMINA uses an identify-then-guide paradigm: first, it identifies prompt-layout inconsistencies by selecting discriminative self- and cross-attention…

Zhengyang Sun, Yu Chen, Xin Zhou, Xiaofan Li, et al.
Published
Apr 2026
Citations
0
Code
68 stars
Read summary Original
59

arXiv.org

WildDet3D: Scaling Promptable 3D Detection in the Wild

WildDet3D introduces a unified, geometry-aware architecture for open-vocabulary monocular 3D object detection that accepts text, point, and box prompts, and optionally incorporates depth cues at inference time. The model uses dual-vision encoders (image and RGBD) with a depth fusion module, a promptable detector, and a 3D detection head with unambiguous…

Weikai Huang, Jieyu Zhang, Sijun Li, Taoyang Jia, et al.
Published
Apr 2026
Citations
7
Code
606 stars
Read summary Original
60

arXiv.org

LPM 1.0: Video-based Character Performance Model

LPM 1.0 is a video-based character performance model that generates identity-consistent conversational videos in real time. It addresses the 'performance trilemma'—the challenge of achieving high expressiveness, real-time inference, and long-horizon identity stability simultaneously. The system comprises a 17B-parameter Diffusion Transformer (Base LPM)…

Ailing Zeng, Casper Yang, Chauncey Ge, Eddie Zhang, et al.
Published
Apr 2026
Citations
5
Code
361 stars
Read summary Original
61

arXiv.org

Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

Video-MME-v2 is a new benchmark for evaluating video multimodal large language models (MLLMs), designed to address the gap between inflated leaderboard scores and real-world capabilities. It introduces a progressive three-level hierarchy (visual information aggregation, temporal dynamics modeling, and complex reasoning) and a group-based evaluation…

Chaoyou Fu, Haozhi Yuan, Yuhao Dong, Yi-Fan Zhang, et al.
Published
Apr 2026
Citations
23
Code
369 stars
Read summary Original
62

arXiv.org

Generative World Renderer

The paper introduces a large-scale dataset for generative world rendering, curated from two AAA games (Cyberpunk 2077 and Black Myth: Wukong) to address the domain gap in inverse and forward rendering. The dataset includes over 4 million frames at 720p/30 FPS, with synchronized RGB and five G-buffer channels (depth, normals, albedo, metallic, roughness),…

Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan, Ruihan Yu, et al.
Published
Apr 2026
Citations
1
Code
709 stars
Read summary Original
63

arXiv.org

A Simple Baseline for Streaming Video Understanding

The paper introduces SIMPLESTREAM, a minimal baseline for streaming video understanding that feeds only the most recent N frames to an off-the-shelf VLM, without any memory, retrieval, or compression mechanisms. Evaluated on OVO-Bench and StreamingBench, SIMPLESTREAM with Qwen3-VL-8B and 4 frames achieves 67.7% average accuracy on OVO-Bench and 80.59% on…

Yujiao Shen, Shulin Tian, Jingkang Yang, Ziwei Liu
Published
Apr 2026
Citations
13
Code
157 stars
Read summary Original
64

arXiv.org

VOID: Video Object and Interaction Deletion

VOID is a video object removal framework that generates physically plausible counterfactual videos when an object is removed, addressing limitations of existing methods that only handle photometric effects like shadows. It uses a two-pass approach: first, a video diffusion model (CogVideoX) synthesizes a counterfactual trajectory guided by a quadmask,…

Saman Motamed, William Harvey, Benjamin Klein, Luc Van Gool, et al.
Published
Apr 2026
Citations
4
Code
2K stars
Read summary Original
65

arXiv.org

Video-Oasis: Rethinking Evaluation of Video Understanding

The paper introduces Video-Oasis, a diagnostic suite for auditing video understanding benchmarks. It finds that 55% of samples in 14 existing benchmarks can be solved without visual or temporal context, and that state-of-the-art Video-LLMs perform only marginally above random chance on the remaining video-native challenges. The suite includes…

Geuntaek Lim, Sungjune Park, Jaeyun Lee, Inwoong Lee, et al.
Published
Mar 2026
Citations
1
Code
34 stars
Read summary Original
66

Google DeepMind

VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward

VGGRPO is a framework for geometry-aware post-training of video diffusion models, addressing geometric drift and unstable camera motion. It introduces a Latent Geometry Model (LGM) that stitches video diffusion latents to a geometry foundation model (e.g., Any4D) via a lightweight connector, enabling direct prediction of 4D scene geometry (camera poses,…

Zhaochong An, Orest Kupyn, Théo Uscidda, Andrea Colaco, et al.
Published
Mar 2026
Citations
14
Code
Not linked
Read summary Original
67

arXiv.org

ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytelling

ShotStream is a novel causal multi-shot video generation architecture that enables interactive storytelling and real-time synthesis at 16 FPS on a single GPU. It reformulates multi-shot generation as a next-shot prediction task, allowing users to guide narratives via streaming prompts. The method first fine-tunes a text-to-video model into a bidirectional…

Yawen Luo, Xiaoyu Shi, Junhao Zhuang, Yutian Chen, et al.
Published
Mar 2026
Citations
14
Code
177 stars
Read summary Original
68

arXiv.org

Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models

The paper introduces Hybrid Memory, a paradigm for video world models that requires maintaining static background consistency while tracking dynamic subjects during out-of-view intervals. The authors construct HM-World, a large-scale dataset of 59K high-fidelity clips with 17 scenes, 49 subjects, and designed exit-entry events, and propose HyDRA, a memory…

Kaijin Chen, Dingkang Liang, Xin Zhou, Yikang Ding, et al.
Published
Mar 2026
Citations
11
Code
269 stars
Read summary Original
69

arXiv.org

Lingshu-Cell: A generative cellular world model for transcriptome modeling toward virtual cells

Lingshu-Cell is a masked discrete diffusion model (MDDM) for generative modeling of single-cell transcriptomics, introduced by Alibaba DAMO Academy. It models transcriptomic state distributions across ~18,000 genes without prior gene selection, operating directly in a discrete token space compatible with sparse, non-sequential scRNA-seq data. The model…

Han Zhang, Guo-Hua Yuan, Chaohao Yuan, Tingyang Xu, et al.
Published
Mar 2026
Citations
3
Code
Not linked
Read summary Original
70

arXiv.org

WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG

WildWorld is a large-scale, action-conditioned world modeling dataset automatically collected from the AAA game Monster Hunter: Wilds. It contains over 108 million frames with more than 450 actions (movement, attacks, skill casting) and per-frame annotations including character skeletons, world states, camera poses, and depth maps. The dataset addresses…

Zhen Li, Zian Meng, Shuwei Shi, Wenshuo Peng, et al.
Published
Mar 2026
Citations
6
Code
420 stars
Read summary Original
71

arXiv.org

Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models

Omni-WorldBench is a new benchmark for evaluating the interactive response capabilities of video-based world models, addressing the gap left by existing benchmarks that focus on visual fidelity or static 3D reconstruction. It comprises Omni-WorldSuite, a set of 1,068 prompts with initial frames and optional camera trajectories, organized into three…

Meiqi Wu, Zhixin Cai, Fufangchen Zhao, Xiaokun Feng, et al.
Published
Mar 2026
Citations
5
Code
107 stars
Read summary Original
72

arXiv.org

Speed by Simplicity: A Single-Stream Architecture for Fast Audio-Video Generative Foundation Model

daVinci-MagiHuman is an open-source audio-video generative foundation model for human-centric generation, jointly producing synchronized video and audio via a single-stream Transformer that processes text, video, and audio in a unified token sequence using self-attention only. This design avoids multi-stream complexity and supports multilingual generation…

SII-GAIR, Sand. ai, :, Ethan Chern, et al.
Published
Mar 2026
Citations
12
Code
2.1K stars
Read summary Original
73

arXiv.org

SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing

SAMA is a framework for instruction-guided video editing that factorizes the task into semantic anchoring and motion modeling. It uses Semantic Anchoring to predict semantic tokens from sparse anchor frames, enabling instruction-aware structural planning, and Motion Alignment, which pre-trains the backbone on motion-centric pretext tasks (cube inpainting,…

Xinyao Zhang, Wenkai Dong, Yuxin Song, Bo Fang, et al.
Published
Mar 2026
Citations
3
Code
207 stars
Read summary Original
74

arXiv.org

3DreamBooth: High-Fidelity 3D Subject-Driven Video Generation Model

The paper introduces 3DreamBooth, a framework for 3D-aware video customization that generates view-consistent videos of a subject from a few multi-view reference images. It addresses the limitation of existing subject-driven video generation methods that treat subjects as 2D entities, lacking 3D geometry priors. The framework comprises two components:…

Hyun-kyu Ko, Jihyeon Park, Younghyun Kim, Dongheok Park, et al.
Published
Mar 2026
Citations
0
Code
60 stars
Read summary Original
75

arXiv.org

MosaicMem: Hybrid Spatial Memory for Controllable Video World Models

MosaicMem is a hybrid spatial memory mechanism for video world models that combines explicit 3D structure with implicit, attention-based conditioning. It lifts video patches into 3D for precise localization and retrieval, then composes them in the queried view via a patch-and-compose interface, allowing the model to preserve persistent elements while…

Wei Yu, Runjia Qian, Yumeng Li, Liquan Wang, et al.
Published
Mar 2026
Citations
10
Code
Not linked
Read summary Original
76

arXiv.org

WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation

WorldCam is an interactive 3D gaming world model that uses camera pose as a unifying geometric representation to achieve precise action control, long-horizon generation, and 3D consistency. It maps user actions (keyboard/mouse) to 6-DoF camera poses via Lie algebra, injecting them into a video DiT (Wan-2.1) for accurate control. Global camera poses serve…

Jisu Nam, Yicong Hong, Chun-Hao Paul Huang, Feng Liu, et al.
Published
Mar 2026
Citations
6
Code
179 stars
Read summary Original
77

Independent research

Demystifying Video Reasoning

This paper challenges the Chain-of-Frames (CoF) hypothesis for reasoning in diffusion-based video models, proposing instead that reasoning primarily emerges along the diffusion denoising steps, termed Chain-of-Steps (CoS). Through qualitative analysis and noise perturbation experiments, the authors show that models explore multiple candidate solutions in…

Ruisi Wang, Zhongang Cai, Fanyi Pu, Junxiang Xu, et al.
Published
Mar 2026
Citations
6
Code
47 stars
Read summary Original
78

arXiv.org

Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation

Kinema4D is a new action-conditioned 4D generative robotic simulator that models robot-world interactions as 4D spatiotemporal events. It disentangles simulation into precise 4D robot control via URDF-based kinematics and generative 4D environmental reactions using a diffusion transformer that synthesizes synchronized RGB and pointmap sequences. The…

Mutian Xu, Tianbao Zhang, Tianqi Liu, Zhaoxi Chen, et al.
Published
Mar 2026
Citations
6
Code
83 stars
Read summary Original
79

arXiv.org

Grounding World Simulation Models in a Real-World Metropolis

The paper introduces Seoul World Model (SWM), a city-scale world model that grounds autoregressive video generation in the real city of Seoul via retrieval-augmented conditioning on street-view images. SWM fine-tunes Cosmos-Predict2.5-2B on 440k Seoul street-view images, real driving videos, and synthetic urban data. It addresses three challenges: temporal…

Junyoung Seo, Hyunwook Choi, Minkyung Kwon, Jinhyeok Choi, et al.
Published
Mar 2026
Citations
6
Code
624 stars
Read summary Original
80

arXiv.org

Spatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training

Spatial-TTT is a framework for streaming visual-based spatial intelligence that uses test-time training (TTT) to maintain adaptive fast weights as a compact memory for accumulating 3D evidence from long-horizon videos. It employs a hybrid architecture interleaving TTT layers with self-attention anchor layers at a 3:1 ratio, large-chunk updates, and…

Fangfu Liu, Diankun Wu, Jiawei Chi, Yimo Cai, et al.
Published
Mar 2026
Citations
3
Code
249 stars
Read summary Original
81

arXiv.org

Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence

Holi-Spatial is a fully automated pipeline that converts raw video streams into holistic 3D spatial annotations without human intervention, addressing the scalability limits of existing spatial intelligence datasets that rely on manually annotated 3D scans. The pipeline has three stages: geometric optimization using 3D Gaussian Splatting (3DGS) with…

Yuanyuan Gao, Hao Li, Yifei Liu, Xinhao Ji, et al.
Published
Mar 2026
Citations
5
Code
373 stars
Read summary Original
82

arXiv.org

Helios: Real Real-Time Long Video Generation Model

Helios is a 14B autoregressive diffusion model for video generation that achieves 19.5 FPS on a single NVIDIA H100 GPU, enabling minute-scale generation without common anti-drifting heuristics (e.g., self-forcing, error-banks) or standard acceleration techniques (e.g., KV-cache, quantization). It unifies T2V, I2V, and V2V tasks via a unified input…

Shenghai Yuan, Yuanyang Yin, Zongjian Li, Xinwei Huang, et al.
Published
Mar 2026
Citations
33
Code
2K stars
Read summary Original
83

arXiv.org

Utonia: Toward One Encoder for All Point Clouds

Utonia introduces a single self-supervised point transformer encoder trained jointly on diverse point cloud domains, including indoor scans, outdoor LiDAR, remote sensing, object CAD, and video-lifted point clouds. The authors identify three core cross-domain mismatches: inconsistent modality availability, sensitivity to granularity shifts, and bias toward…

Yujia Zhang, Xiaoyang Wu, Yunhan Yang, Xianzhe Fan, et al.
Published
Mar 2026
Citations
9
Code
717 stars
Read summary Original
84

Google DeepMind

LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory

LoGeR (Long-context Geometric Reconstruction) is a novel architecture for scaling feedforward dense 3D reconstruction to extremely long video sequences (up to 19k frames) without post-optimization. It processes video in chunks, using bidirectional attention for intra-chunk reasoning and a hybrid memory module for inter-chunk coherence. The hybrid memory…

Junyi Zhang, Charles Herrmann, Junhwa Hur, Chen Sun, et al.
Published
Mar 2026
Citations
22
Code
609 stars
Read summary Original
85

arXiv.org

OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

OmniLottie is a versatile framework for generating high-quality vector animations from multi-modal instructions (text, image, and video). It addresses the challenge of raw Lottie JSON files containing extensive invariant structural metadata by introducing a Lottie tokenizer that converts JSON into compact sequences of commands and parameters. This…

Yiying Yang, Wei Cheng, Sijin Chen, Honghao Fu, et al.
Published
Mar 2026
Citations
0
Code
732 stars
Read summary Original
86

arXiv.org

The Trinity of Consistency as a Defining Principle for General World Models

This paper proposes that a General World Model must be grounded in the Trinity of Consistency: Modal Consistency (semantic interface), Spatial Consistency (geometric basis), and Temporal Consistency (causal engine). The authors systematically review the evolution of multimodal learning from specialized modules to unified architectures, arguing that…

Jingxuan Wei, Siyuan Li, Yuhang Xu, Zheng Sun, et al.
Published
Feb 2026
Citations
4
Code
Not linked
Read summary Original
87

arXiv.org

SkyReels-V4: Multi-modal Video-Audio Generation, Inpainting and Editing model

SkyReels-V4 is a unified multi-modal video foundation model for joint video-audio generation, inpainting, and editing. It uses a dual-stream Multimodal Diffusion Transformer (MMDiT) architecture with separate video and audio branches sharing a Multimodal Large Language Model (MLLM) text encoder. The model accepts text, images, video clips, masks, and audio…

Guibin Chen, Dixuan Lin, Jiangping Yang, Youqiang Zhang, et al.
Published
Feb 2026
Citations
6
Code
Not linked
Read summary Original
88

arXiv.org

A Very Big Video Reasoning Suite

The paper introduces the Very Big Video Reasoning (VBVR) suite, comprising a large-scale dataset and benchmark for video reasoning. VBVR-Dataset contains 2,015,000 images and 1,007,500 video clips across 200 tasks, organized by five cognitive faculties (Abstraction, Knowledge, Perception, Spatiality, Transformation), and is approximately 1000x larger than…

Maijunxian Wang, Ruisi Wang, Juyi Lin, Ran Ji, et al.
Published
Feb 2026
Citations
19
Code
225 stars
Read summary Original
89

arXiv.org

Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions

The paper introduces ASID-1M, an open-source dataset of one million attribute-structured audiovisual instruction annotations, along with ASID-Verify, a multi-stage data curation pipeline for automatic annotation, verification, and refinement, and ASID-Captioner, a video understanding model trained via supervised fine-tuning on this dataset. The pipeline…

Yunheng Li, Hengrui Zhang, Meng-Hao Guo, Wenzhao Gao, et al.
Published
Feb 2026
Citations
5
Code
68 stars
Read summary Original
90

arXiv.org

MOVA: Towards Scalable and Synchronized Video-Audio Generation

MOVA is an open-source model for generating synchronized video and audio, including lip-synced speech, sound effects, and music. It uses a Mixture-of-Experts architecture with 32B total parameters (18B active) and supports Image-Text to Video-Audio generation. The model couples a pretrained A14B video DiT and a 1.3B audio DiT via a 2.6B bidirectional…

SII-OpenMOSS Team, :, Donghua Yu, Mingshu Chen, et al.
Published
Feb 2026
Citations
29
Code
1.1K stars
Read summary Original
91

arXiv.org

VidVec: Unlocking Video MLLM Embeddings for Video-Text Retrieval

VidVec introduces a method to leverage generative Multimodal Large Language Models (MLLMs) for video–text retrieval without visual fine-tuning. The authors show that intermediate layers of off-the-shelf video MLLMs (e.g., VideoLLaMA3-7B) encode substantial retrieval-relevant information, outperforming final-layer embeddings in zero-shot settings. They…

Issar Tzachor, Dvir Samuel, Rami Ben-Ari
Published
Feb 2026
Citations
3
Code
Not linked
Read summary Original
92

arXiv.org

3D-Aware Implicit Motion Control for View-Adaptive Human Video Generation

The paper introduces 3DiMo, a framework for 3D-aware human motion control in video generation. It addresses limitations of existing methods that use 2D poses (viewpoint-bound) or explicit 3D models like SMPL (inaccurate, overriding generator priors). 3DiMo jointly trains a Transformer-based motion encoder with a pretrained DiT video generator to distill…

Zhixue Fang, Xu He, Songlin Tang, Haoxian Zhang, et al.
Published
Feb 2026
Citations
3
Code
Not linked
Read summary Original
93

arXiv.org

Advancing Open-source World Models

LingBot-World is an open-source world simulator built from video generation, offering high-fidelity dynamics across diverse environments, minute-level long-term memory, and real-time interactivity with sub-second latency. It uses a three-stage training pipeline: pre-training on a 14B-parameter Wan2.2 model, middle-training with a mixture-of-experts (MoE)…

Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, et al.
Published
Jan 2026
Citations
87
Code
4.3K stars
Read summary Original
94

arXiv.org

The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation

This paper introduces an agentic framework for generating long-horizon cinematic videos from dialogue. The framework comprises three agents: ScripterAgent, which translates dialogue into detailed cinematic scripts; DirectorAgent, which orchestrates video generation using a cross-scene continuous generation strategy with frame-anchoring to ensure temporal…

Chenyu Mu, Xin He, Qu Yang, Wanshun Chen, et al.
Published
Jan 2026
Citations
3
Code
Not linked
Read summary Original
95

arXiv.org

Why Can't I Open My Drawer? Mitigating Object-Driven Shortcuts in Zero-Shot Compositional Action Recognition

This paper addresses object-driven shortcuts in Zero-Shot Compositional Action Recognition (ZS-CAR), where models predict verbs by relying on object class labels rather than temporal evidence. The authors identify two root causes: sparse and skewed compositional supervision, and asymmetric learning difficulty (objects are easier to recognize than verbs).…

Geo Ahn, Inwoong Lee, Taeoh Kim, Minho Shim, et al.
Published
Jan 2026
Citations
0
Code
9 stars
Read summary Original
96

Annual Meeting of the Association for Computational Linguistics

HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding

HERMES is a training-free framework for efficient streaming video understanding in Multimodal Large Language Models (MLLMs). It conceptualizes the KV cache as a hierarchical memory system based on a mechanistic attention analysis, where shallow layers act as sensory memory with recency bias, deep layers as long-term memory focusing on frame-level anchor…

Haowei Zhang, Shudong Yang, Jinlan Fu, See-Kiong Ng, et al.
Published
Jan 2026
Citations
22
Code
94 stars
Read summary Original
97

arXiv.org

FlowAct-R1: Towards Interactive Humanoid Video Generation

FlowAct-R1 is a framework for real-time interactive humanoid video generation, built on a MMDiT architecture (Seedance). It enables streaming synthesis of arbitrary-length videos with low latency, achieving 25fps at 480p resolution with a time-to-first-frame of about 1.5 seconds. The method uses a chunkwise diffusion forcing strategy with a self-forcing…

Lizhen Wang, Yongming Zhu, Zhipeng Ge, Youwei Zheng, et al.
Published
Jan 2026
Citations
7
Code
Not linked
Read summary Original
98

NVIDIA

Motion Attribution for Video Generation

The paper introduces Motive, a gradient-based data attribution framework for video generation models that isolates motion from static appearance. It uses motion-weighted loss masks, computed via AllTracker optical flow, to focus influence scores on dynamic regions. Motive scales via single-timestep estimation, common randomness, frame-length normalization,…

Xindi Wu, Despoina Paschalidou, Jun Gao, Antonio Torralba, et al.
Published
Jan 2026
Citations
2
Code
Not linked
Read summary Original
99

arXiv.org

InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields

InfiniDepth introduces a novel depth representation that models depth as neural implicit fields, enabling arbitrary-resolution and fine-grained monocular depth estimation. The method uses a Vision Transformer encoder to extract multi-scale features, which are queried at continuous 2D coordinates via bilinear interpolation and fused hierarchically through a…

Hao Yu, Haotong Lin, Jiawei Wang, Jiaxin Li, et al.
Published
Jan 2026
Citations
12
Code
1.1K stars
Read summary Original
100

arXiv.org

LTX-2: Efficient Joint Audio-Visual Foundation Model

LTX-2 is an open-source text-to-audio+video (T2AV) foundation model that jointly generates synchronized video and audio from text. It uses an asymmetric dual-stream transformer with a 14B-parameter video stream and a 5B-parameter audio stream, connected via bidirectional cross-attention layers with temporal positional embeddings and cross-modality AdaLN.…

Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, et al.
Published
Jan 2026
Citations
135
Code
8.5K stars
Read summary Original