The Year in AI Papers
The year/Independent research

Paper 2603.25716

Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models

Kaijin Chen, Dingkang Liang, Xin Zhou, Yikang Ding, Xiaoqiang Liu, Pengfei Wan, Xiang Bai

Published
Mar 2026
Research lab
Independent
Citations
11
GitHub
269 stars

01 In brief

Summary

The paper introduces Hybrid Memory, a paradigm for video world models that requires maintaining static background consistency while tracking dynamic subjects during out-of-view intervals.

The authors construct HM-World, a large-scale dataset of 59K high-fidelity clips with 17 scenes, 49 subjects, and designed exit-entry events, and propose HyDRA, a memory architecture using a 3D-convolution-based tokenizer and spatiotemporal relevance-driven retrieval to preserve subject identity and motion.

Experiments show HyDRA outperforms baselines and state-of-the-art methods on metrics like PSNR, SSIM, and Dynamic Subject Consistency, with ablations validating the design choices.

Limitations include degraded performance in complex scenes with multiple subjects or severe occlusions, and future work aims to address these challenges and scale to real-world environments.

The code is publicly available at https://github.com/H-EmbodVis/HyDRA.

The work addresses a gap in existing memory mechanisms that treat environments as static, and the authors hope to inspire further research in hybrid memory for video world models.

The dataset and method are intended to provide a foundation for advancing spatiotemporal consistency in dynamic environments.

The paper includes a detailed description of the dataset construction, the HyDRA architecture, and extensive experimental validation, including comparisons with commercial models like WorldPlay.

The authors also provide supplementary material with additional qualitative results and ablations.

The main contributions are the identification of limitations in static-centric memory, the introduction of HM-World, and the proposal of HyDRA.

The work is supported by Kuaishou Technology and Huazhong University of Science and Technology.

The paper is available on arXiv and the project page is provided.

The authors…

02 From the paper

Abstract

Video world models have shown immense potential in simulating the physical world, yet existing memory mechanisms primarily treat environments as static canvases. When dynamic subjects hide out of sight and later re-emerge, current methods often struggle, leading to frozen, distorted, or vanishing subjects. To address this, we introduce Hybrid Memory, a novel paradigm requiring models to simultaneously act as precise archivists for static backgrounds and vigilant trackers for dynamic subjects, ensuring motion continuity during out-of-view intervals. To facilitate research in this direction, we construct HM-World, the first large-scale video dataset dedicated to hybrid memory. It features 59K high-fidelity clips with decoupled camera and subject trajectories, encompassing 17 diverse scenes, 49 distinct subjects, and meticulously designed exit-entry events to rigorously evaluate hybrid coherence. Furthermore, we propose HyDRA, a specialized memory architecture that compresses memory into tokens and utilizes a spatiotemporal relevance-driven retrieval mechanism. By selectively attending to relevant motion cues, HyDRA effectively preserves the identity and motion of hidden subjects. Extensive experiments on HM-World demonstrate that our method significantly outperforms state-of-the-art approaches in both dynamic subject consistency and overall generation quality. Code is publicly available at https://github.com/H-EmbodVis/HyDRA.