The Year in AI Papers
The year/Independent research

Paper 2604.14268

HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

Team HY-World, Chenjie Cao, Xuhui Zuo, Zhenwei Wang, Yisu Zhang, Junta Wu, Zhenyang Liu, Yuning Gong, Yang Liu, Bo Yuan, Chao Zhang, Coopers Li, et al.

Published
Apr 2026
Research lab
Independent
Citations
19
GitHub
2.5K stars

01 In brief

Summary

HY-World 2.0 is an open-source multi-modal world model framework that unifies 3D world generation and reconstruction.

For sparse inputs (text or single-view images), it generates navigable 3D Gaussian Splatting (3DGS) scenes via a four-stage pipeline: panorama generation (HY-Pano 2.0), trajectory planning (WorldNav), world expansion (WorldStereo 2.0), and world composition (WorldMirror 2.0).

For dense inputs (multi-view images or videos), it performs world reconstruction using WorldMirror 2.0, which features normalized position encoding, explicit normal supervision, and depth mask prediction.

WorldStereo 2.0 operates in a keyframe latent space with global-geometric and spatial-stereo memories, and is distilled for fast inference.

WorldNav plans five trajectory types (regular, surrounding, reconstruction-aware, wandering, aerial) using scene parsing.

World composition aligns depths and optimizes 3DGS with MaskGaussian and tailored losses.

Experiments show state-of-the-art performance on benchmarks, comparable to closed-source Marble, with end-to-end generation in ~10 minutes on H20 GPUs.

All models, code, and technical details are released.

02 From the paper

Abstract

We introduce HY-World 2.0, a multi-modal world model framework that advances our prior project HY-World 1.0. HY-World 2.0 accommodates diverse input modalities, including text prompts, single-view images, multi-view images, and videos, and produces 3D world representations. With text or single-view image inputs, the model performs world generation, synthesizing high-fidelity, navigable 3D Gaussian Splatting (3DGS) scenes. This is achieved through a four-stage method: a) Panorama Generation with HY-Pano 2.0, b) Trajectory Planning with WorldNav, c) World Expansion with WorldStereo 2.0, and d) World Composition with WorldMirror 2.0. Specifically, we introduce key innovations to enhance panorama fidelity, enable 3D scene understanding and planning, and upgrade WorldStereo, our keyframe-based view generation model with consistent memory. We also upgrade WorldMirror, a feed-forward model for universal 3D prediction, by refining model architecture and learning strategy, enabling world reconstruction from multi-view images or videos. Also, we introduce WorldLens, a high-performance 3DGS rendering platform featuring a flexible engine-agnostic architecture, automatic IBL lighting, efficient collision detection, and training-rendering co-design, enabling interactive exploration of 3D worlds with character support. Extensive experiments demonstrate that HY-World 2.0 achieves state-of-the-art performance on several benchmarks among open-source approaches, delivering results comparable to the closed-source model Marble. We release all model weights, code, and technical details to facilitate reproducibility and support further research on 3D world models.