Paper 2603.15583
Grounding World Simulation Models in a Real-World Metropolis
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 6
- GitHub
- 624 stars
01 In brief
Summary
The paper introduces Seoul World Model (SWM), a city-scale world model that grounds autoregressive video generation in the real city of Seoul via retrieval-augmented conditioning on street-view images.
SWM fine-tunes Cosmos-Predict2.5-2B on 440k Seoul street-view images, real driving videos, and synthetic urban data.
It addresses three challenges: temporal misalignment via cross-temporal pairing, limited trajectory diversity via a synthetic dataset and an intermittent freeze-frame view interpolation pipeline, and long-horizon error accumulation via a Virtual Lookahead Sink that re-anchors each chunk to a retrieved future image.
Evaluated on Busan and Ann Arbor (unseen cities), SWM outperforms recent world models in visual quality, camera adherence, temporal coherence, and structural fidelity over trajectories of hundreds of meters, while supporting diverse camera movements and text-prompted scenarios.
Ablations confirm the importance of each component, with the Virtual Lookahead Sink providing the best long-horizon stability.
The work demonstrates the feasibility of real-world grounded world simulation at city scale, enabling applications in urban planning and autonomous driving scenario generation.
02 From the paper
Abstract
What if a world simulation model could render not an imagined environment but a city that actually exists? Prior generative world models synthesize visually plausible yet artificial environments by imagining all content. We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul. SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images. However, this design introduces several challenges, including temporal misalignment between retrieved references and the dynamic target scene, limited trajectory diversity and data sparsity from vehicle-mounted captures at sparse intervals. We address these challenges through cross-temporal pairing, a large-scale synthetic dataset enabling diverse camera trajectories, and a view interpolation pipeline that synthesizes coherent training videos from sparse street-view images. We further introduce a Virtual Lookahead Sink to stabilize long-horizon generation by continuously re-grounding each chunk to a retrieved image at a future location. We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor. SWM outperforms existing methods in generating spatially faithful, temporally consistent, long-horizon videos grounded in actual urban environments over trajectories reaching hundreds of meters, while supporting diverse camera movements and text-prompted scenario variations.