Paper 2512.01816
Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 1
- GitHub
- 32 stars
01 In brief
Summary
The paper introduces Envision, a benchmark for evaluating text-to-image (T2I) and unified multimodal models (UMMs) on causal, multi-image event generation.
It addresses the limitation of static single-image benchmarks by proposing chained text-to-multi-image generation with 1,000 four-stage prompts across six scientific and humanities domains.
Envision-Score, a weighted metric (40% consistency, 40% physicality, 20% aesthetics), is used to evaluate 15 models (10 specialized T2I, 5 UMMs).
Results show specialized T2I models excel in aesthetics but lack world knowledge, while UMMs perform better in causal coherence but still trail closed-source models like GPT-4o.
The benchmark reveals a universal challenge in spatiotemporal consistency, highlighting the 'Understanding-Generation Paradox' where models fail to interleave understanding and generation for dynamic world modeling.
The authors argue for a paradigm shift from static pattern matching to integrated world models, potentially incorporating video or causally linked image sequences in training.
02 From the paper
Abstract
Current multimodal models aim to transcend the limitations of single-modality representations by unifying understanding and generation, often using text-to-image (T2I) tasks to calibrate semantic consistency. However, their reliance on static, single-image generation in training and evaluation leads to overfitting to static pattern matching and semantic fusion, while fundamentally hindering their ability to model dynamic processes that unfold over time. To address these constraints, we propose Envision-a causal event progression benchmark for chained text-to-multi-image generation. Grounded in world knowledge and structured by spatiotemporal causality, it reorganizes existing evaluation dimensions and includes 1,000 four-stage prompts spanning six scientific and humanities domains. To transition evaluation from single images to sequential frames and assess whether models truly internalize world knowledge while adhering to causal-temporal constraints, we introduce Envision-Score, a holistic metric integrating multi-dimensional consistency, physicality, and aesthetics. Comprehensive evaluation of 15 models (10 specialized T2I models, 5 unified models) uncovers: specialized T2I models demonstrate proficiency in aesthetic rendering yet lack intrinsic world knowledge. Unified multimodal models bridge this gap, consistently outperforming specialized counterparts in causal narrative coherence. However, even these unified architectures remain subordinate to closed-source models and struggle to overcome the core challenge of spatiotemporal consistency. This demonstrates that a focus on causally-isolated single images impedes multi-frame reasoning and generation, promoting static pattern matching over dynamic world modeling-ultimately limiting world knowledge internalization, generation.