Paper 2605.12500
SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
- Published
- May 2026
- Research lab
- Independent
- Citations
- 12
- GitHub
- 4.5K stars
01 In brief
Summary
SenseNova-U1 is a native unified multimodal model built on the NEO-unify architecture, designed to overcome the traditional divide between understanding and generation.
It operates directly on raw pixels and text, eliminating the need for pretrained vision encoders (VEs) and variational autoencoders (VAEs).
The model uses a near-lossless visual interface with lightweight patch encoding/decoding and a native Mixture-of-Transformers (MoT) backbone, jointly optimizing autoregressive text loss and pixel-space flow matching.
Two variants are introduced: SenseNova-U1-8B-MoT (dense 8B) and SenseNova-U1-A3B-MoT (30B total, 3B active).
Training proceeds through progressive stages, including understanding warmup, generation pre-training, unified mid-training, SFT, and RL post-training, with a disaggregated inference architecture using LightLLM and LightX2V.
The model rivals top-tier understanding-only VLMs on benchmarks like MMMU, MathVista, and spatial intelligence tasks, while achieving strong performance in image generation (e.g., GenEval 0.91, DPG-Bench top scores), text-centric generation, infographic synthesis, and image editing.
It also shows promising capabilities in vision-language-action and world modeling, indicating a path toward unified multimodal intelligence.
02 From the paper
Abstract
Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.