Paper 2605.28820
From Pixels to Words -- Towards Native One-Vision Models at Scale
- Published
- May 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 881 stars
01 In brief
Summary
NEO-ov is a native vision-language foundation model that unifies single-image, multi-image, video understanding, and spatial intelligence in a single monolithic backbone, eliminating external visual encoders, adapters, and post-hoc fusion.
It uses a unified serialization scheme with spatiotemporal attention (THW-decoupled) and Native-RoPE to enable end-to-end pixel-word and pixel-pixel interactions.
Training proceeds in three stages: pre-training on 20M image-text pairs, mid-training on 60M multimodal samples (including videos up to 128 frames), and supervised fine-tuning on 6M high-quality samples.
NEO-ov outperforms prior native VLMs and approaches or matches modular counterparts like Qwen3-VL and InternVL3.5 on benchmarks such as MMMU, HallusionBench, VideoMME, and spatial intelligence tasks (e.g., VSI-Bench, MMSI).
Ablations show the Pre-Buffer mechanism outperforms conventional visual encoders, and native architectures benefit more from spatial intelligence supervision.
Limitations include a remaining gap to top-tier modular systems on some tasks and underexplored OCR-intensive domains.
The work suggests native one-vision architectures are feasible and competitive at scale.
02 From the paper
Abstract
Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular framework that inevitably fragments pixel-level signals across frames and scatters early pixel-word interactions. In parallel, native VLMs, despite impressive performance on single images, remain largely unexplored in multi-image, video understanding, and spatial intelligence. Hence, we introduce NEO-ov, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end, without any external encoders, auxiliary adapters, or post-hoc fusion. By eliminating module boundaries entirely, NEO-ov enables fine-grained and unified spatiotemporal modeling to emerge natively inside the model. Notably, NEO-ov largely narrows the gap to modular counterparts while excelling at fine-grained visual perception, validating that native "one-vision" architectures are not only feasible but competitive at scale. Beyond empirical performance, we unveil systematic architectural analyses and detailed training recipes to facilitate subsequent native multimodal modeling. Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.