The Year in AI Papers
The year/Topics/Vision-language models

Research collection

Vision-language models

Multimodal LLM architecture and training: unified understanding and generation, visual representation alignment, and general-purpose VLM technical reports.

Papers
65
Research labs
5
Official code
52

1–50 of 65 papers in this collection

01

Moonshot AI

PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models

PerceptionBench is a benchmark introduced by Moonshot AI to evaluate atomic visual perception in Multimodal Large Language Models (MLLMs). It addresses limitations of existing benchmarks that conflate perception with reasoning or knowledge. The benchmark was constructed bottom-up: failures of frontier MLLMs on 42 existing benchmarks were attributed to…

Zichao Lin, Yifeng Xie, Bowen Qu, Haiming Wang, et al.
Published
Jul 2026
Citations
0
Code
170 stars
Read summary Original
02

Independent research

Scalable Visual Pretraining for Language Intelligence

This paper introduces Visual Pretraining (VP), a framework that trains foundation models directly on raw document images without text extraction or image-text pairing, using a next-visual-latent prediction objective. VP consistently outperforms text-only pretraining (TP) on scientific reasoning benchmarks across multiple backbones (Qwen3.5, Qwen3, Llama3.2…

Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
03

Independent research

Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models

Wan-Streamer is a native-streaming, end-to-end interactive foundation model from Alibaba Group designed for real-time, low-latency, full-duplex audio-visual interaction. It models language, audio, and video as both input and output within a single Transformer, using block-causal attention for incremental streaming. Unlike cascaded systems, it does not rely…

Lianghua Huang, Zhi-Fan Wu, Wei Wang, Yupeng Shi, et al.
Published
Jun 2026
Citations
4
Code
Not linked
Read summary Original
04

Independent research

Kwai Keye-VL-2.0 Technical Report

The report introduces Kwai Keye-VL-2.0-30B-A3B, an open-source Mixture-of-Experts (MoE) multimodal foundation model with 30B total parameters and 3B active, designed for long-video understanding and agentic intelligence. It is the first to adapt DeepSeek Sparse Attention (DSA) to GQA-based architectures, enabling lossless 256K context processing. The model…

Kwai Keye Team, Bin Wen, Changyi Liu, Chengru Song, et al.
Published
Jun 2026
Citations
0
Code
809 stars
Read summary Original
05

Independent research

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

Multimodal Large Language Models (MLLMs) degrade under real-world visual corruptions. Existing robustness methods are limited: black-box feature alignment lacks interpretability, and text-based reasoning cannot restore pixel-level details. This paper proposes Robust-U1, a framework that equips MLLMs with explicit visual self-recovery capability. It uses a…

Jiaqi Tang, Jianmin Chen, Youyang Zhai, Wei Wei, et al.
Published
Jun 2026
Citations
1
Code
363 stars
Read summary Original
06

Independent research

Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models

The paper introduces Imaginative Perception Tokens (IPTs), intermediate visual representations that externalize what a VLM would perceive under an alternative spatial configuration, to improve spatial reasoning. Three tasks requiring imaginative perception are formulated: Perspective Taking (PET), Path Tracing (PT), and Multiview Counting (MVC), with…

Mahtab Bigverdi, Linjie Li, Weikai Huang, Yiming Liu, et al.
Published
Jun 2026
Citations
0
Code
102 stars
Read summary Original
07

arXiv.org

Representation Forcing for Bottleneck-Free Unified Multimodal Models

The paper introduces Representation Forcing (RF), a technique for unified multimodal models (UMMs) that eliminates the need for a separately pretrained VAE in image generation. RF trains the decoder to autoregressively predict discrete visual representation tokens, derived from the model's own understanding encoder via online vector quantization, before…

Yuqing Wang, Zhijie Lin, Ceyuan Yang, Yang Zhao, et al.
Published
May 2026
Citations
3
Code
Not linked
Read summary Original
08

arXiv.org

From Pixels to Words -- Towards Native One-Vision Models at Scale

NEO-ov is a native vision-language foundation model that unifies single-image, multi-image, video understanding, and spatial intelligence in a single monolithic backbone, eliminating external visual encoders, adapters, and post-hoc fusion. It uses a unified serialization scheme with spatiotemporal attention (THW-decoupled) and Native-RoPE to enable…

Haiwen Diao, Jiahao Wang, Penghao Wu, Yuhao Dong, et al.
Published
May 2026
Citations
1
Code
881 stars
Read summary Original
09

NVIDIA

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

LocateAnything is a unified vision-language model for visual grounding and detection that introduces Parallel Box Decoding (PBD). Unlike standard next-token prediction (NTP) which serializes bounding box coordinates into 1D token streams, PBD treats each bounding box as an atomic unit, predicting all its coordinates in a single forward pass. This…

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, et al.
Published
May 2026
Citations
7
Code
Not linked
Read summary Original
10

arXiv.org

IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools

IndusAgent is a tool-augmented agentic framework for open-vocabulary industrial anomaly detection (IAD). It addresses limitations of multimodal large language models (MLLMs), such as domain-misaligned reasoning and structural hallucinations, by combining supervised fine-tuning (SFT) with reinforcement learning (RL). The framework constructs Indus-CoT, a…

Rongbin Tan, Fangfang Lin, Zhenlong Yuan, Min Qiu, et al.
Published
May 2026
Citations
0
Code
Not linked
Read summary Original
11

arXiv.org

Lance: Unified Multimodal Modeling by Multi-Task Synergy

Lance is a lightweight native unified multimodal model from ByteDance that supports understanding, generation, and editing for both images and videos. It uses a dual-stream mixture-of-experts architecture on shared interleaved multimodal sequences, combining autoregressive language modeling for understanding with flow matching for generation. A…

Fengyi Fu, Mengqi Huang, Shaojin Wu, Yunsheng Jiang, et al.
Published
May 2026
Citations
3
Code
1.3K stars
Read summary Original
12

arXiv.org

SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

SenseNova-U1 is a native unified multimodal model built on the NEO-unify architecture, designed to overcome the traditional divide between understanding and generation. It operates directly on raw pixels and text, eliminating the need for pretrained vision encoders (VEs) and variational autoencoders (VAEs). The model uses a near-lossless visual interface…

Haiwen Diao, Penghao Wu, Hanming Deng, Jiahao Wang, et al.
Published
May 2026
Citations
12
Code
4.5K stars
Read summary Original
13

arXiv.org

MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction

MiniCPM-o 4.5 is a 9B-parameter open-source multimodal large language model (MLLM) designed for real-time full-duplex omni-modal interaction, enabling simultaneous perception and response. It introduces Omni-Flow, a unified streaming framework that aligns multimodal inputs and outputs along a shared temporal axis, converting turn-based interaction into a…

Junbo Cui, Bokai Xu, Chongyi Wang, Tianyu Yu, et al.
Published
Apr 2026
Citations
16
Code
26K stars
Read summary Original
14

Z.ai / GLM

GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents

GLM-5V-Turbo is a native multimodal foundation model for agentic tasks, integrating perception, reasoning, planning, and execution. It introduces CogViT, a vision encoder trained via distillation and contrastive learning, and Multimodal Multi-Token Prediction (MMTP) using a shared <|image|> token for efficiency. The model undergoes joint RL over 30+ task…

GLM-V Team, :, Wenyi Hong, Xiaotao Gu, et al.
Published
Apr 2026
Citations
11
Code
Not linked
Read summary Original
15

Meta AI

Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

Tuna-2 is a native unified multimodal model that performs visual understanding and generation directly from raw pixel embeddings, eliminating pretrained vision encoders such as VAEs and representation encoders. It uses simple patch embedding layers to encode images and a single transformer decoder for joint processing, with pixel-space flow matching for…

Zhiheng Liu, Weiming Ren, Xiaoke Huang, Shoufa Chen, et al.
Published
Apr 2026
Citations
10
Code
744 stars
Read summary Original
16

Independent research

EXAONE 4.5 Technical Report

EXAONE 4.5 is LG AI Research's first open-weight vision-language model, integrating a custom 1.2B-parameter vision encoder with the EXAONE 4.0 32B language backbone. It supports six languages and a 256K token context, achieved via context extension during SFT. The model uses hybrid attention, GQA, 2D RoPE, and MTP for efficiency. Pre-training includes two…

Eunbi Choi, Kibong Choi, Sehyun Chun, Seokhee Hong, et al.
Published
Apr 2026
Citations
0
Code
45 stars
Read summary Original
17

arXiv.org

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

Tencent's HY-Embodied-0.5 is a family of vision-language foundation models designed for real-world embodied agents, bridging the gap between general VLMs and physical-world tasks. The suite includes an efficient 2B-activated-parameter model (MoT-2B) for edge deployment and a powerful 32B-activated-parameter model (MoE-A32B) for complex reasoning. Key…

Tencent Robotics X, HY Vision Team, :, Xumin Yu, et al.
Published
Apr 2026
Citations
12
Code
843 stars
Read summary Original
18

arXiv.org

Steerable Visual Representations

The paper introduces Steerable Visual Representations (SteerViT), a method to make pretrained Vision Transformers (ViTs) steerable by natural language. SteerViT injects text into the visual encoder via lightweight gated cross-attention layers (early fusion), unlike late-fusion models like CLIP. It is trained on a referential segmentation pretext task using…

Jona Ruthardt, Manu Gaur, Deva Ramanan, Makarand Tapaswi, et al.
Published
Apr 2026
Citations
2
Code
119 stars
Read summary Original
19

arXiv.org

LongCat-Next: Lexicalizing Modalities as Discrete Tokens

LongCat-Next, developed by Meituan's LongCat team, introduces the Discrete Native Autoregressive (DiNA) paradigm, which unifies text, vision, and audio into a shared discrete token space, enabling a single autoregressive model to handle all modalities. A key innovation is the Discrete Native Any-resolution Visual Transformer (dNaViT), which uses…

Meituan LongCat Team, Bin Xiao, Chao Wang, Chengjiang Li, et al.
Published
Mar 2026
Citations
18
Code
467 stars
Read summary Original
20

arXiv.org

Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding

VEGA-3D is a plug-and-play framework that repurposes pre-trained video generation models as Latent World Simulators to provide implicit 3D priors for Multimodal Large Language Models (MLLMs), addressing their spatial blindness. The method extracts spatiotemporal features from intermediate noise levels of a frozen video diffusion model (e.g., Wan2.1-T2V)…

Xianjin Wu, Dingkang Liang, Tianrui Feng, Kui Xia, et al.
Published
Mar 2026
Citations
10
Code
421 stars
Read summary Original
21

arXiv.org

Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs

The paper presents AwaRes, a framework for efficient vision-language model (VLM) inference that processes a low-resolution global image and uses tool-calling to retrieve only the high-resolution crops needed for a query. AwaRes trains a coupled-decision policy (CDP) that jointly decides whether to escalate resolution and which crops to request. Supervision…

Nimrod Shabtay, Moshe Kimhi, Artem Spector, Sivan Haray, et al.
Published
Mar 2026
Citations
1
Code
11 stars
Read summary Original
22

arXiv.org

Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders

Penguin-VL, developed by Tencent AI Lab, introduces compact 2B and 8B vision-language models that challenge the reliance on contrastive pretraining (e.g., CLIP/SigLIP) for vision encoders. The authors argue that contrastive learning suppresses fine-grained visual cues needed for reasoning. Instead, they propose Penguin-Encoder, initialized from a text-only…

Boqiang Zhang, Lei Ke, Ruihan Yang, Qi Gao, et al.
Published
Mar 2026
Citations
4
Code
206 stars
Read summary Original
23

Meta AI

Beyond Language Modeling: An Exploration of Multimodal Pretraining

This paper presents controlled, from-scratch experiments to clarify the design space of unified multimodal pretraining, using the Transfusion framework (next-token prediction for language, diffusion for vision) on text, video, image-text pairs, and action-conditioned video. Key findings: (1) Representation Autoencoders (RAE), e.g., SigLIP 2, provide a…

Shengbang Tong, David Fan, John Nguyen, Ellis Brown, et al.
Published
Mar 2026
Citations
21
Code
Not linked
Read summary Original
24

arXiv.org

UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?

UniG2U-Bench is a new benchmark for evaluating whether unified multimodal models (UMMs) benefit from generation when performing understanding tasks. It includes 3,000 samples across 7 categories and 30 subtasks, and evaluates over 30 models, including base VLMs, unified models, and agentic models. The study finds that unified models generally underperform…

Zimo Wen, Boxiu Li, Wanbo Zhang, Junxiang Lei, et al.
Published
Mar 2026
Citations
3
Code
Not linked
Read summary Original
25

arXiv.org

MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs

MedXIAOHE is a medical vision-language foundation model from ByteDance that achieves state-of-the-art performance across 30+ medical benchmarks, surpassing leading closed-source systems like GPT-5.2 Thinking and Gemini 3.0 Pro. The model uses a Seed-ViT vision encoder and a large language model, trained via a three-stage pipeline. Continual pretraining…

Baorong Shi, Bo Cui, Boyuan Jiang, Deli Yu, et al.
Published
Feb 2026
Citations
3
Code
Not linked
Read summary Original
26

arXiv.org

Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception

The paper introduces Region-to-Image Distillation (R2I), a method that internalizes the benefits of inference-time zooming into a single forward pass of a multimodal large language model (MLLM). R2I synthesizes fine-grained VQA data by zooming into micro-cropped regions, using strong teacher models to generate high-consensus question-answer pairs, and then…

Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, et al.
Published
Feb 2026
Citations
23
Code
184 stars
Read summary Original
27

arXiv.org

GENIUS: Generative Fluid Intelligence Evaluation Suite

The paper introduces GENIUS, the first benchmark for evaluating Generative Fluid Intelligence (GFI) in Unified Multimodal Models (UMMs), distinguishing it from Crystallized Intelligence (CI). GFI is formalized into three primitives: Inducing Implicit Patterns, Executing Ad-hoc Constraints, and Adapting to Contextual Knowledge. The benchmark comprises 510…

Ruichuan An, Sihan Yang, Ziyu Guo, Wei Dai, et al.
Published
Feb 2026
Citations
12
Code
43 stars
Read summary Original
28

arXiv.org

P1-VL: Bridging Visual Perception and Scientific Reasoning in Physics Olympiads

The P1-VL technical report introduces a family of open-source vision-language models (VLMs) designed for advanced scientific reasoning, specifically targeting physics Olympiad problems. The models are trained exclusively via reinforcement learning (RL) using a curriculum that progressively increases problem difficulty and expands exploration space,…

Yun Luo, Futing Wang, Qianjia Cheng, Fangchen Yu, et al.
Published
Feb 2026
Citations
5
Code
15 stars
Read summary Original
29

arXiv.org

ERNIE 5.0 Technical Report

ERNIE 5.0 is a natively autoregressive foundation model from Baidu for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, using an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. A…

Haifeng Wang, Hua Wu, Tian Wu, Yu Sun, et al.
Published
Feb 2026
Citations
8
Code
Not linked
Read summary Original
30

arXiv.org

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models

The paper addresses the Modality Gap in multimodal contrastive learning, where embeddings of different modalities for the same semantics occupy offset regions. Prior methods rely on isotropic assumptions, which are flawed. The authors propose the Fixed-frame Modality Gap Theory, decomposing the gap into stable biases (PMB, POB) and anisotropic residuals,…

Xiaomin Yu, Yi Xin, Yuhui Zhang, Wenjie Zhang, et al.
Published
Feb 2026
Citations
9
Code
76 stars
Read summary Original
31

arXiv.org

CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding

This paper presents the first systematic study on the effectiveness of Vision Language Models (MLLMs) for code understanding by representing source code as images. The authors evaluate seven MLLMs across four tasks (code summarization, completion, clone detection, and question answering) with compression ratios from 1x to 8x and rendering strategies…

Yuling Shi, Chaoxiang Xie, Zhensu Sun, Yeheng Chen, et al.
Published
Feb 2026
Citations
16
Code
30 stars
Read summary Original
32

Moonshot AI

WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models

WorldVQA is a benchmark introduced to evaluate the atomic visual world knowledge of Multimodal Large Language Models (MLLMs), decoupling visual knowledge retrieval from reasoning. It comprises 3,500 VQA pairs across nine semantic categories, from common head-class entities to long-tail rarities. The benchmark follows four design principles: atomic…

Runjie Zhou, Youbo Shao, Haoyu Lu, Bowei Xing, et al.
Published
Jan 2026
Citations
4
Code
121 stars
Read summary Original
33

arXiv.org

Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders

This paper investigates scaling Representation Autoencoders (RAEs) for large-scale text-to-image (T2I) generation. The authors train RAE decoders on a frozen SigLIP-2 encoder using web, synthetic, and text-rendering data, finding that data composition is crucial for text reconstruction. They show that dimension-dependent noise scheduling remains essential,…

Shengbang Tong, Boyang Zheng, Ziteng Wang, Bingda Tang, et al.
Published
Jan 2026
Citations
40
Code
255 stars
Read summary Original
34

arXiv.org

Urban Socio-Semantic Segmentation with Vision-Language Reasoning

The paper introduces urban socio-semantic segmentation, targeting entities defined by social attributes (e.g., schools, parks) rather than physical ones. The authors present SocioSeg, a benchmark with over 13,000 samples, organizing labels into three hierarchical tasks: socio-name, socio-class, and socio-function. A key innovation is representing…

Yu Wang, Yi Wang, Rui Dai, Yujie Wang, et al.
Published
Jan 2026
Citations
2
Code
175 stars
Read summary Original
35

arXiv.org

STEP3-VL-10B Technical Report

Step3-VL-10B is a 10B-parameter open-source multimodal foundation model that rivals or surpasses models 10-20x larger, such as GLM-4.6V-106B and Qwen3-VL-235B, and proprietary systems like Gemini 2.5 Pro. It achieves 92.2% on MMBench, 80.11% on MMMU, 94.43% on AIME2025, and 75.95% on MathVision. The model uses a unified pre-training strategy on 1.2T…

Ailin Huang, Chengyuan Yao, Chunrui Han, Fanqi Wan, et al.
Published
Jan 2026
Citations
26
Code
412 stars
Read summary Original
36

arXiv.org

BabyVision: Visual Reasoning Beyond Language

The paper introduces BABYVISION, a benchmark to evaluate core visual abilities in Multimodal LLMs (MLLMs) that are independent of linguistic knowledge, targeting skills humans develop before language. It contains 388 questions across 22 subtypes in four categories: Fine-grained Discrimination, Visual Tracking, Spatial Perception, and Visual Pattern…

Liang Chen, Weichu Xie, Yiyan Liang, Hongfeng He, et al.
Published
Jan 2026
Citations
24
Code
238 stars
Read summary Original
37

arXiv.org

NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation

NextFlow is a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image tokens. It uses a dual-codebook tokenizer for semantic and pixel-level features, and adopts next-scale prediction for visual generation instead of raster-scan, enabling 1024x1024 image generation in 5 seconds. The model retains next-token prediction…

Huichao Zhang, Liao Qu, Yiheng Liu, Hang Chen, et al.
Published
Jan 2026
Citations
12
Code
331 stars
Read summary Original
38

AAAI Conference on Artificial Intelligence

Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding

Robust-R1 is a framework designed to improve the robustness of Multimodal Large Language Models (MLLMs) against real-world visual degradations. Unlike existing methods that rely on implicit training or adaptation of visual encoders, Robust-R1 explicitly models degradations through structured reasoning chains. The approach consists of three stages:…

Jiaqi Tang, Jianmin Chen, Wei Wei, Xiaogang Xu, et al.
Published
Dec 2025
Citations
8
Code
530 stars
Read summary Original
39

Meta AI

TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models

Tuna is a native unified multimodal model (UMM) that creates a unified continuous visual representation by cascading a VAE encoder with a representation encoder (SigLIP 2). This design avoids the representation format mismatches of decoupled models, improving both understanding and generation. The model uses an LLM decoder (Qwen2.5) for autoregressive text…

Zhiheng Liu, Weiming Ren, Haozhe Liu, Zijian Zhou, et al.
Published
Dec 2025
Citations
31
Code
94 stars
Read summary Original
40

Qwen

Qwen3-VL Technical Report

Qwen3-VL is a state-of-the-art vision-language model family from the Qwen team, released on December 1, 2025. It supports interleaved contexts up to 256K tokens and comes in dense (2B/4B/8B/32B) and MoE (30B-A3B/235B-A22B) variants. Key architectural innovations include interleaved-MRoPE for balanced spatial-temporal encoding, DeepStack for multi-level ViT…

Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, et al.
Published
Nov 2025
Citations
1.8K
Code
20K stars
Read summary Original
41

arXiv.org

MedSAM3: Delving into Segment Anything with Medical Concepts

MedSAM-3 adapts the SAM 3 architecture for medical image and video segmentation, enabling Promptable Concept Segmentation (PCS) via open-vocabulary text prompts. The model is fine-tuned on medical images paired with concise concept phrases (≤3 words) across modalities like X-ray, MRI, Ultrasound, CT, and video. The MedSAM-3 Agent integrates a Multimodal…

Anglin Liu, Rundong Xue, Xu R. Cao, Yifan Shen, et al.
Published
Nov 2025
Citations
28
Code
315 stars
Read summary Original
42

Meta AI

SAM 3: Segment Anything with Concepts

SAM 3 is a unified model for promptable concept segmentation (PCS) in images and videos, accepting noun phrases, image exemplars, or both as prompts to detect, segment, and track all matching instances. It decouples recognition and localization via a presence head, improving detection accuracy. A data engine with human and AI verifiers produced 4M unique…

Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, et al.
Published
Nov 2025
Citations
711
Code
11K stars
Read summary Original
43

arXiv.org

Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data

Uni-MoE-2.0-Omni is a fully open-source omnimodal large model (OLM) built from the dense Qwen2.5-7B LLM, designed for unified understanding, reasoning, and generation across text, image, audio, and video. Its architecture introduces a dynamic-capacity Mixture-of-Experts (MoE) with shared, routed, and null experts for efficient computation and modality…

Yunxin Li, Xinyu Chen, Shenyuan Jiang, Haoyuan Shi, et al.
Published
Nov 2025
Citations
21
Code
1.1K stars
Read summary Original
44

arXiv.org

VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

The paper introduces VCode, a benchmark that reframes multimodal understanding as visual-centric code generation: given an image, a model must produce SVG code that preserves symbolic meaning for downstream reasoning. VCode covers three domains: general commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric perception (CV-Bench),…

Kevin Qinghong Lin, Yuhao Zheng, Hangyu Ran, Dantong Zhu, et al.
Published
Nov 2025
Citations
12
Code
135 stars
Read summary Original
45

arXiv.org

Emu3.5: Native Multimodal Models are World Learners

Emu3.5 is a large-scale multimodal world model from BAAI that natively predicts the next state across vision and language. It is pre-trained end-to-end with a unified next-token prediction objective on over 10 trillion tokens from interleaved vision-language data, primarily derived from internet videos. The model accepts and generates interleaved…

Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, et al.
Published
Oct 2025
Citations
109
Code
1.5K stars
Read summary Original
46

arXiv.org

Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representations

Concerto is a self-supervised learning framework that combines 3D intra-modal self-distillation with 2D-3D cross-modal joint embedding prediction to learn spatial representations from point clouds and images. It outperforms standalone 2D (DINOv2) and 3D (Sonata) self-supervised models by 14.2% and 4.8% respectively in linear probing for 3D scene…

Yujia Zhang, Xiaoyang Wu, Yixing Lao, Chengyao Wang, et al.
Published
Oct 2025
Citations
29
Code
532 stars
Read summary Original
47

NVIDIA

OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM

OmniVinci is an open-source omni-modal LLM from NVIDIA that jointly understands vision, audio, and text. The paper introduces three architectural innovations: OmniAlignNet, which aligns vision and audio embeddings in a shared latent space via contrastive learning; Temporal Embedding Grouping (TEG), which organizes embeddings by timestamps to capture…

Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, et al.
Published
Oct 2025
Citations
50
Code
675 stars
Read summary Original
48

arXiv.org

From Pixels to Words -- Towards Native Vision-Language Primitives at Scale

This paper introduces NEO, a family of native Vision-Language Models (VLMs) that unify vision and language processing in a single monolithic architecture, contrasting with modular VLMs that combine separate visual encoders and language models. NEO is built from first principles using three key primitives: flexible position encoding, Multi-Head Native…

Haiwen Diao, Mingxuan Li, Silei Wu, Linjun Dai, et al.
Published
Oct 2025
Citations
12
Code
881 stars
Read summary Original
49

arXiv.org

Detect Anything via Next Point Prediction

Rex-Omni is a 3B-parameter multimodal large language model (MLLM) that unifies object detection and other visual perception tasks under a next-point prediction framework. It uses quantized relative coordinates represented by 1,000 special tokens, reducing token usage per bounding box to four. Training combines supervised fine-tuning on 22 million samples…

Qing Jiang, Junan Huo, Xingyu Chen, Yuda Xiong, et al.
Published
Oct 2025
Citations
60
Code
1.5K stars
Read summary Original
50

Neural Information Processing Systems

Scaling Language-Centric Omnimodal Representation Learning

This paper introduces LCO-EMB, a language-centric framework for multimodal representation learning that leverages the implicit cross-modal alignment in multimodal large language models (MLLMs). The authors show that text-only contrastive fine-tuning of the language decoder reduces anisotropy and improves kernel-level similarity across image, audio, and…

Chenghao Xiao, Hou Pong Chan, Hao Zhang, Weiwen Xu, et al.
Published
Oct 2025
Citations
15
Code
48 stars
Read summary Original