The Year in AI Papers
The year/Independent research

Paper 2508.18265

InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, et al.

Published
Aug 2025
Research lab
Independent
Citations
1.2K
GitHub
Not linked

01 In brief

Summary

InternVL3.5 is a new family of open-source multimodal models that improves versatility, reasoning, and efficiency over its predecessor InternVL3.

Key innovations include Cascade Reinforcement Learning (Cascade RL), which combines offline RL (MPO) for stable convergence and online RL (GSPO) for refined alignment, boosting reasoning performance by up to 16.0%.

The Visual Resolution Router (ViR) dynamically adjusts visual token resolution, reducing inference cost by 50% with minimal performance loss, while Decoupled Vision-Language Deployment (DvD) separates vision and language processing across GPUs, achieving up to 4.05x inference speedup.

The models range from 1B to 241B parameters, with the largest, InternVL3.5-241B-A28B, achieving state-of-the-art results among open-source MLLMs and narrowing the gap with commercial models like GPT-5.

InternVL3.5 also supports novel capabilities such as GUI interaction and embodied agency.

All models and code are publicly released.

Extensive evaluations across general, reasoning, text, and agentic benchmarks demonstrate leading performance, with InternVL3.5-241B-A28B scoring 77.7 on MMMU and 82.7 on MathVista, and outperforming GPT-4o on several real-world tasks.

The training pipeline includes pre-training on 250B tokens, SFT on 130B tokens, and Cascade RL, with test-time scaling via deep and parallel thinking further enhancing reasoning.

02 From the paper

Abstract

We introduce InternVL 3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the Cascade Reinforcement Learning (Cascade RL) framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a Visual Resolution Router (ViR) that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled Vision-Language Deployment (DvD) strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0\% gain in overall reasoning performance and a 4.05$\times$ inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks -- narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.