Paper 2603.25040
Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 14
- GitHub
- Not linked
01 In brief
Summary
Intern-S1-Pro, developed by Shanghai AI Laboratory, is the first one-trillion-parameter scientific multimodal foundation model, built on the SAGE framework.
It scales from Intern-S1 via expert expansion with Grouped Routing to ensure load balance and training stability, and uses a Straight-Through Estimator for efficient router updates.
The model integrates a Native Vision Transformer, Fourier Position Encoding (FoPE), and an enhanced time-series encoder supporting domains like physiology and bioacoustics.
Pre-training on 6T tokens includes a PDF-based caption pipeline generating ~270B tokens of dense scientific captions, plus strategies to resolve scientific-general data conflicts.
Post-training employs stable mixed-precision RL with FP8 quantization, rollout router replay, and importance sampling.
Evaluations show Intern-S1-Pro outperforms proprietary models on scientific benchmarks (e.g., SciReasoner 55.5 vs.
Gemini-3-Pro 14.7) and matches or exceeds open-source models on general tasks (AIME-2025 93.1, MMLU-Pro 86.6).
A case study shows it surpasses a specialized biology model on the same data, supporting the 'Specializable Generalist' concept.
Infrastructure co-design with XTuner and LMDeploy enables efficient trillion-scale training with only ~20% efficiency loss vs.
Intern-S1.
The model is available on Hugging Face.
- Intern-S1-Pro achieves state-of-the-art results on scientific benchmarks, including SmolInstruct (74.8), MatBench (72.8), and Biology-Instruction (52.5).
- It demonstrates strong agent capabilities with scores of 77.4 on GAIA (Text-Only), 80.9 on Tau2-Bench, and 93.6 on ScreenSpot V2.
- The model outperforms specialized models on biological tasks, e.g., Protein-Fluorescence 78.14 vs.
2.57, and Protein-FunctionEC 72.70 vs.
19.79.
- Training efficiency is maintained via Grouped Routing and mixed-precision RL, with FP8 matching BF16 performance.
- The caption…
02 From the paper
Abstract
We introduce Intern-S1-Pro, the first one-trillion-parameter scientific multimodal foundation model. Scaling to this unprecedented size, the model delivers a comprehensive enhancement across both general and scientific domains. Beyond stronger reasoning and image-text understanding capabilities, its intelligence is augmented with advanced agent capabilities. Simultaneously, its scientific expertise has been vastly expanded to master over 100 specialized tasks across critical science fields, including chemistry, materials, life sciences, and earth sciences. Achieving this massive scale is made possible by the robust infrastructure support of XTuner and LMDeploy, which facilitates highly efficient Reinforcement Learning (RL) training at the 1-trillion parameter level while ensuring strict precision consistency between training and inference. By seamlessly integrating these advancements, Intern-S1-Pro further fortifies the fusion of general and specialized intelligence, working as a Specializable Generalist, demonstrating its position in the top tier of open-source models for general capabilities, while outperforming proprietary models in the depth of specialized scientific tasks.