The Year in AI Papers
The year/Independent research

Paper 2602.12705

MedXIAOHE: A Comprehensive Recipe for Building Medical MLLMs

Baorong Shi, Bo Cui, Boyuan Jiang, Deli Yu, Fang Qian, Haihua Yang, Huichao Wang, Jiale Chen, Jianfei Pan, Jieqiong Cao, Jinghao Lin, Kai Wu, et al.

Published
Feb 2026
Research lab
Independent
Citations
3
GitHub
Not linked

01 In brief

Summary

MedXIAOHE is a medical vision-language foundation model from ByteDance that achieves state-of-the-art performance across 30+ medical benchmarks, surpassing leading closed-source systems like GPT-5.2 Thinking and Gemini 3.0 Pro.

The model uses a Seed-ViT vision encoder and a large language model, trained via a three-stage pipeline.

Continual pretraining uses an entity-aware approach with a Medical Entity Tree (MET) of 1.4 million entities to improve knowledge coverage and reduce long-tail gaps, trained on 640B tokens.

Mid-training enhances reasoning through knowledge-graph-guided QA synthesis, multi-expert reject sampling, structured chain-of-thought, and agentic reasoning with tools like medical search and image zoom.

Post-training includes SFT and reinforcement learning with a multi-layered hybrid reward system and an RFT-enhanced iterative curriculum.

The model excels in visual diagnosis, medical imaging, diagnosis, medical text, and report generation, though it shows lower performance on IU-Xray.

The paper also introduces a Unified Med-VLM Benchmark with standardized protocols and in-house benchmarks for VQA, OCR, and captioning to address evaluation fragmentation.

02 From the paper

Abstract

We present MedXIAOHE, a medical vision-language foundation model designed to advance general-purpose medical understanding and reasoning in real-world clinical applications. MedXIAOHE achieves state-of-the-art performance across diverse medical benchmarks and surpasses leading closed-source multimodal systems on multiple capabilities. To achieve this, we propose an entity-aware continual pretraining framework that organizes heterogeneous medical corpora to broaden knowledge coverage and reduce long-tail gaps (e.g., rare diseases). For medical expert-level reasoning and interaction, MedXIAOHE incorporates diverse medical reasoning patterns via reinforcement learning and tool-augmented agentic training, enabling multi-step diagnostic reasoning with verifiable decision traces. To improve reliability in real-world use, MedXIAOHE integrates user-preference rubrics, evidence-grounded reasoning, and low-hallucination long-form report generation, with improved adherence to medical instructions. We release this report to document our practical design choices, scaling insights, and evaluation framework, hoping to inspire further research.