The Year in AI Papers
The year/Independent research

Paper 2603.13398

Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

Daxiang Dong, Mingming Zheng, Dong Xu, Chunhua Luo, Bairong Zhuang, Yuxuan Li, Ruoyun He, Haoran Wang, Wenyu Zhang, Wenbo Wang, Yicheng Wang, Xue Xiong, et al.

Published
Mar 2026
Research lab
Independent
Citations
11
GitHub
421 stars

01 In brief

Summary

Qianfan-OCR is a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and understanding, outperforming all end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8).

It introduces Layout-as-Thought, an optional thinking phase triggered by ⟨think⟩ tokens that generates structured layout representations (bounding boxes, element types, reading order) before final output, recovering layout analysis functionality and improving accuracy on complex documents.

The model uses a Qianfan-ViT vision encoder with AnyResolution tiling, Qwen3-4B language backbone, and a two-layer MLP adapter.

Training involves four stages: cross-modal alignment (50B tokens), foundational OCR (2T tokens), domain-specific enhancement (800B tokens), and instruction tuning.

Ablations on Qianfan-VL-8B show the full pipeline achieves 84.39% average accuracy, +13.02% over baseline.

Qianfan-OCR achieves 880 on OCRBench, leads on CCOCR, and excels in document understanding (CharXiv_DQ 94.0, ChartQA 88.1) and key information extraction (87.9 mean, surpassing Gemini-3.1-Pro and Qwen3-VL-235B).

With W8A8 quantization, it reaches 1.024 pages per second on a single A100, comparable to pipeline systems.

Limitations include unexplored Layout-as-Thought on other tasks and deployment constraints on edge devices.

02 From the paper

Abstract

We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.