The Year in AI Papers
The year/Independent research

Paper 2606.23050

Unlimited OCR Works

Youyang Yin, Huanhuan Liu, YY, Qunyi Xie, Chaorun Liu, Shiqi Yang, Shaohua Wang, Zhanlong Liu, Hao Zou, Jinyue Chen, Shu Wei, Jingjing Wu, et al.

Published
Jun 2026
Research lab
Independent
Citations
4
GitHub
22K stars

01 In brief

Summary

Baidu's Unlimited OCR introduces Reference Sliding Window Attention (R-SWA) to enable one-shot long-horizon document parsing.

R-SWA replaces all attention layers in the decoder of DeepSeek OCR, allowing each generated token to attend to all reference tokens (visual and prompt) and a causal sliding window of the previous 128 output tokens.

This maintains a constant KV cache during decoding, reducing memory and computational costs.

Unlimited OCR retains DeepSeek OCR's DeepEncoder, which compresses a 1024x1024 image to 256 tokens, and uses a 3B MoE architecture with 500M activated parameters.

Trained on 2M samples (9:1 single-page to multi-page) for 4,000 steps, it achieves 93.23% on OmniDocBench v1.5 (6.22% higher than DeepSeek OCR) and 93.92% on v1.6, setting a new SOTA.

It also parses up to 40+ pages in a single forward pass with an edit distance below 0.11.

R-SWA provides a 12.7% TPS improvement over DeepSeek OCR on short documents, and up to 35% on longer outputs.

The authors plan to extend R-SWA to ASR and translation tasks.

02 From the paper

Abstract

Recently, end-to-end OCR models, exemplified by DeepSeek OCR, have once again thrust OCR into the spotlight. A widely held view is that employing a large language model (LLM) as the decoder allows the model to leverage the prior distribution of language, leading to improved OCR performance. However, the downside is equally evident: as the output sequence lengthens, the accumulated KV cache drives up memory consumption and progressively slows down generation. This stands in stark contrast to humans, who exhibit no such decline in efficiency during long-horizon copying tasks. In this technical report, we propose Unlimited OCR, a model designed to emulate human parsing working memory. Taking DeepSeek OCR as the baseline, we replace all attention layers in the decoder with our proposed Reference Sliding Window Attention (R-SWA), which reduces attention computation costs while maintaining a constant KV cache throughout the entire decoding process. By combining the high compression rate of DeepSeek OCR's encoder with our constant KV cache design, Unlimited OCR can transcribe dozens of pages of documents in a single forward pass under a standard maximum length of 32K. More importantly, R-SWA is a general-purpose parsing attention mechanism - beyond OCR, it is equally applicable to tasks such as ASR, translation, etc. Codes and model weights are publicly available at http://github.com/baidu/Unlimited-OCR.