The Year in AI Papers
The year/NVIDIA

Paper 2605.27365

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei, Yangzhou Liu, Zhiqi Li, Yunze Man, Guo Chen, Andrew Tao, Guilin Liu, Jan Kautz, Lei Zhang, et al.

Published
May 2026
Research lab
NVIDIA
Citations
7
GitHub
Not linked

01 In brief

Summary

LocateAnything is a unified vision-language model for visual grounding and detection that introduces Parallel Box Decoding (PBD).

Unlike standard next-token prediction (NTP) which serializes bounding box coordinates into 1D token streams, PBD treats each bounding box as an atomic unit, predicting all its coordinates in a single forward pass.

This box-aligned approach preserves intra-box geometric coherence, improves localization accuracy, and significantly increases decoding throughput.

The model is trained with a dual-formulation strategy combining NTP and block-wise multi-token prediction (MTP) losses, using a specialized attention mask to isolate streams while enabling bidirectional intra-block attention.

Three inference modes are offered: Fast (MTP), Slow (NTP), and Hybrid (MTP with NTP fallback for unreliable blocks).

The model is trained on LocateAnything-Data, a large-scale dataset with 138M queries and 785M bounding boxes across six domains.

Experiments show LocateAnything achieves state-of-the-art results on benchmarks like LVIS, COCO, Dense200, VisDrone, ScreenSpot-Pro, DocLayNet, and M6Doc, with up to 2.5× higher decoding throughput than prior methods (e.g., 12.7 BPS vs.

5.0 BPS for Rex-Omni).

Ablations confirm PBD's benefits over structure-agnostic MTP and NTP baselines, and the hybrid mode balances speed and accuracy effectively.

02 From the paper

Abstract

Vision-language models (VLMs) commonly formulate visual grounding and detection as a coordinate-token generation problem, serializing each 2D box into multiple 1D tokens that are learned and decoded largely independently. This token-by-token decoding mismatches the coupled structure of box geometry and creates a practical inference bottleneck due to strictly sequential generation. We introduce LocateAnything, a unified generative grounding and detection framework based on Parallel Box Decoding (PBD). By decoding geometric elements such as bounding boxes and points as atomic units in a single step, LocateAnything preserves intra-box geometric coherence and unlocks substantial parallelism. We show that PBD improves both decoding throughput and localization accuracy. We further develop a scalable data engine and curate LocateAnything-Data, a large-scale dataset with more than 138 million training samples, substantially increasing data diversity for high-precision localization. Extensive evaluations show that LocateAnything advances the speed-accuracy frontier, achieving significantly higher decoding throughput while improving high-IoU localization quality across diverse benchmarks. The results highlight the complementary benefits of Parallel Box Decoding and large-scale training data in enabling efficient and precise unified visual grounding and detection.