WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models

WorldVQA is a benchmark introduced to evaluate the atomic visual world knowledge of Multimodal Large Language Models (MLLMs), decoupling visual knowledge retrieval from reasoning.

It comprises 3,500 VQA pairs across nine semantic categories, from common head-class entities to long-tail rarities.

The benchmark follows four design principles: atomic isolation, taxonomic diversity, data integrity, and high performance headroom.

Data curation involved expert annotators, distributional balancing, visual deduplication against LAION and Common Crawl, and model-based difficulty stratification.

A dual-verification process with automated auditing and human validation ensures ground-truth fidelity.

Results show that no model surpasses 50% accuracy, with Gemini-3-pro leading at 47.4% and Kimi K2.5 at 46.3%.

Models perform better in Brands and Sports but struggle with Nature and Culture.

Calibration analysis reveals severe overconfidence across models, with Kimi K2.5 achieving the best calibration (ECE: 37.9%).

The benchmark aims to serve as a standard for assessing visual factuality and hallucination rates in frontier models.