The Year in AI Papers
The year/Independent research

Paper 2511.02778

VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

Kevin Qinghong Lin, Yuhao Zheng, Hangyu Ran, Dantong Zhu, Dongxing Mao, Linjie Li, Philip Torr, Alex Jinpeng Wang

Published
Nov 2025
Research lab
Independent
Citations
12
GitHub
135 stars

01 In brief

Summary

The paper introduces VCode, a benchmark that reframes multimodal understanding as visual-centric code generation: given an image, a model must produce SVG code that preserves symbolic meaning for downstream reasoning.

VCode covers three domains: general commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric perception (CV-Bench), totaling 464 image-question pairs.

To evaluate symbolic fidelity, the authors propose CodeVQA, where a policy model answers questions about the rendered SVG; correct answers indicate faithful preservation.

Experiments show that frontier VLMs struggle to generate faithful SVGs, with GPT-5 achieving the best overall CodeVQA score of 46.8, still far below the raw-image upper bound of 61.7.

To address this gap, the authors propose VCoder, an agentic framework with two components: (i) Thinking with Revision, which iteratively analyzes discrepancies and refines SVG code, and (ii) Acting with Visual Tools, which uses detectors and parsers to supply structured cues (objects, shapes, text).

VCoder, built on Claude-4-Opus, improves the overall score from 41.7 to 54.0 (+12.3 points).

Human studies show that both humans and VLMs perform worse on rendered SVGs than on original images, but their consistency suggests the promise of symbolic visual representation.

02 From the paper

Abstract

Code has emerged as a precise and executable medium for reasoning and action in the agent era. Yet, progress has largely focused on language-centric tasks such as program synthesis and debugging, leaving visual-centric coding underexplored. Inspired by how humans reason over sketches, we advocate SVG code as a compact, interpretable, and executable visual representation. We introduce VCode, a benchmark that reframes multimodal understanding as code generation: given an image, a model must produce SVG that preserves symbolic meaning for downstream reasoning. VCode covers three domains - general commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric perception (CV-Bench). To assess symbolic fidelity, we propose CodeVQA, a novel evaluation protocol in which a policy model answers questions over rendered SVGs; correct answers indicate faithful symbolic preservation. Empirically, frontier VLMs struggle to generate faithful SVGs, revealing a persistent gap between language-centric and visual-centric coding. To close this gap, we introduce VCoder, an agentic framework that augments VLMs along two axes: (i) Thinking with Revision, which iteratively analyzes discrepancies and refines SVG code; and (ii) Acting with Visual Tools, where detectors and parsers supply structured cues such as objects, shapes, and text beyond the model's intrinsic capacity. Across benchmarks, frontier VLMs with strong reasoning capabilities score well overall yet remain limited in professional knowledge and 3D reasoning. VCoder delivers a 12.3-point overall gain over the top-performing Claude-4-Opus. Human studies show that both humans and VLMs perform worse on rendered SVGs, their consistency reveals the promise of symbolic visual representation. The benchmark and code are available at https://github.com/CSU-JPG/VCode.