Paper 2604.26752
GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents
- Published
- Apr 2026
- Research lab
- Z.ai / GLM
- Citations
- 11
- GitHub
- Not linked
01 In brief
Summary
GLM-5V-Turbo is a native multimodal foundation model for agentic tasks, integrating perception, reasoning, planning, and execution.
It introduces CogViT, a vision encoder trained via distillation and contrastive learning, and Multimodal Multi-Token Prediction (MMTP) using a shared <|image|> token for efficiency.
The model undergoes joint RL over 30+ task categories, improving perception, reasoning, and agentic performance.
It expands its multimodal toolchain and integrates with frameworks like Claude Code and AutoClaw.
A new benchmark, ImageMining, evaluates vision-centric deep search.
Results show strong performance on multimodal coding (Design2Code 94.8), tool use (ImageMining 30.7, BrowseComp-VL 51.9), GUI agents (AndroidWorld 75.7, OSWorld 62.3), and text coding (CC-Backend 22.8).
Development insights highlight perception as foundational, hierarchical optimization for agent training, and the need for clear task specification and reliable verification.
Remaining challenges include strategy emergence, multimodal context management, and model-harness co-evolution.
02 From the paper
Abstract
We present GLM-5V-Turbo, a step toward native foundation models for multimodal agents. As foundation models are increasingly deployed in real environments, agentic capability depends not only on language reasoning, but also on the ability to perceive, interpret, and act over heterogeneous contexts such as images, videos, webpages, documents, GUIs. GLM-5V-Turbo is built around this objective: multimodal perception is integrated as a core component of reasoning, planning, tool use, and execution, rather than as an auxiliary interface to a language model. This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcement learning, toolchain expansion, and integration with agent frameworks. These developments lead to strong performance in multimodal coding, visual tool use, and framework-based agentic tasks, while preserving competitive text-only coding capability. More importantly, our development process offers practical insights for building multimodal agents, highlighting the central role of multimodal perception, hierarchical optimization, and reliable end-to-end verification.