The Year in AI Papers
The year/Independent research

Paper 2607.23588

JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

Yunlong Lin, Zixu Lin, Zhaohu Xing, Biqiang Li, Chenxin Li, Haonan Wang, Haitao Wu, Hengyu Liu, Jianghai Chen, Kaituo Feng, Kaixin Li, Shawn Chen, et al.

Published
Jul 2026
Research lab
Independent
Citations
0
GitHub
277 stars

01 In brief

Summary

JarvisHub is an open, canvas-native agent harness designed for long-horizon multimodal creative tasks, addressing the limitations of prompt-to-output tools, chat-based agents, and node-based workflows that fail to maintain a unified project state.

It treats an editable canvas as the shared workspace, external memory, and action space, representing artifacts, dependencies, versions, and feedback as typed nodes and links.

The architecture comprises three layers: a canvas state layer storing project materials, a protocol bridge validating and logging agent actions, and an agent runtime orchestrating tools, skills, memory, and subagents.

This design enables agents to plan, generate, revise, and organize projects while users can inspect and intervene.

The harness supports persistent project state, controlled canvas actions, tool execution, feedback-guided revision, and traceability.

Experiments on narrative media generation, interactive web development, and presentation deck generation demonstrate its capability to produce coherent outputs with inspectable processes.

Limitations include qualitative evaluations, dependence on external models, and the need for trajectory filtering before use as research data.

JarvisHub aims to enable project-state benchmarks, process-level evaluation, and data flywheels for training future creative agents.

02 From the paper

Abstract

Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative work requires more than isolated prompt-output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt-based, chat-based, and node-based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent-assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas-native creative agent harness for long-horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent's external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three-layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.