Paper 2606.29538
RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 368 stars
01 In brief
Summary
Resource2Skill is a framework that automatically distills multimodal human-created resources—tutorial videos, repositories, articles, and reference artifacts—into executable skills for software agents, organized as a hierarchical multimodal Skill Wiki.
Each skill entry combines structured text, code, visual examples, metadata, and provenance, preserving complementary signals from different resource types.
At inference, agents retrieve and compose relevant skills via a two-stage MetaBrowse selector (BM25 then language model), and the same construction operator can acquire new skills online when coverage is insufficient.
Across seven authoring domains (Web, Excel, Reaper, PPT, Blender, CAD, UE5) and four agent backends, Resource2Skill improves average overall scores by +11.9 percentage points over no-skill agents and outperforms strong harness baselines (ClaudeCode-H, Codex-H) in 26 of 28 main-aggregate model–domain cells.
Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity (especially video), selection strategy, and online acquisition.
The system is evaluated with a vision/audio judge and validated by a human A/B study (85.5% win rate excluding ties).
Limitations include reliance on programmatic tool interfaces and public procedural content, and online acquisition latency.
02 From the paper
Abstract
Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.