Paper 2606.21337
DataClaw0: Agentic Tailoring Multimodal Data from Raw Streams
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 117 stars
01 In brief
Summary
DataClaw0 is a framework for intent-conditioned multimodal data tailoring, converting raw, noisy streams (e.g., tutorial videos, embodied trajectories, GUI logs) into schema-aligned, evidence-grounded training instances.
The authors formalize this as a learnable capability and train models at 4B, 9B, and 27B scales, comparing a jointly trained omni model (DataClaw0-O) against per-domain experts with a router (DataClaw0-E).
They find that the optimal configuration depends on capacity: sharding wins at 4B and 9B, while joint training wins at 27B, placing the crossover near 18B parameters.
Matched-data experiments attribute this reversal to positive cross-domain transfer, which helps most in data-poor domains and recovers 59% of in-domain performance on held-out domains.
Downstream post-training on GUI navigation, action video generation, and spatio-temporal VQA reproduces this ordering, and the joint model is also cheaper to deploy.
The paper introduces a two-stage data construction pipeline (deterministic anchor extraction plus VLM synthesis) and a benchmark (DataClaw0-val) with a fuzzy-intent subset.
Cost analysis shows self-hosted tailoring breaks even after ~80K samples for 9B and ~324K for 27B against API annotation, depending on accelerator prices.
02 From the paper
Abstract
Raw multimodal streams are abundant but noisy, redundant, and unaligned with any particular training objective. Turning them into supervision today means either brittle heuristics or repeatedly querying a proprietary vision-language model, a cost that recurs with every new sample. We ask whether this conversion can instead be learned once and reused, and formalise intent-conditioned Data Tailoring: given a raw stream and a high-level intent, a model must return schema-aligned, evidence-grounded training instances. Training DataClaw0 at 4B, 9B and 27B, we find that whether five heterogeneous domains should share one model depends on capacity. A jointly trained model is worse than per-domain experts at the two smaller scales and better at the largest, placing the crossover near 18B parameters. Matched-data comparisons, in which the joint model sees exactly the same data per domain as that domain's expert, attribute the reversal to cross-domain transfer rather than to data volume: it gains most where a domain is data-poor, and recovers 59\% of in-domain performance on domains withheld from training entirely. Downstream post-training reproduces this ordering on GUI navigation, action video generation and spatio-temporal VQA, and the joint configuration is also the cheaper to deploy, serving one model instead of five. Github: https://github.com/vancyland/DataClaw0