Paper 2607.16617
DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data Pipelines
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- Not linked
01 In brief
Summary
DataFlow-Harness is a platform that bridges the NL2Pipeline gap, where coding agents produce scripts that are not persistent, editable platform artifacts.
It guides an LLM agent to construct platform-native DAGs via typed, incremental mutations, combining DataFlow-Skills for procedural guidance, an MCP layer for live operator registry and pipeline state, and DataFlow-WebUI for synchronized conversational and visual editing.
On a 12-task benchmark, DataFlow-Harness achieves a 93.3% end-to-end pass rate, reducing cost by 72.5% and latency by 49.9% versus Vanilla Claude Code, and is within 0.9 points of Context-Aware Claude Code with 42.8% lower cost.
Per-task analysis shows Skills are most beneficial for tasks requiring implicit procedural knowledge.
In a textbook-to-VQA task, it achieves 97.2% precision and 87.3% coverage.
Downstream training evaluations show data from DataFlow-Harness pipelines improves math and code benchmark accuracy compared to Vanilla CC pipelines, with gains on AIME and MBPP.
The platform produces persistent, editable workflows with reliability close to script baselines and lower cost and latency.
02 From the paper
Abstract
Large language models (LLMs) are increasingly used to automate data-processing workflows, yet coding agents typically produce scripts that are not automatically materialized as persistent, editable platform artifacts. We call this disconnect the \textit{NL2Pipeline gap}. To bridge it, we introduce \textsc{DataFlow-Harness}, a platform that guides an LLM agent to construct platform-native directed acyclic graphs (DAGs) through typed, incremental mutations rather than free-form scripts. The platform combines \textsc{DataFlow-Skills} for procedural guidance, a Model Context Protocol (MCP) layer that exposes the live operator registry and current pipeline state, and \textsc{DataFlow-WebUI}, which synchronizes conversational authoring with a visual DAG editor. On a 12-task data-engineering benchmark, \textsc{DataFlow-Harness} achieves a 93.3\% observed end-to-end pass rate. Relative to Vanilla Claude Code, it reduces measured monetary cost by 72.5\% and generation latency by 49.9\%; its observed pass rate is within 0.9 percentage points of the Context-Aware Claude Code baseline while its cost is 42.8\% lower. Per-task analysis indicates that Skills are most useful when construction depends on implicit procedural knowledge. These results show that live platform grounding can produce persistent, editable workflow artifacts with an observed reliability close to script-generation baselines and with lower measured construction cost and latency.