Paper 2606.07591
ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research
- Published
- May 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- 232 stars
01 In brief
Summary
ResearchClawBench (RCBench) is a benchmark for evaluating end-to-end autonomous scientific research, comprising 40 tasks across 10 scientific domains (Astronomy, Chemistry, Earth Science, Energy, Information, Life, Material, Math, Neuroscience, Physics).
Each task is derived from a real published paper, provides related literature and raw data, and hides the target paper during evaluation.
Expert-curated multimodal rubrics decompose target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery (score 50) while allowing for new discovery (scores above 50).
The benchmark introduces the Reference-Anchored Discovery Score (RADS) and a lightweight tool-use harness (ResearchHarness) for evaluating native LLMs.
Seven autonomous agents and seventeen LLMs were evaluated.
The strongest agent, Claude Code, averaged 21.5, and the strongest LLM, Claude-Opus-4.7, averaged 20.7, with frontier means of 25.8 and 26.5 respectively.
Error analysis shows failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core.
The benchmark provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.
02 From the paper
Abstract
AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation. Expert-curated multimodal rubrics decompose the target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery while leaving room for new discovery. We evaluate seven autonomous research (auto-research) agents under a unified protocol and seventeen native LLMs through the lightweight ResearchHarness. Current systems remain far from reliable re-discovery: the strongest autonomous agent, Claude Code, averages 21.5, and the strongest ResearchHarness LLM, Claude-Opus-4.7, averages 20.7, with an LLM frontier mean of only 26.5. Error analysis shows that failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core. ResearchClawBench provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.