SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

SKILLSBENCH is a benchmark for evaluating Agent Skills, which are structured packages of procedural knowledge that augment LLM agents at inference time.

The benchmark contains 87 tasks across 8 domains, each with curated Skills and deterministic verifiers.

In an evaluation of 18 model–harness configurations, curated Skills raised the average pass rate from 33.9% to 50.5% (+16.6 percentage points; 25.5% normalized gain), with configuration-level gains ranging from +4.1 to +25.7 pp.

Focused Skills with at most three modules outperformed larger or exhaustive bundles, and smaller models with Skills could match larger models without them.

Self-generated Skills underperformed the no-Skills baseline on all three tested configurations.

The benchmark uses a paired evaluation framework (with vs.

without Skills) to isolate Skill efficacy from model and harness effects, and it is built on the BenchFlow containerized harness.

The paper also analyzes the Skill ecosystem, finding 2,014,000 source-partitioned Skills with a median bundle size of about 1.8k tokens, and provides detailed appendices on task construction, quality filtering, and experimental protocol.