Paper 2509.14760
Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 3
- GitHub
- 24 stars
01 In brief
Summary
The paper introduces specification alignment, a challenge for LLMs to follow dynamic, scenario-specific behavioral and safety specifications.
To evaluate this, the authors present SPECBENCH, a benchmark covering 5 scenarios, 103 specifications, and 1,500 prompts.
Experiments on 33 models reveal significant alignment gaps and a safety-behavior trade-off.
The authors propose ALIGN3, a lightweight test-time deliberation method that uses hierarchical reflection and revision to reason over specification boundaries.
ALIGN3 improves Qwen3-14B's SAR from 51.03% to 62.92% with minimal token overhead, approaching GPT-4.1's 69.20%.
The results demonstrate that test-time deliberation is an effective strategy for improving specification alignment.
02 From the paper
Abstract
Large language models (LLMs) are increasingly applied in diverse real-world scenarios, each governed by bespoke behavioral and safety specifications (spec) custom-tailored by users or organizations. These spec, categorized into safety-spec and behavioral-spec, vary across scenarios and evolve with changing preferences and requirements. We formalize this challenge as specification alignment, focusing on LLMs' ability to follow dynamic, scenario-specific spec from both behavioral and safety perspectives. To address this challenge, we propose Align3, a lightweight method that employs Test-Time Deliberation (TTD) with hierarchical reflection and revision to reason over the specification boundaries. We further present SpecBench, a unified benchmark for measuring specification alignment, covering 5 scenarios, 103 spec, and 1,500 prompts. Experiments on 15 reasoning and 18 instruct models with several TTD methods, including Self-Refine, TPO, and MoreThink, yield three key findings: (i) test-time deliberation enhances specification alignment; (ii) Align3 advances the safety-helpfulness trade-off frontier with minimal overhead; (iii) SpecBench effectively reveals alignment gaps. These results highlight the potential of test-time deliberation as an effective strategy for reasoning over the real-world specification boundaries. Our code and resources are available at https://github.com/zzzhr97/SpecBench.