Paper 2509.06160
Reverse-Engineered Reasoning for Open-Ended Generation
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 33
- GitHub
- 98 stars
01 In brief
Summary
The paper introduces REverse-Engineered Reasoning (REER), a new paradigm for instilling deep reasoning in LLMs for open-ended, non-verifiable tasks like creative writing, where reinforcement learning (RL) and instruction distillation fail due to lack of clear rewards and high costs.
REER works backwards from known good solutions, using a gradient-free local search that iteratively refines an initial reasoning trajectory to minimize the perplexity of the reference solution, thereby discovering plausible human-like thought processes.
The authors curate DeepWriting-20K, a dataset of 20,000 reasoning trajectories across 25 categories, and fine-tune Qwen3-8B to create DeepWriter-8B.
Experiments on LongBench, HelloBench, and WritingBench show DeepWriter-8B outperforms open-source baselines (e.g., LongWriter-8B) and matches or exceeds proprietary models like GPT-4o and Claude 3.5, especially in creative and professional writing tasks.
Ablations confirm the importance of synthesized data, iterative refinement, reflection tokens, and trajectory length.
The work provides a scalable, cost-effective third path for teaching complex reasoning without RL or distillation, democratizing deep reasoning capabilities for open-ended generation.
02 From the paper
Abstract
While the ``deep reasoning'' paradigm has spurred significant advances in verifiable domains like mathematics, its application to open-ended, creative generation remains a critical challenge. The two dominant methods for instilling reasoning -- reinforcement learning (RL) and instruction distillation -- falter in this area; RL struggles with the absence of clear reward signals and high-quality reward models, while distillation is prohibitively expensive and capped by the teacher model's capabilities. To overcome these limitations, we introduce REverse-Engineered Reasoning (REER), a new paradigm that fundamentally shifts the approach. Instead of building a reasoning process ``forwards'' through trial-and-error or imitation, REER works ``backwards'' from known-good solutions to computationally discover the latent, step-by-step deep reasoning process that could have produced them. Using this scalable, gradient-free approach, we curate and open-source DeepWriting-20K, a large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks. Our model, DeepWriter-8B, trained on this data, not only surpasses strong open-source baselines but also achieves performance competitive with, and at times superior to, leading proprietary models like GPT-4o and Claude 3.5.