Paper 2602.08222
Weak-Driven Learning: How Weak Agents make Strong Agents Stronger
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 4
- GitHub
- 79 stars
01 In brief
Summary
The paper introduces Weak-Driven Learning, a post-training paradigm that repurposes weak historical checkpoints as corrective signals to strengthen a strong language model, contrasting with knowledge distillation that imitates stronger teachers.
The authors instantiate this with WMSS (Weak agents Make Strong agents Stronger), which combines Weak-Driven Discrepancy Selection (WD-DS) and Weak-Driven Joint-Logit Training (WD-JT).
WD-DS selects hard or regressed samples using entropy discrepancies between weak and strong agents, while WD-JT mixes weak and strong logits and optimizes cross-entropy against ground truth, converting weak-model errors on hard negatives into corrective gradients.
A theoretical analysis shows that logit mixing can amplify gradient signal on weak-revealed hard negatives under margin conditions.
Experiments on math, code, and logic reasoning across Qwen3-4B/8B, Qwen2.5-3B, and Gemma-3-4B-PT show WMSS improves Math-Avg by up to +5.9 points and Code-Avg by up to +4.4 points over SFT, outperforming baselines like SPIN and SSB, with no additional deployment-time inference cost.
Ablations confirm both modules contribute, and WD-DS reduces training steps by about 44% while improving accuracy.
The method requires an extra weak-agent forward pass during training but no extra cost at inference.
02 From the paper
Abstract
As post-training optimization becomes central to improving large language models, we observe a persistent saturation bottleneck: once models grow highly confident, further training yields diminishing returns. While existing methods continue to reinforce target predictions, we find that informative supervision signals remain latent in models' own historical weak states. Motivated by this observation, we propose WMSS (Weak Agents Can Make Strong Agents Stronger), a post-training paradigm that leverages weak checkpoints to guide continued optimization. By identifying recoverable learning gaps via entropy dynamics and reinforcing them through compensatory learning, WMSS enables strong agents to improve beyond conventional post-training saturation. Experiments on mathematical reasoning and code generation datasets show that agents trained with our approach achieve effective performance improvements, while incurring zero additional inference cost.