Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability

This preprint challenges the narrative that supervised fine-tuning (SFT) memorizes while reinforcement learning generalizes, showing that cross-domain generalization in reasoning SFT is conditional on optimization dynamics, training data, and base-model capability.

Using math-only long chain-of-thought (CoT) SFT on pretrained base models, the authors find that apparent non-generalization can be an under-optimization artifact: out-of-domain performance exhibits a dip-and-recovery pattern, improving beyond the base model with extended training.

Data quality and structure matter: low-quality solutions broadly hurt generalization, while verified long-CoT traces, even from a toy arithmetic game (Countdown), yield consistent cross-domain gains.

Model capability is essential: stronger models internalize transferable procedural patterns (e.g., backtracking), while weaker models imitate surface verbosity.

Generalization is asymmetric: reasoning improves but safety degrades, with long-CoT SFT increasing attack success rates on harmful queries.

The authors conclude that the question is not whether reasoning SFT generalizes, but under what conditions and at what cost, and they provide a controlled comparison of optimization schedules, data configurations, and model scales to map this conditional landscape.