Paper 2509.01363
Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 5
- GitHub
- Not linked
01 In brief
Summary
This paper introduces the concept of a reasoning vector, which captures the reasoning capability instilled by reinforcement learning (RL) in large language models (LLMs) and can be transferred to other models via simple tensor arithmetic.
The authors source two publicly available QWEN2.5 models (1.5B and 7B) with identical initialization, one fine-tuned with supervised fine-tuning (SFT) and the other with Group Relative Policy Optimization (GRPO) on the same GSM8K dataset.
The reasoning vector is defined as the difference between their parameters: v_reason = θ_GRPO − θ_SFT.
Adding this vector to compatible instruction-tuned models (QWEN2.5-Instruct) consistently improves performance across benchmarks: for the 1.5B model, GSM8K improves by 4.9%, HumanEval by 4.3%, and BigBenchHard by 12.3%; for the 7B model, GSM8K improves by 5.0% and HumanEval by 3.7%.
These gains persist under adversarial perturbations (e.g., noise injection, sentence shuffling).
Conversely, subtracting the vector degrades GSM8K accuracy by 11.8%, demonstrating the vector's strong contribution.
The method relies on Linear Mode Connectivity, which ensures that models fine-tuned from the same initialization lie in a connected low-loss region, making the transfer safe.
The approach requires identical architecture, tokenizer, and initialization family between donor and target models.
The authors also show cross-domain transfer, where a math-derived vector improves code generation and vice versa, though with smaller gains.
The work offers a training-free, lightweight method to enhance reasoning by reusing existing open-source checkpoints.
02 From the paper
Abstract
Large language models often require costly optimization, such as reinforcement learning, to master complex reasoning tasks. This work demonstrates that reasoning ability, once learned, can be extracted and transferred between models as a compact task vector. We source two publicly available, identically initialized Qwen2.5 models, one fine-tuned with supervised fine-tuning (SFT) and the other with group relative policy optimization (GRPO) on the same dataset. From these, we extract a reasoning vector: $v_{\text{reason}} = θ_{\text{GRPO}} - θ_{\text{SFT}}$. We hypothesize that this vector captures the reasoning capability instilled by reinforcement learning while factoring out shared knowledge from the SFT process. When added to compatible instruction-tuned models through simple arithmetic, this vector consistently improves performance across diverse reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and BigBenchHard (+12.3% for the 1.5B model). The performance improvements persist under adversarial conditions. Conversely, subtracting the vector causes significant performance degradation (-11.8% on GSM8K), demonstrating the vector's strong contribution to the model's reasoning abilities. This work shows how reasoning capabilities, typically developed through expensive training, can be extracted from existing open-source models and reused through simple tensor arithmetic, offering a practical way to enhance models by recycling prior computational investments.