Paper 2512.12967
QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 23
- GitHub
- 550 stars
01 In brief
Summary
QwenLong-L1.5 is a long-context reasoning model built on Qwen3-30B-A3B-Thinking, introduced by Alibaba's Tongyi Lab.
It achieves performance comparable to GPT-5 and Gemini-2.5-Pro on long-context benchmarks, surpassing its baseline by 9.90 points on average.
The model's post-training recipe includes three key innovations: (1) a data synthesis pipeline that generates challenging multi-hop reasoning tasks from atomic facts and relationships, creating 14.1K high-quality training samples; (2) stabilized reinforcement learning using task-balanced sampling, task-specific advantage estimation, and Adaptive Entropy-Controlled Policy Optimization (AEPO) to handle long-context training instability; and (3) a memory-augmented architecture with multi-stage fusion RL training for tasks exceeding 4M tokens.
On ultra-long tasks (1M-4M tokens), the memory-agent framework yields a 9.48-point gain over the agent baseline.
The model also shows improved performance in general domains like scientific reasoning, memory tool use, and extended dialogue.
The training pipeline progresses through four full-context RL stages with increasing input lengths (32K to 120K tokens) and a memory-RL stage, using model merging with SCE to combine capabilities.
02 From the paper
Abstract
We introduce QwenLong-L1.5, a model that achieves superior long-context reasoning capabilities through systematic post-training innovations. The key technical breakthroughs of QwenLong-L1.5 are as follows: (1) Long-Context Data Synthesis Pipeline: We develop a systematic synthesis framework that generates challenging reasoning tasks requiring multi-hop grounding over globally distributed evidence. By deconstructing documents into atomic facts and their underlying relationships, and then programmatically composing verifiable reasoning questions, our approach creates high-quality training data at scale, moving substantially beyond simple retrieval tasks to enable genuine long-range reasoning capabilities. (2) Stabilized Reinforcement Learning for Long-Context Training: To overcome the critical instability in long-context RL, we introduce task-balanced sampling with task-specific advantage estimation to mitigate reward bias, and propose Adaptive Entropy-Controlled Policy Optimization (AEPO) that dynamically regulates exploration-exploitation trade-offs. (3) Memory-Augmented Architecture for Ultra-Long Contexts: Recognizing that even extended context windows cannot accommodate arbitrarily long sequences, we develop a memory management framework with multi-stage fusion RL training that seamlessly integrates single-pass reasoning with iterative memory-based processing for tasks exceeding 4M tokens. Based on Qwen3-30B-A3B-Thinking, QwenLong-L1.5 achieves performance comparable to GPT-5 and Gemini-2.5-Pro on long-context reasoning benchmarks, surpassing its baseline by 9.90 points on average. On ultra-long tasks (1M~4M tokens), QwenLong-L1.5's memory-agent framework yields a 9.48-point gain over the agent baseline. Additionally, the acquired long-context reasoning ability translates to enhanced performance in general domains like scientific reasoning, memory tool using, and extended dialogue.