Paper 2605.19577
GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment
- Published
- May 2026
- Research lab
- Independent
- Citations
- 2
- GitHub
- 75 stars
01 In brief
Summary
GoLongRL is a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR).
It introduces a dataset of 22,965 samples spanning 9 task types, each paired with its natural evaluation metric (e.g., EM, F1, NDCG), and a four-phase construction pipeline.
The dataset combines curated open-source samples and synthetic QA pairs generated from real documents.
Under vanilla GRPO, this dataset improves long-context average scores from 53.0 to 62.2 on Qwen3-4B-Thinking and from 60.1 to 69.8 on Qwen3-30B-A3B, outperforming QwenLong-L1.5 trained with GRPO.
The paper also proposes TMN-Reweight, which combines task-level mean normalization for cross-task reward scale alignment with difficulty-adaptive reweighting.
TMN-Reweight further improves the 4B average to 63.0, with gains on aggregation-intensive benchmarks like CorpusQA (+4.5).
General capabilities (MMLU-Pro, AIME24/25, GPQA) are preserved or improved, and long-context skills transfer to agentic and dialogue memory benchmarks.
All resources are publicly available.
02 From the paper
Abstract
We present GoLongRL, a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). Existing long-context RL methods often treat data construction as a matter of designing increasingly complex retrieval paths, leading to homogeneous task coverage and reward formulations that inadequately reflect practical long-context requirements. Our work offers two contributions. (1) Capability-oriented data construction with full open release. We openly release a dataset of 23K RLVR samples, the complete construction pipeline, and all training code. Guided by a taxonomy of long-context capabilities, the dataset spans 9 task types, each paired with its natural evaluation metric. It comprises curated open-source samples from established corpora and synthetic samples whose QA pairs are generated from real source documents such as books, academic papers, and multi-turn dialogues. Under the same vanilla GRPO setup, our dataset alone outperforms the closed-source QwenLong-L1.5 dataset. Moreover, our Qwen3-30B-A3B model trained on this data delivers long-context performance comparable to DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507, suggesting that broader coverage and greater reward diversity substantially benefit long-context capability improvement. (2) TMN-Reweight for heterogeneous multitask optimization. To address optimization challenges from heterogeneous rewards, we propose TMN-Reweight, which combines task-level mean normalization for cross-task reward scale alignment with difficulty-adaptive weighting for more reliable advantage estimation. TMN-Reweight further improves average performance over vanilla GRPO, with general capabilities preserved or improved across reported evaluations.