Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

The paper introduces Native Parallel Reasoner (NPR), a teacher-free framework that enables large language models to develop genuine parallel reasoning capabilities through self-distillation and reinforcement learning.

NPR uses a three-stage progressive training paradigm: Stage 1 applies format-following RL (DAPO) to induce a structured parallel format, producing NPR-ZERO; Stage 2 uses rejection sampling and parallel SFT with parallel attention masks and positional encoding to create NPR-BETA; Stage 3 applies a novel Parallel-Aware Policy Optimization (PAPO) algorithm that optimizes branching policies directly within the parallel execution graph.

The authors also developed the NPR Engine, which refactors SGLang's memory management and flow control to support stable large-scale parallel RL training.

Experiments on Qwen3-4B across eight reasoning benchmarks show performance gains up to 24.5% and inference speedups up to 4.6x compared to autoregressive baselines.

NPR achieves 100% genuine parallel execution, unlike prior baselines that often fall back to autoregressive decoding.

The self-distilled dataset outperforms teacher-generated trajectories by an average of 10.1 points, and parallel RL consistently surpasses sequential RL baselines.

The framework demonstrates that native parallel reasoning is a promising direction for more scalable and efficient agentic intelligence.