Paper 2601.22813
Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
- Published
- Jan 2026
- Research lab
- Independent
- Citations
- 11
- GitHub
- 80 stars
01 In brief
Summary
The paper introduces Quartet II, a fully-NVFP4 quantization scheme for LLM pre-training on NVIDIA Blackwell GPUs.
It proposes MS-EDEN, a new unbiased quantization routine for microscaling formats that reduces quantization error by more than 2x compared to stochastic rounding (SR), by applying randomized Hadamard rotations and merging EDEN correction factors into FP8 group scales via stochastic rounding.
Quartet II combines a forward pass using RTN with native NVFP4 scaling and the 'Four Over Six' heuristic, with a backward pass using MS-EDEN and weight re-quantization.
Analytical and empirical results show consistently better gradient estimation across all major matrix multiplications.
End-to-end LLM training up to 1.9B parameters on 38B tokens shows 15-25% reduction in validation loss gap vs.
prior NVFP4 methods.
Custom CUDA kernels achieve up to 4.2x speedup over BF16 on RTX 5090 and up to 2.5x on B200 for linear layers, with 1.8x end-to-end training throughput improvement for 1B models.
The code is available at https://github.com/IST-DASLab/Quartet-II.
02 From the paper
Abstract
The NVFP4 lower-precision format, supported in hardware by NVIDIA Blackwell GPUs, promises to allow, for the first time, end-to-end fully-quantized pre-training of massive models such as LLMs. Yet, existing quantized training methods still sacrifice some of the representation capacity of this format in favor of more accurate unbiased quantized gradient estimation by stochastic rounding (SR), losing noticeable accuracy relative to standard FP16 and FP8 training. In this paper, improve the state of the art for quantized training in NVFP4 via a novel unbiased quantization routine for micro-scaled formats, called MS-EDEN, that has more than 2x lower quantization error than SR. We integrate it into a novel fully-NVFP4 quantization scheme for linear layers, called Quartet II. We show analytically that Quartet II achieves consistently better gradient estimation across all major matrix multiplications, both on the forward and on the backward passes. In addition, our proposal synergizes well with recent training improvements aimed specifically at NVFP4. We further validate Quartet II on end-to-end LLM training with up to 1.9B parameters on 38B tokens. We provide kernels for execution on NVIDIA Blackwell GPUs with up to 4.2x speedup over BF16. Our code is available at https://github.com/IST-DASLab/Quartet-II .