Paper 2509.14008
Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scale
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 5
- GitHub
- 5 stars
01 In brief
Summary
The HALA technical report introduces a family of Arabic-centric instruction and translation models built with a translate-and-tune pipeline.
The pipeline compresses a strong AR↔EN teacher model (CohereLabs/command-a-translate-08-2025) to FP8, achieving ~2x higher throughput with no quality loss.
This teacher translates 405K Open-Orca pairs into Arabic, and a filtered subset of OPUS-100 (439,592 pairs) is added, totaling ~1.26M bilingual examples.
A lightweight translator (LFM2-1.2B) is fine-tuned on this data and then used to translate multiple English instruction datasets (Hermes 3, SCP-116K, ReAlign-Alpaca, LaMini, Tulu 3, Synthetic Instruct-GPT-J Pairwise, Open-Orca) into Arabic, creating a ~4.5M-sample Arabic instruction corpus.
HALA models are trained at 350M, 700M, 1.2B, and 9B parameters, with slerp merging (t=0.5) to balance Arabic specialization with base-model strengths.
On six Arabic-centric benchmarks (AlGhafa, ArabicMMLU, EXAMS, MadinahQA, AraTrust, ArbMMLU-HT), HALA-1.2B achieves the best average (51.4%) in the nano (≤2B) category, and HALA-9B achieves the best average (69.9%) in the small (7–9B) category, outperforming their bases.
The EN→AR translator quality is validated on 500 MMLU questions, with the lightweight translator improving BLEU from 16.0 to 48.2 over its base.
Training cost was under $1,000 on 8×H100 GPUs, with translation on 12×A100 GPUs costing ~$500.
Models, data, and recipes are released.
02 From the paper
Abstract
We present Hala, a family of Arabic-centric instruction and translation models built with our translate-and-tune pipeline. We first compress a strong AR$\leftrightarrow$EN teacher to FP8 (yielding $\sim$2$\times$ higher throughput with no quality loss) and use it to create high-fidelity bilingual supervision. A lightweight language model LFM2-1.2B is then fine-tuned on this data and used to translate high-quality English instruction sets into Arabic, producing a million-scale corpus tailored to instruction following. We train Hala models at 350M, 700M, 1.2B, and 9B parameters, and apply slerp merging to balance Arabic specialization with base-model strengths. On Arabic-centric benchmarks, Hala achieves state-of-the-art results within both the "nano" ($\leq$2B) and "small" (7-9B) categories, outperforming their bases. We release models, data, evaluation, and recipes to accelerate research in Arabic NLP.