Paper 2606.00683
OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
- Published
- May 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 43 stars
01 In brief
Summary
The paper introduces OCC-RAG, a family of small language models (SLMs) specialized for faithful, context-grounded question answering (QA).
The models are mid-trained from Qwen3 base models (0.6B and 1.7B) on a synthetic corpus of over 3 million QA examples generated via a novel pipeline.
The corpus includes single-hop, multi-hop (single- and multi-context), and abstention examples, with structured reasoning traces and source citations.
The models are designed to perform multi-hop reasoning, avoid memorization (faithfulness), and abstain when evidence is insufficient.
Evaluated on HotpotQA, MuSiQue, TAT-QA, ConFiQA, and MuSiQue-Un, OCC-RAG models outperform general-purpose models 2–6× their size on faithfulness and refusal, and match or exceed them on multi-hop reasoning.
For example, OCC-RAG-0.6B exceeds Qwen3-1.7B by 9.5 points on ConFiQA and reduces memorization ratio from 8.2 to 5.2.
The results demonstrate that compact, task-specialized SLMs can achieve competitive performance with much larger models when trained for context grounding and evidence-based reasoning.
02 From the paper
Abstract
Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practical applications benefit more from robust reasoning than from extensive parametric knowledge. In this setting, task-specialized small language models (SLMs) offer a principled design choice. We introduce Optimal Cognitive Core (OCC), a family of SLMs built around this premise. As a variant of OCC, we present OCC-RAG, optimized for faithful question answering (QA) grounded in the provided context. This task directly aligns with the OCC design approach, requiring multi-hop reasoning over supplied passages while ignoring memorized knowledge. To train OCC-RAG, we implement a novel pipeline for synthesizing multi-context, multi-hop QA data at scale, producing a corpus of over three million examples targeting multi-hop reasoning, strict context faithfulness, and calibrated abstention. We release OCC-RAG-0.6B and OCC-RAG-1.7B, both mid-trained on this corpus. The models produce structured reasoning traces with source citations grounded in literal quotes from the context. Through OCC-RAG, we demonstrate that compact, task-specialized SLMs can match or exceed general-purpose models 2 -- 6x their size across multi-hop reasoning (HotpotQA, MuSiQue, TAT-QA), faithfulness (ConFiQA), and refusal (MuSiQue-Un) benchmarks.