arXiv.org
Why Language Models Hallucinate
The paper argues that language model hallucinations arise from statistical pressures during pretraining and persist due to misaligned evaluation metrics. The authors formalize hallucinations as errors in binary classification, showing that even with error-free training data, the cross-entropy objective leads to errors. They introduce the Is-It-Valid (IIV)…
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang- Published
- Sep 2025
- Citations
- 282
- Code
- Not linked