The Year in AI Papers
The year/Independent research

Paper 2604.02176

Adam's Law: Textual Frequency Law on Large Language Models

Hongyuan Adam Lu, Z. L., Victor Wei, Zefan Zhang, Zhao Hong, Qiqi Xiang, Bowen Cao, Wai Lam

Published
Apr 2026
Research lab
Independent
Citations
3
GitHub
Not linked

01 In brief

Summary

The paper proposes the Textual Frequency Law (TFL), which states that for LLMs, higher-frequency textual data should be preferred for both prompting and fine-tuning when meaning is kept constant.

The framework includes Textual Frequency Distillation (TFD) to enhance frequency estimation via story completion, and Curriculum Textual Frequency Training (CTFT) to fine-tune models in increasing order of sentence-level frequency.

Experiments on math reasoning, machine translation, commonsense reasoning, and tool calling show that high-frequency inputs improve performance across models like GPT-4o-mini, DeepSeek-V3, and Llama-3.3-70B-Instruct.

A theoretical proof is provided, linking token frequency to lower loss under Zipf's law.

The work introduces the Textual Frequency Paired Dataset (TFPD) for evaluation.

Results show high-frequency paraphrases consistently outperform low-frequency ones, with improvements in accuracy and translation metrics, and CTFT further boosts fine-tuning performance.

The paper concludes that textual frequency is a significant factor in LLM performance, with implications for data selection and training order.

Limitations include computational costs of TFD and reliance on approximate frequency estimation from online resources.

The work is supported by an ethics statement and references prior work on frequency effects in human cognition and language models.

The appendix includes detailed results, hyperparameters, and a formal proof of the law.

The authors are from FaceMind Corporation and The Chinese University of Hong Kong.

The code is available on GitHub.

The paper is under review at arXiv.

The study was approved by FaceMind ethics review.

The dataset is curated with human annotation to ensure semantic equivalence.

The results are consistent across multiple tasks…

02 From the paper

Abstract

While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel research direction in terms of textual data frequency, which is an understudied topic, to the best of our knowledge. Our framework is composed of three units. First, this paper proposes Textual Frequency Law (TFL), which indicates that frequent textual data should be preferred for LLMs for both prompting and fine-tuning. Since many LLMs are closed-source in their training data, we propose using online resources to estimate the sentence-level frequency. We then utilize an input paraphraser to paraphrase the input into a more frequent textual expression. Next, we propose Textual Frequency Distillation (TFD) by querying LLMs to conduct story completion by further extending the sentences in the datasets, and the resulting corpora are used to adjust the initial estimation. Finally, we propose Curriculum Textual Frequency Training (CTFT) that fine-tunes LLMs in an increasing order of sentence-level frequency. Experiments are conducted on our curated dataset Textual Frequency Paired Dataset (TFPD) on math reasoning, machine translation, commonsense reasoning and agentic tool calling. Results show the effectiveness of our framework.