Paper 2509.26507
The Dragon Hatchling: The Missing Link between the Transformer and Models of the Brain
- Published
- Sep 2025
- Research lab
- Independent
- Citations
- 5
- GitHub
- 3.4K stars
01 In brief
Summary
The paper introduces Dragon Hatchling (BDH), a new Large Language Model architecture based on a scale-free, biologically inspired network of n locally-interacting neuron particles.
BDH couples strong theoretical foundations and inherent interpretability without sacrificing Transformer-like performance.
It is a practical, performant attention-based state space sequence learning architecture that admits a GPU-friendly formulation (BDH-GPU).
Empirically, BDH rivals GPT2-architecture Transformer performance on language and translation tasks at parameter scales from 10M to 1B, with the same training data.
BDH provides theoretical foundations for understanding model behavior in the limit of large size and reasoning time, showing a macro-to-micro correspondence between general attention mechanisms in state-of-the-art Language Models and attention mechanisms observed in the brain.
These attention mechanisms formally converge as closed-form local graph dynamics at neurons and synapses, termed 'the equations of reasoning'.
BDH can be represented as a brain model with n neurons organized as excitatory and inhibitory circuits with integrate-and-fire thresholding, and its working memory during inference relies entirely on synaptic plasticity with Hebbian learning using spiking neurons.
The model exhibits emergent modularity and heavy-tailed degree distribution in its neuron interaction network, and demonstrates monosemanticity on language tasks, including representation of concept abstractions, even for small models below 100M parameters.
BDH is designed for interpretability with sparse, positive activation vectors, and opens the door to a new theory of 'Thermodynamic Limit' behavior for language and reasoning models, with the ultimate goal of Probably Approximately Correct (PAC)-like bounds for generalization of reasoning over time.
The paper also explores model merging by…
02 From the paper
Abstract
The relationship between computing systems and the brain has served as motivation for pioneering theoreticians since John von Neumann and Alan Turing. Uniform, scale-free biological networks, such as the brain, have powerful properties, including generalizing over time, which is the main barrier for Machine Learning on the path to Universal Reasoning Models. We introduce `Dragon Hatchling' (BDH), a new Large Language Model architecture based on a scale-free biologically inspired network of \$n\$ locally-interacting neuron particles. BDH couples strong theoretical foundations and inherent interpretability without sacrificing Transformer-like performance. BDH is a practical, performant state-of-the-art attention-based state space sequence learning architecture. In addition to being a graph model, BDH admits a GPU-friendly formulation. It exhibits Transformer-like scaling laws: empirically BDH rivals GPT2 performance on language and translation tasks, at the same number of parameters (10M to 1B), for the same training data. BDH can be represented as a brain model. The working memory of BDH during inference entirely relies on synaptic plasticity with Hebbian learning using spiking neurons. We confirm empirically that specific, individual synapses strengthen connection whenever BDH hears or reasons about a specific concept while processing language inputs. The neuron interaction network of BDH is a graph of high modularity with heavy-tailed degree distribution. The BDH model is biologically plausible, explaining one possible mechanism which human neurons could use to achieve speech. BDH is designed for interpretability. Activation vectors of BDH are sparse and positive. We demonstrate monosemanticity in BDH on language tasks. Interpretability of state, which goes beyond interpretability of neurons and model parameters, is an inherent feature of the BDH architecture.