Paper 2605.12357
$δ$-mem: Efficient Online Memory for Large Language Models
- Published
- May 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- 253 stars
01 In brief
Summary
δ-mem is a lightweight memory mechanism that augments a frozen full-attention LLM with a compact online state of associative memory, updated via delta-rule learning.
It compresses past information into a fixed-size state matrix (e.g., 8×8) and uses its readout to generate low-rank corrections to the backbone's attention computation, avoiding full fine-tuning or context extension.
Evaluated on Qwen3-4B-Instruct, Qwen3-8B, and SmolLM3-3B, δ-mem improves average scores by 1.10× over the frozen backbone and 1.15× over the strongest non-δ-mem baseline, with larger gains on memory-heavy benchmarks (1.31× on MemoryAgentBench, 1.20× on LoCoMo).
The paper explores three writing strategies (token-level, segment-level, multi-state) and ablations on injection heads and depth, showing that query-output corrections and all-layer insertion are effective.
δ-mem adds only 4.87M trainable parameters (0.12% of backbone) and maintains inference efficiency comparable to vanilla models, while recovering useful historical information even when explicit context is removed.
02 From the paper
Abstract
Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply expanding the context window is costly and often fails to ensure effective context utilization. We propose $δ$-mem, a lightweight memory mechanism that augments a frozen full-attention backbone with a compact online state of associative memory. $δ$-mem compresses past information into a fixed-size state matrix updated by delta-rule learning, and uses its readout to generate low-rank corrections to the backbone's attention computation during generation. With only an $8\times8$ online memory state, $δ$-mem improves the average score to $1.10\times$ that of the frozen backbone and $1.15\times$ that of the strongest non-$δ$-mem memory baseline. It achieves larger gains on memory-heavy benchmarks, reaching $1.31\times$ on MemoryAgentBench and $1.20\times$ on LoCoMo, while largely preserving general capabilities. These results show that effective memory can be realized through a compact online state directly coupled with attention computation, without full fine-tuning, backbone replacement, or explicit context extension.