Paper 2510.18866
LightMem: Lightweight and Efficient Memory-Augmented Generation
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 111
- GitHub
- 1.1K stars
01 In brief
Summary
LightMem is a lightweight and efficient memory-augmented generation system for Large Language Models (LLMs), presented at ICLR 2026.
It addresses inefficiencies in existing memory systems by drawing inspiration from the Atkinson-Shiffrin human memory model, organizing memory into three stages: sensory memory (pre-compression and topic segmentation), topic-aware short-term memory (STM), and long-term memory (LTM) with sleep-time updates.
The system filters redundant tokens, groups related utterances by topic, and decouples memory consolidation from online inference via offline parallel updates.
On LongMemEval and LoCoMo benchmarks, using GPT-4o-mini, Qwen3-30B-A3B-Instruct-2507, and GLM-4.6 backbones, LightMem consistently outperforms strong baselines (e.g., A-MEM, Mem0, MemoryOS), improving QA accuracy by up to 7.7% (LongMemEval) and 29.3% (LoCoMo), while reducing total token usage by up to 38x and 20.9x, API calls by up to 30x and 55.5x, and runtime by up to 12.4x and 8.2x, respectively.
Online-only test-time costs are even lower, achieving up to 106x/117x token reduction and 159x/310x fewer API calls.
Ablation studies confirm the importance of topic segmentation and pre-compression, and case studies show sleep-time consolidation mitigates information loss.
The code is available at https://github.com/zjunlp/LightMem.
02 From the paper
Abstract
Despite their remarkable capabilities, Large Language Models (LLMs) struggle to effectively leverage historical interaction information in dynamic and complex environments. Memory systems enable LLMs to move beyond stateless interactions by introducing persistent information storage, retrieval, and utilization mechanisms. However, existing memory systems often introduce substantial time and computational overhead. To this end, we introduce a new memory system called LightMem, which strikes a balance between the performance and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of human memory, LightMem organizes memory into three complementary stages. First, cognition-inspired sensory memory rapidly filters irrelevant information through lightweight compression and groups information according to their topics. Next, topic-aware short-term memory consolidates these topic-based groups, organizing and summarizing content for more structured access. Finally, long-term memory with sleep-time update employs an offline procedure that decouples consolidation from online inference. On LongMemEval and LoCoMo, using GPT and Qwen backbones, LightMem consistently surpasses strong baselines, improving QA accuracy by up to 7.7% / 29.3%, reducing total token usage by up to 38x / 20.9x and API calls by up to 30x / 55.5x, while purely online test-time costs are even lower, achieving up to 106x / 117x token reduction and 159x / 310x fewer API calls. The code is available at https://github.com/zjunlp/LightMem.