Paper 2606.24775
Are We Ready For An Agent-Native Memory System?
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 7
- GitHub
- 129 stars
01 In brief
Summary
This paper presents a systematic experimental study of agent memory systems for LLM agents, decomposing them into four core modules: representation/storage, extraction, retrieval/routing, and maintenance.
The authors evaluate 12 representative memory systems and two baselines across five benchmark workloads (11 datasets), finding that no single architecture dominates; effectiveness depends on alignment with workload bottlenecks.
Key findings include: graph-based methods excel in factual recall but struggle with temporal reasoning; retrieval accuracy degrades with temporal distance; graph methods handle knowledge updates best; append-only stores suffer from catastrophic degradation over long horizons; and highly structured systems incur higher operational costs without proportional accuracy gains.
Fine-grained ablations show that preserving raw content is more important than abstraction, conservative consolidation is the best maintenance strategy, and explicit planning with balanced hybrid retrieval improves effectiveness.
The paper identifies promising directions for building agent-native memory systems and releases code and a testbed publicly.
The study emphasizes the need for evidence-centric organization, temporal update fidelity, and cost-efficient localized maintenance over global reorganization.
The authors conclude that memory systems should preserve task-critical evidence at the right abstraction level, and that stronger LLM backbones refine answers only after grounding has succeeded.
The work provides a taxonomy and framework for principled comparison and future research in agent memory data management.
The code is available at https://github.com/OpenDataBox/MemoryData.
The paper's findings highlight the importance of workload-aligned memory design, evidence-completion over top-1 ranking, and the trade-offs between structure and operational cost.
The authors also note that raw long-context retrieval can outperform memory-backed approaches for…
02 From the paper
Abstract
Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.