Paper 2606.13681
EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 21 stars
01 In brief
Summary
The paper introduces EvoArena, a benchmark suite for evaluating LLM agents under persistent environment evolution, and EvoMem, a patch-based memory paradigm that records memory updates as structured histories.
EvoArena comprises three subsets: Terminal-Bench-Evo (evolving terminal workflows), SWE-Chain-Evo (evolving codebases), and PersonaMem-Evo (evolving user preferences).
Experiments show current agents struggle, with average step accuracy of 39.6% across domains.
EvoMem consistently improves performance, yielding average gains of 1.5% on EvoArena, 6.1% on GAIA, and 4.8% on LoCoMo, and improves chain-level accuracy by 3.7%.
Mechanistic analysis shows EvoMem improves evidence capture in memory, especially for temporal and dispersed preference signals, and reduces regression rates in software evolution.
The results highlight the importance of modeling evolution in both evaluation and memory for reliable agent deployment in dynamic real-world settings.
The paper also discusses efficiency-accuracy trade-offs, showing higher token usage does not guarantee better performance, and provides detailed construction and implementation details for both the benchmark and the memory paradigm.
The work aims to support future research on agents that can track changes, recover historical context, and adapt reliably in evolving environments.
02 From the paper
Abstract
Large language model (LLM) agents have achieved strong performance on a wide range of benchmarks, yet most evaluations assume static environments. In contrast, real-world deployment is inherently dynamic, requiring agents to continually align their knowledge, skills, and behavior with changing environments and updated task conditions. To address this gap, we introduce EvoArena, a benchmark suite that models environment changes as sequences of progressive updates across terminal, software, and social domains. We further propose EvoMem, a patch-based memory paradigm that records memory evolution as structured update histories, enabling agents to reason about environmental evolution through changes in their memory. Experiments show that current agents struggle on EvoArena, achieving an average accuracy of 39.6% across evolving terminal, software, and social-preference domains. EvoMem consistently improves performance, yielding an average gain of 1.5% on EvoArena and also improving standard benchmarks such as GAIA and LoCoMo by 6.1% and 4.8%. Beyond individual tasks, EvoMem further improves chain-level accuracy by 3.7% on EvoArena, where success requires completing a consecutive sequence of related evolutionary subtasks. Mechanistic analysis shows that EvoMem improves evidence capture in the memory, indicating better preservation of complete evolving environment states. Our results highlight the importance of modeling evolution in both evaluation and memory for reliable agent deployment.