Paper 2512.23959
HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modeling
- Published
- Dec 2025
- Research lab
- Independent
- Citations
- 4
- GitHub
- 131 stars
01 In brief
Summary
HGMEM is a hypergraph-based working memory system for multi-step retrieval-augmented generation (RAG) that improves complex relational modeling in long contexts.
Unlike existing memory mechanisms that passively store isolated facts, HGMEM represents memory as a hypergraph where hyperedges serve as memory points, enabling the progressive formation of high-order correlations through update, insertion, and merging operations.
It uses adaptive memory-based evidence retrieval, combining local investigation and global exploration.
Evaluated on generative sense-making QA (LongBench V2) and long narrative understanding benchmarks (NarrativeQA, NoCha, Prelude), HGMEM consistently outperforms strong baselines, including DeepRAG, ComoRAG, GraphRAG, and LightRAG.
Notably, HGMEM with Qwen2.5-32B-Instruct matches or exceeds baselines using GPT-4o.
Ablation studies show that merging operations are critical for building high-order correlations, and analysis confirms that sense-making queries benefit from hyperedges connecting more entities.
HGMEM also shows moderate sensitivity to offline graph quality, maintaining advantages even with ablated or LLM-free graphs.
02 From the paper
Abstract
Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs) on tasks that demand global comprehension and intensive reasoning. Although many RAG systems incorporate a working memory to consolidate information, existing designs primarily function as a passive storage for isolated facts. This static nature overlooks crucial high-order correlations among primitive facts, thereby limiting models' capacity for multi-step reasoning and resulting in fragmented reasoning and weak global sense-making within extended contexts. We introduce HGMem, a hypergraph-based working memory system, extending the concept of memory beyond simple storage into a dynamic, expressive structure for complex reasoning and global understanding. In our approach, memory is represented as a hypergraph where hyperedges correspond to distinct memory units, enabling the progressive formation of high-order interactions within memory. This mechanism connects facts and thoughts around the focal problem, evolving the memory into an integrated and situated knowledge structure that provides strong propositions for deeper reasoning. We evaluate HGMem on several challenging global sense-making benchmarks. Extensive experiments and in-depth analyses demonstrate that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse datasets.