Paper 2510.03215
Cache-to-Cache: Direct Semantic Communication Between Large Language Models
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 35
- GitHub
- 425 stars
01 In brief
Summary
This paper introduces Cache-to-Cache (C2C), a new paradigm for direct semantic communication between large language models (LLMs) that bypasses text-based interaction.
C2C projects and fuses the KV-cache of a source model (Sharer) into that of a target model (Receiver) using a neural cache fuser with a learnable gating mechanism.
Oracle experiments showed that enriching KV-cache semantics improves accuracy without increasing cache length, and that caches are convertible across models.
C2C outperforms individual models by 6.4-14.2% average accuracy and text-to-text (T2T) communication by 3.1-5.4%, while achieving an average 2.5x latency speedup.
Experiments across benchmarks and model families (Qwen, Llama, Gemma) demonstrated consistent gains, including in strong-to-weak settings.
Ablations confirmed that gains come from complementary semantics of heterogeneous Sharers, not just added capacity.
Limitations include performance degradation with much weaker Sharers and open challenges in scaling to multiple models.
02 From the paper
Abstract
Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.