Paper 2510.00446
LongCodeZip: Compress Long Context for Code Language Models
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 39
- GitHub
- 163 stars
01 In brief
Summary
LongCodeZip is a training-free, model-agnostic, plug-and-play framework for compressing long code contexts for code LLMs.
It uses a dual-stage strategy: coarse-grained compression selects function-level chunks ranked by conditional perplexity relative to the instruction, and fine-grained compression segments retained functions into blocks via perplexity-based chunking, then selects an optimal subset under an adaptive token budget using a 0/1 knapsack formulation.
Evaluated on Long Code Completion, Long Module Summarization, and RepoQA, LongCodeZip consistently outperforms baselines (RAG, LLMLingua, DietCode, SlimCode) across models (DeepSeek-Coder-6.7B, Qwen2.5-Coder-7B, Seed-Coder-8B, GPT-4o, Claude-3.7-Sonnet), achieving up to 5.6x compression without performance loss.
Ablations show coarse-grained ranking is most critical, and fine-grained optimizations add gains.
It generalizes across models, even with a 0.5B compressor, and reduces generation time from 15.7s to 6.6s with 2.6s compression overhead.
02 From the paper
Abstract
Code generation under long contexts is becoming increasingly critical as Large Language Models (LLMs) are required to reason over extensive information in the codebase. While recent advances enable code LLMs to process long inputs, high API costs and generation latency remain substantial bottlenecks. Existing context pruning techniques, such as LLMLingua, achieve promising results for general text but overlook code-specific structures and dependencies, leading to suboptimal performance in programming tasks. In this paper, we propose LongCodeZip, a novel plug-and-play code compression framework designed specifically for code LLMs. LongCodeZip employs a dual-stage strategy: (1) coarse-grained compression, which identifies and ranks function-level chunks using conditional perplexity with respect to the instruction, retaining only the most relevant functions; and (2) fine-grained compression, which segments retained functions into blocks based on perplexity and selects an optimal subset under an adaptive token budget to maximize relevance. Evaluations across multiple tasks, including code completion, summarization, and question answering, show that LongCodeZip consistently outperforms baseline methods, achieving up to a 5.6x compression ratio without degrading task performance. By effectively reducing context size while preserving essential information, LongCodeZip enables LLMs to better scale to real-world, large-scale code scenarios, advancing the efficiency and capability of code intelligence applications.