The Year in AI Papers
The year/Independent research

Paper 2602.03152

FASA: Frequency-aware Sparse Attention

Yifei Wang, Yueqi Wang, Zhenrui Yue, Huimin Zeng, Yong Wang, Ismini Lourentzou, Zhengzhong Tu, Xiangxiang Chu, Julian McAuley

Published
Feb 2026
Research lab
Independent
Citations
5
GitHub
Not linked

01 In brief

Summary

FASA is a training-free framework for query-aware token eviction in LLMs, addressing KV cache memory bottlenecks.

It exploits functional sparsity in RoPE frequency chunks (FCs), where a small subset of 'dominant' FCs shows high contextual agreement with full attention heads.

FASA uses a two-stage process: Token Importance Prediction (TIP) selects critical tokens using dominant FCs, and Focused Attention Computation (FAC) performs full attention on this subset.

It achieves near-oracle performance on long-context tasks, with FASA-M providing 8x memory compression and FASA-C delivering 2.56x speedup.

The method is robust across models and tasks, and extends to non-RoPE architectures like ALiBi and Partial-RoPE.

FASA is orthogonal to other KV cache compression methods, such as PyramidKV, and maintains output lengths similar to full KV cache.

Experiments on LongBench, PG-19, WikiText, C4, MATH500, and AIME24 show FASA consistently outperforms baselines like Stream, SnapKV, and Quest, with minimal performance loss compared to full KV cache.

The dominant FCs are identified via one-time offline calibration, which is task-agnostic and robust to calibration data choice.

FASA's efficiency stems from reduced memory I/O and computational complexity, making it suitable for long-context inference.

The method is validated across multiple LLMs, including Llama, Mistral, and Qwen, and demonstrates strong performance in long-CoT reasoning tasks.

FASA's design respects the indivisibility of FCs and uses FC scores as a ranking proxy, not a substitute for attention weights.

The framework is compatible with various position encodings and can be integrated with other optimization techniques, offering a practical solution for efficient long-context LLM deployment.

02 From the paper

Abstract

The deployment of Large Language Models (LLMs) faces a critical bottleneck when handling lengthy inputs: the prohibitive memory footprint of the Key Value (KV) cache. To address this bottleneck, the token pruning paradigm leverages attention sparsity to selectively retain a small, critical subset of tokens. However, existing approaches fall short, with static methods risking irreversible information loss and dynamic strategies employing heuristics that insufficiently capture the query-dependent nature of token importance. We propose FASA, a novel framework that achieves query-aware token eviction by dynamically predicting token importance. FASA stems from a novel insight into RoPE: the discovery of functional sparsity at the frequency-chunk (FC) level. Our key finding is that a small, identifiable subset of "dominant" FCs consistently exhibits high contextual agreement with the full attention head. This provides a robust and computationally free proxy for identifying salient tokens. Building on this insight, FASA first identifies a critical set of tokens using dominant FCs, and then performs focused attention computation solely on this pruned subset. Across a spectrum of long-context tasks, from sequence modeling to complex CoT reasoning, FASA consistently outperforms all token-eviction baselines and achieves near-oracle accuracy, demonstrating remarkable robustness even under constraint budgets. Notably, on LongBench-V1, FASA reaches nearly 100\% of full-KV performance when only keeping 256 tokens, and achieves 2.56$\times$ speedup using just 18.9\% of the cache on AIME24.