Paper 2602.18283
HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation
- Published
- Feb 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- Not linked
01 In brief
Summary
HyTRec is a generative recommendation framework designed to efficiently model ultra-long user behavior sequences.
It addresses the trade-off between linear attention's efficiency and softmax attention's precision by using a hybrid architecture: a linear attention branch (Temporal-Aware Delta Network, TADN) processes long-term history, while a softmax attention branch handles recent interactions.
TADN incorporates a temporal decay factor to upweight fresh signals and suppress historical noise.
Experiments on Amazon datasets show HyTRec outperforms baselines, achieving over 8% improvement in Hit Rate for users with long sequences while maintaining linear inference speed.
Ablation studies confirm the contribution of both branches, and a 3:1 ratio of linear to softmax layers provides the best efficiency-performance balance.
The model also shows robustness in cold-start scenarios and cross-domain transfer.
02 From the paper
Abstract
Modeling long sequences of user behaviors has emerged as a critical frontier in generative recommendation. However, existing solutions face a dilemma: linear attention mechanisms achieve efficiency at the cost of retrieval precision due to limited state capacity, while softmax attention suffers from prohibitive computational overhead. To address this challenge, we propose HyTRec, a model featuring a Hybrid Attention architecture that explicitly decouples long-term stable preferences from short-term intent spikes. By assigning massive historical sequences to a linear attention branch and reserving a specialized softmax attention branch for recent interactions, our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions. To mitigate the lag in capturing rapid interest drifts within the linear layers, we furthermore design Temporal-Aware Delta Network (TADN) to dynamically upweight fresh behavioral signals while effectively suppressing historical noise. Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines, notably delivering over 8% improvement in Hit Rate for users with ultra-long sequences with great efficiency.