Paper 2603.27538
LongCat-Next: Lexicalizing Modalities as Discrete Tokens
- Published
- Mar 2026
- Research lab
- Independent
- Citations
- 18
- GitHub
- 467 stars
01 In brief
Summary
LongCat-Next, developed by Meituan's LongCat team, introduces the Discrete Native Autoregressive (DiNA) paradigm, which unifies text, vision, and audio into a shared discrete token space, enabling a single autoregressive model to handle all modalities.
A key innovation is the Discrete Native Any-resolution Visual Transformer (dNaViT), which uses Semantic-and-Aligned Encoders (SAE) and Residual Vector Quantization (RVQ) to tokenize and de-tokenize images at arbitrary resolutions, achieving up to 28x compression while preserving semantic completeness.
The model, built on a 68.5B-parameter MoE backbone (LongCat-Flash-Lite A3B) and trained on over 2 trillion tokens, excels in visual understanding, generation, and audio tasks.
It outperforms unified models like Qwen3-Omni and specialized models like Qwen3VL-A3B on several benchmarks, and competes with Flux-dev in image generation.
The audio tokenizer, based on Whisper and RVQ, supports both parallel and serial text-guided speech generation.
The paper also discusses training dynamics, including a modality-agnostic MoE that develops functional specialization, and demonstrates that discrete modeling can close the performance gap with continuous approaches given sufficient data.
The model and tokenizers are open-sourced on GitHub and Hugging Face.
02 From the paper
Abstract
The prevailing Next-Token Prediction (NTP) paradigm has driven the success of large language models through discrete autoregressive modeling. However, contemporary multimodal systems remain language-centric, often treating non-linguistic modalities as external attachments, leading to fragmented architectures and suboptimal integration. To transcend this limitation, we introduce Discrete Native Autoregressive (DiNA), a unified framework that represents multimodal information within a shared discrete space, enabling a consistent and principled autoregressive modeling across modalities. A key innovation is the Discrete Native Any-resolution Visual Transformer (dNaViT), which performs tokenization and de-tokenization at arbitrary resolutions, transforming continuous visual signals into hierarchical discrete tokens. Building on this foundation, we develop LongCat-Next, a native multimodal model that processes text, vision, and audio under a single autoregressive objective with minimal modality-specific design. As an industrial-strength foundation model, it excels at seeing, painting, and talking within a single framework, achieving strong performance across a wide range of multimodal benchmarks. In particular, LongCat-Next addresses the long-standing performance ceiling of discrete vision modeling on understanding tasks and provides a unified approach to effectively reconcile the conflict between understanding and generation. As an attempt toward native multimodality, we open-source the LongCat-Next and its tokenizers, hoping to foster further research and development in the community. GitHub: https://github.com/meituan-longcat/LongCat-Next