Paper 2607.23782
$N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 51 stars
01 In brief
Summary
N0-VTLA is a vision-tactile-language-action (VTLA) foundation model for contact-rich manipulation.
It introduces latent tactile tokens that predict future tactile changes over the action horizon, conditioning a flow-matching action expert, rather than using current tactile readings as observation context.
The model is pretrained on NeoData, a large-scale visuo-tactile robot dataset, using a three-stage recipe: grounding the predictor with contrastive and reconstruction losses, aligning the action expert with the latents, and end-to-end training.
For offline policy improvement, ALTER converts deployment data into advantage labels via a pairwise progress model, enabling advantage-conditioned offline RL.
N0-VTLA outperforms baselines on real-robot and simulation benchmarks, achieving 47.2% mean success on nine NeoReal tasks (vs.
29.4% for pi0.5) and 63.8% on a twenty-task simulation suite (vs.
44.0% for the strongest baseline).
With ALTER, it reaches 75-95% success on three long-horizon tasks.
The latent tactile tokens retrieve future-tactile targets at 92.3% top-1 accuracy, demonstrating anticipatory contact representation.
02 From the paper
Abstract
We present $N_0$-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, $N_0$-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, $N_0$-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. $N_0$-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.