Paper 2607.17423
TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 117 stars
01 In brief
Summary
TimeLens2 introduces a generalist video temporal grounding model that predicts variable-cardinality sets of evidence intervals across diverse video lengths, domains, query forms, and viewpoints.
It addresses two structural mismatches: unreliable long-video supervision and optimization that lacks interval-level geometry.
The TimeLens2-93K dataset pipeline uses hierarchical captions, dual-agent localization, cross-agent consensus, semantic verification, and boundary refinement to produce reliable multi-span labels.
A temporal Wasserstein reward computes exact 1D W1 distance over merged interval supports, providing dense, matching-free feedback that complements tIoU.
Across seven benchmarks, TimeLens2-2B, 4B, and 8B achieve average mIoU of 44.5, 47.7, and 48.0, respectively, outperforming all size-matched baselines and surpassing larger open-source models, including Qwen3.5-397B-A17B.
The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.
Ablations show the data pipeline and reward design each contribute significant gains, with the Wasserstein reward rescuing 75.8% of zero-tIoU GRPO groups and improving recovery of near misses by 21.9% versus 5.7% for far misses.
The work demonstrates that treating evidence as an interval set throughout supervision and optimization enables auditable, generalist temporal grounding in compact models.
Future work includes extending to spatiotemporal grounding for embodied intelligence.
02 From the paper
Abstract
Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.