The Year in AI Papers
The year/Independent research

Paper 2603.14935

Video-CoE: Reinforcing Video Event Prediction via Chain of Events

Qile Su, Jing Tang, Rui Chen, Lei Sun, Xiangxiang Chu

Published
Mar 2026
Research lab
Independent
Citations
2
GitHub
Not linked

01 In brief

Summary

The paper introduces Video-CoE, a method to improve video event prediction (VEP) in multimodal large language models (MLLMs).

The authors first evaluate leading MLLMs on VEP benchmarks, identifying two main failure causes: lack of logical reasoning for future events and insufficient use of visual information.

To address these, they propose the Chain of Events (CoE) paradigm, which segments the video into a temporal event chain (each event with timestamps and description) to enhance visual grounding and logical reasoning.

They implement CoE via two training stages: CoE-SFT, which uses a small dataset generated by Qwen2.5-VL-72B to teach logical reasoning from video to future events, and CoE-GRPO, a reinforcement learning method that trains the model to construct event chains using rewards for correct event tags, length control, and cross-modal similarity between event descriptions and video clips.

Experiments on FutureBench and AVEP show that CoE-GRPO outperforms existing open-source and commercial MLLMs, achieving state-of-the-art results (e.g., 75.00 average on FutureBench with Qwen2.5-VL-7B).

Ablations confirm the effectiveness of the similarity reward and event chain length, and attention analysis shows increased visual token attention.

The method also performs well in open-set prediction evaluated by a judge model.

02 From the paper

Abstract

Despite advances in the application of MLLMs for various video tasks, video event prediction (VEP) remains relatively underexplored. VEP requires the model to perform fine-grained temporal modeling of videos and establish logical relationships between videos and future events, which current MLLMs still struggle with. In this work, we first present a comprehensive evaluation of current leading MLLMs on the VEP task, revealing the reasons behind their inaccurate predictions, including lack of logical reasoning ability for future events prediction and insufficient utilization of visual information. To address these challenges, we propose \textbf{C}hain \textbf{o}f \textbf{E}vents (\textbf{CoE}) paradigm, which constructs temporal event chains to implicitly enforce MLLM focusing on the visual content and the logical connections between videos and future events, incentivizing model's reasoning capability with multiple training protocols. Experimental results on public benchmarks demonstrate that our method outperforms both leading open-source and commercial MLLMs, establishing a new state-of-the-art on the VEP task. Codes and models will be released soon.