Paper 2604.22554
Video Analysis and Generation via a Semantic Progress Function
- Published
- Apr 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 38 stars
01 In brief
Summary
This paper introduces the Semantic Progress Function (SPF), a one-dimensional representation that captures how semantic meaning evolves over time in video sequences.
The SPF is computed by measuring pairwise semantic distances between frames using a pretrained embedder (SigLIP) and fitting a smooth curve via regularized least squares.
Departures from a straight line reveal uneven semantic pacing, such as long static stretches followed by abrupt jumps.
The authors propose semantic linearization, which reparameterizes the sequence to achieve constant semantic change.
For generated videos, this is done by warping temporal positional embeddings (RoPE) with frequency-aware and timestep-dependent modulation, applied iteratively at inference time without retraining.
For existing videos, the SPF is segmented into linear components, and intermediate clips are regenerated using models like Wan2.2 or LTX-2.
Experiments show the method improves pacing (88% user preference) while preserving visual fidelity (VBench metrics within one standard deviation).
The framework also supports arbitrary target pacing functions and is model-agnostic.
Limitations include sensitivity to camera motion and lighting changes, and potential quality degradation with excessive iterations.
Future work includes motion-aware embeddings and disentangling semantic factors.
02 From the paper
Abstract
Transformations produced by image and video generation models often evolve in a highly non-linear manner: long stretches where the content barely changes are followed by sudden, abrupt semantic jumps. To analyze and correct this behavior, we introduce a Semantic Progress Function, a one-dimensional representation that captures how the meaning of a given sequence evolves over time. For each frame, we compute distances between semantic embeddings and fit a smooth curve that reflects the cumulative semantic shift across the sequence. Departures of this curve from a straight line reveal uneven semantic pacing. Building on this insight, we propose a semantic linearization procedure that reparameterizes (or retimes) the sequence so that semantic change unfolds at a constant rate, yielding smoother and more coherent transitions. Beyond linearization, our framework provides a model-agnostic foundation for identifying temporal irregularities, comparing semantic pacing across different generators, and steering both generated and real-world video sequences toward arbitrary target pacing.