Paper 2511.08633
Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
- Published
- Nov 2025
- Research lab
- Independent
- Citations
- 11
- GitHub
- 370 stars
01 In brief
Summary
Time-to-Move (TTM) is a training-free, plug-and-play framework for motion- and appearance-controlled video generation using image-to-video (I2V) diffusion models.
It uses crude reference animations (e.g., cut-and-drag or depth-based reprojection) as motion cues, adapting SDEdit's noise injection to video.
To preserve appearance, it anchors generation to the input image via image conditioning.
A novel dual-clock denoising process assigns different noise levels to different regions: strong alignment in user-specified motion regions and weaker alignment elsewhere, enabling spatially varying conditioning without retraining.
TTM is architecture-agnostic and adds no runtime cost.
Experiments on MC-Bench (object motion) and DL3DV (camera motion) show TTM matches or exceeds training-based baselines like DragAnything, MotionPro, and Go-With-the-Flow (GWTF) in realism and motion control.
It also enables joint motion and appearance control, such as changing an object's color while moving it, a capability beyond text-only prompting.
TTM works with SVD, CogVideoX, and WAN 2.2 backbones.
Limitations include needing to tune (t_weak, t_strong) and requiring full object masks.
02 From the paper
Abstract
Diffusion-based video generation can create realistic videos, yet existing image- and text-based conditioning fails to offer precise motion control. Prior methods for motion-conditioned synthesis typically require model-specific fine-tuning, which is computationally expensive and restrictive. We introduce Time-to-Move (TTM), a training-free, plug-and-play framework for motion- and appearance-controlled video generation with image-to-video (I2V) diffusion models. Our key insight is to use crude reference animations obtained through user-friendly manipulations such as cut-and-drag or depth-based reprojection. Motivated by SDEdit's use of coarse layout cues for image editing, we treat the crude animations as coarse motion cues and adapt the mechanism to the video domain. We preserve appearance with image conditioning and introduce dual-clock denoising, a region-dependent strategy that enforces strong alignment in motion-specified regions while allowing flexibility elsewhere, balancing fidelity to user intent with natural dynamics. This lightweight modification of the sampling process incurs no additional training or runtime cost and is compatible with any backbone. Extensive experiments on object and camera motion benchmarks show that TTM matches or exceeds existing training-based baselines in realism and motion control. Beyond this, TTM introduces a unique capability: precise appearance control through pixel-level conditioning, exceeding the limits of text-only prompting. Visit our project page for video examples and code: https://time-to-move.github.io/.