OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

OmniLottie is a versatile framework for generating high-quality vector animations from multi-modal instructions (text, image, and video).

It addresses the challenge of raw Lottie JSON files containing extensive invariant structural metadata by introducing a Lottie tokenizer that converts JSON into compact sequences of commands and parameters.

This tokenizer enables building OmniLottie upon pretrained vision-language models (Qwen2.5-VL) to follow interleaved multi-modal instructions.

The authors also curate MMLottie-2M, a large-scale dataset of two million Lottie animations with paired text, images, and videos, and establish MMLottie-Bench for standardized evaluation.

Experiments show OmniLottie outperforms baselines (DeepSeek, GPT-5, Recraft, AniClipart, etc.) in visual fidelity and semantic alignment across all three tasks, achieving high success rates (up to 97.3%) and superior FVD, CLIP, object/motion alignment, PSNR, SSIM, and DINO scores.

Ablations confirm the benefits of the tokenizer and moderate SVG data mixing.

Limitations include occasional invalid sequences and challenges with complex animations, suggesting future work on constrained decoding or reinforcement learning with renderability rewards.

The dataset is released for research and non-commercial use only.

The framework's key contributions are the first end-to-end multi-modal vector animation generation framework, the MMLottie-2M dataset, the Lottie tokenizer, and state-of-the-art performance.

The tokenizer reduces token length by 81% compared to raw JSON, improving training efficiency and generation quality.

The model is trained with standard cross-entropy loss on the token sequences.

The benchmark includes real and synthetic subsets to avoid train-test overlap and contamination.

The user study confirms the effectiveness of the proposed metrics.

The failure analysis reveals that OmniLottie's failures are…