Paper 2606.26058
DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 165 stars
01 In brief
Summary
DomainShuttle is a novel framework for open-domain subject-driven text-to-video (S2V) generation, addressing both in-domain (high subject fidelity) and cross-domain (flexible adaptation of subject-irrelevant features) scenarios.
It introduces three key components: Domain-MoT, which decouples video and reference features and uses domain-aware AdaLN for domain-specific reference modeling; Video-Reference DualRoPE, which places reference and video tokens in separate RoPE spaces for precise subject-level spatial modeling; and Cross-Pair Consistent Loss, which aligns multiple reference sets to extract intrinsic subject features.
Trained on a 750K video dataset with a two-stage process on Wan2.1-14B and Wan2.2-14B, DomainShuttle outperforms existing methods, achieving an 18.7% improvement in Cross-Domain Score over state-of-the-art baselines.
Ablations confirm the effectiveness of each module, and human preference evaluations show superior performance in video quality, text controllability, and open-domain subject consistency.
02 From the paper
Abstract
Open domain subject-driven text-to-video (S2V) generation has drawn significant interest in academia and industry. Open domain S2V mainly involves two scenarios: in-domain, which requires retaining the reference subject features as much as possible, and cross-domain, which preserves the intrinsic features of the subject while allowing subject-irrelevant properties to vary flexibly according to the text prompt. Existing methods primarily focus on maximizing subject fidelity in in-domain scenarios, which limits their editability and adaptability in cross-domain scenarios, such as novel styles, semantic combinations, or domain attributes. In this study, we propose that an ideal S2V method should flexibly shuttle between different domains, achieving strong performance in both in-domain and cross-domain scenarios. To this end, we propose DomainShuttle, which could achieve high fidelity and generative flexibility for open domain video personalization. Specifically, we introduce Domain-MoT, which decouples videos and reference features and introduces the domain-aware AdaLN for domain-specific modeling of reference images. We then introduce the Video-Reference DualRoPE scheme, which places reference image tokens and video tokens in separate RoPE spaces to enable precise subject-level spatial modeling, and Cross-Pair Consistent Loss, which aims to extract intrinsic subject features unaffected by irrelevant features. Extensive experiments demonstrate that DomainShuttle achieves significant performance improvements over existing methods, exhibiting high subject fidelity and generative flexibility across diverse open domain application scenarios.