The Year in AI Papers
The year/Independent research

Paper 2607.18217

HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement

Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang, Cheng Chen, Wen Zhou, Zhenbang Sun, Wei Xue, Wenhan Luo, Yike Guo

Published
Jul 2026
Research lab
Independent
Citations
0
GitHub
166 stars

01 In brief

Summary

HOMIE is a framework for human-object centric video personalization (HOCVP) that unifies inter-subject (distinct subjects) and intra-subject (multiple references of the same subject) personalization.

It addresses limitations of existing methods by integrating Multimodal Large Language Models (MLLMs) while preserving the text encoder, avoiding costly re-alignment.

Key components include Global Multimodal Guidance (GMG), which injects MLLM-derived semantic features into video tokens during self-attention via affine transformations, and Modality-Reference Embedding (MRE), which distinguishes token modalities and binds intra-subject references.

Training uses a multi-stage pipeline on curated datasets (300K single-subject, 100K multi-subject, 40K high-resolution samples) and is based on Wan2.1-14B and Wan2.2-14B.

Experiments show state-of-the-art performance across metrics including video quality, text following, and subject consistency, with notable improvements in OCR accuracy (21.8% relative over SkyReels-V3) and multi-view consistency.

Ablations confirm the effectiveness of GMG and MRE, and a user study shows HOMIE outperforms baselines in overall quality, text following, and subject consistency.

02 From the paper

Abstract

Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-alignment. Specifically, we introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens. Furthermore, we propose modality-reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra-subject reference image tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/