Paper 2510.14975
WithAnyone: Towards Controllable and ID Consistent Image Generation
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 14
- GitHub
- 573 stars
01 In brief
Summary
The paper introduces WithAnyone, a diffusion-based model for controllable and identity-consistent image generation, addressing the 'copy-paste' artifact where models replicate reference faces instead of preserving identity across variations.
The authors construct MultiID-2M, a large-scale dataset with 500k group photos and paired references for 3k identities, and MultiID-Bench, a benchmark quantifying copy-paste artifacts and the fidelity-variation trade-off.
WithAnyone, built on FLUX, uses a four-phase training pipeline (reconstruction pre-training, caption supervision, paired tuning, quality tuning) and novel losses: a ground-truth-aligned ID loss and an ID contrastive loss with extended negatives.
Experiments show WithAnyone achieves state-of-the-art identity similarity (Sim(GT) 0.460 on single-person subset) while significantly reducing copy-paste (CP 0.144), breaking the trade-off seen in other models.
User studies confirm higher rankings in identity similarity, copy-paste reduction, prompt adherence, and aesthetics.
02 From the paper
Abstract
Identity-consistent generation has become an important focus in text-to-image research, with recent models achieving notable success in producing images aligned with a reference identity. Yet, the scarcity of large-scale paired datasets containing multiple images of the same individual forces most approaches to adopt reconstruction-based training. This reliance often leads to a failure mode we term copy-paste, where the model directly replicates the reference face rather than preserving identity across natural variations in pose, expression, or lighting. Such over-similarity undermines controllability and limits the expressive power of generation. To address these limitations, we (1) construct a large-scale paired dataset MultiID-2M, tailored for multi-person scenarios, providing diverse references for each identity; (2) introduce a benchmark that quantifies both copy-paste artifacts and the trade-off between identity fidelity and variation; and (3) propose a novel training paradigm with a contrastive identity loss that leverages paired data to balance fidelity with diversity. These contributions culminate in WithAnyone, a diffusion-based model that effectively mitigates copy-paste while preserving high identity similarity. Extensive qualitative and quantitative experiments demonstrate that WithAnyone significantly reduces copy-paste artifacts, improves controllability over pose and expression, and maintains strong perceptual quality. User studies further validate that our method achieves high identity fidelity while enabling expressive controllable generation.