PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation

摘要

Recent advances in text-to-video (T2V) generation have achieved good visual quality, yet synthesizing videos that faithfully followphysical laws remains an open challenge. Existing methods mainly basedon graphics or prompt extension struggle to generalize beyond simplesimulated environments or learn implicit physics reasoning. The scarcityof training data with rich physics interactions and phenomena is alsoa problem. In this paper, we first introduce a Physics-Augmented videodata construction Pipeline, PhyAugPipe, that leverages a vision–languagemodel (VLM) with chain-of-thought reasoning to collect a training dataset,PhyVidGen-135K. Then we formulate a principled Physics-aware Group-wise Direct Preference Optimization, PhyGDPO, framework that usesreal-world video as winning case to guarantee correct physics learningand builds upon the groupwise Plackett–Luce probabilistic model tocapture holistic preferences beyond pairwise comparisons. We design aPhysics-Guided Rewarding (PGR) scheme that uses VLM-based physicalrewards to direct the optimization to focus on challenging physics. Plus,we propose a LoRA-Switch Reference (LoRA-SR) scheme that avoidsfull-model duplication as reference for efficient training. Experimentsshow that our method outperforms state-of-the-art methods. Our codeand data is at https://github.com/caiyuanhao1998/Open-PhyGDPO