LibraGen: Playing a Balance Game in Subject-Driven Video Generation

摘要

With the advancement of video generation foundation mod-els (VGFMs), customized generation, particularly subject-to-video (S2V),has attracted growing attention. However, a key challenge lies in balanc-ing the intrinsic priors of a VGFM, such as motion coherence, visualaesthetics, and prompt alignment, with its newly derived S2V capa-bility. Existing methods often neglect this balance by enhancing oneaspect at the expense of others. To address this, we propose Libra-Gen, a novel framework that views extending foundation models forS2V generation as a balance game between intrinsic VGFM strengthsand S2V capability. Specifically, guided by the core philosophy of “Rais-ing the Fulcrum, Tuning to Balance,” we identify data quality as thefulcrum and advocate a quality-over-quantity approach. We constructa hybrid pipeline that combines automated and manual data filteringto improve overall data quality. To further harmonize the VGFM’s na-tive capabilities with its S2V extension, we introduce a Tune-to-Balancepost-training paradigm. During supervised fine-tuning, both cross-pairand in-pair data are incorporated, and model merging is employed toachieve an effective trade-off. Subsequently, two tailored direct prefer-ence optimization (DPO) pipelines, namely Consis-DPO and Real-FakeDPO, are designed and merged to consolidate this balance. During in-ference, we introduce a time-dependent dynamic classifier-free guidancescheme to enable flexible and fine-grained control. Experimental resultsdemonstrate that LibraGen outperforms both open-source and commer-cial S2V models using only thousand-scale training samples. ProjectPage: https://github.com/Phantom-video/LibraGen