Beyond Linear Shortcuts: Rectifying Diffusion Preference Optimization with Intrinsic Generative Geometry

摘要

Direct Preference Optimization (DPO) has proven effectivefor aligning diffusion models with human preferences. However, by re-lying solely on endpoint supervision, existing DPO fine-tuning practicesignore the curved trajectory of the pretrained generation process. Thisforces a linear shortcut in the latent space, disregarding the intrinsicgeometry and leading to off-trajectory and off-manifold generation. Toaddress this, we propose Trajectory-Consistent Preference Optimization(TCPO), a fine-tuning paradigm respecting the model’s intrinsic genera-tive geometry. TCPO first performs Curvature-Adaptive Sampling (CAS)to dynamically select a curvature-aware intermediate latent as a geo-metric anchor. Then, Trajectory-Rectified Fusion (TRF) combines thisgeometric anchor with the conventional endpoint target which acts as asemantic anchor to form a trajectory-consistent supervision signal. Ex-tensive experiments show that TCPO achieves superior visual fidelityand text-to-image alignment on standard benchmarks, with improvedtraining efficiency. 4