Beyond the Boundary: RL-Driven Solution Space Exploration for Blind Face Restoration

Abstract

Blind Face Restoration (BFR) encounters inherent challengesin exploring its large solution space, leading to missing details and identityambiguity in deterministic outputs. While diffusion models offer powerfulgenerative priors, their potential is often constrained by deterministic sam-pling processes that lack exploration in the vast solution space. To tacklethis, we propose Likelihood-Regularized Policy Optimization (LRPO),the first online reinforcement learning (RL) framework that uniquelyleverages the inherent stochasticity of diffusion models for the BFR task.Instead of relying on a single deterministic mapping, LRPO systematicallyexplores multiple diverse restoration candidates per input. By evaluatingthese trajectories, our framework steers the denoising policy towardsoptimal solutions, significantly increasing the likelihood of high-qualityoutputs. To effectively facilitate this exploration mechanism, LRPO incor-porates three key innovations: 1) a composite reward function balancinghuman preference and fidelity, 2) ground-truth-guided likelihood regular-ization, and 3) a noise-level advantage assignment. Extensive experimentsdemonstrate that LRPO significantly improves face restoration qualityover baselines and achieves state-of-the-art performance. Source code isavailable at the GitHub repository BFR_RL.