Differentiable Polarized Path Tracing

Abstract

Physically based differentiable rendering has proven to be apowerful tool for inverse rendering problems (e.g., 3D reconstruction, re-flectance estimation, lighting estimation). However, most existing meth-ods operate solely on radiometric intensity, discarding valuable polariza-tion cues that constrain scene geometry and material properties. Whileforward simulation of polarized light is well-defined via Mueller-Stokescalculus, extending reverse-mode differentiation to this domain presentssignificant challenges. The rank-deficient nature of common polarimet-ric operators, such as linear polarizers and diffuse reflections, violatesthe invertibility assumptions of standard gradient estimators like pathreplay backpropagation and results in numerical instability. We addressthis by proposing a robust, polarization-aware differentiable path tracingmethod. Our approach estimates unbiased gradients through a combina-tion of path replay and local caching. This formulation enables efficientand stable optimization of material and lighting parameters in complexscenes, broadening the applicability of physically based inverse rendering.Project page: https://vcai.mpi-inf.mpg.de/projects/DPPT/