Render-FM: Feedforward Model for Real-time Photorealistic Volumetric Rendering

Abstract

Photorealistic volumetric rendering of CT scans greatly ben-e_x001C_ts clinical work_x001D_ows, yet neural approaches such as Neural RadianceFields (NeRF) and 3D Gaussian Splatting (3DGS) require prohibitiveper-scan optimization (hours for NeRF, about 30 minutes for 3DGS),making them impractical in clinical settings. We propose Render-FM,a feedforward model that eliminates this bottleneck by directly regress-ing 6D Gaussian Splatting (6DGS) parameters from a CT volume ina single 2.8-second forward pass, a 500× speedup over per-scan opti-mization. To bridge the domain gap between natural scene reconstruc-tion and medical volumetric rendering, we introduce Anatomy-GuidedPriming (AGP), which incorporates segmentation masks and transferfunctions as structural and appearance priors, information that exist-ing Gaussian splatting methods overlook. Built on an nnU-Net-inspired3D U-Net trained on diverse CT scans, Render-FM predicts per-voxel6DGS parameters and supports immediate real-time rendering. Unlikeper-scan methods, it generalizes to unseen anatomies, novel transferfunctions, and enables compositional organ visualization with zero ad-ditional preparation time. Optional 89-second _x001C_ne-tuning further im-proves quality, surpassing per-scan optimized baselines. Project page:https://gaozhongpai.github.io/renderfm/.