Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding

Abstract

Current expressive avatar systems rely heavily on visual cuesand often fail when faces are occluded or emotions remain internal. Wepresent Mind-to-Face, the first framework to decode non-invasive elec-troencephalogram (EEG) signals directly into high-fidelity facial expres-sions. We build a dual-modality recording setup that captures synchro-nized EEG and multi-view facial video during emotion-eliciting stimuli,providing precise supervision for neural-to-visual learning. Our modeluses a CNN-Transformer encoder to map EEG signals into dense 3Dposition maps that sample over 65k vertices, capturing fine-scale geom-etry and subtle emotional dynamics, and renders them through a mod-ified 3D Gaussian Splatting pipeline for photorealistic, view-consistentresults. Extensive evaluations show that EEG alone can reliably predictdynamic, subject-specific facial expressions, including subtle emotionalresponses, demonstrating that neural signals contain far richer affectiveand geometric information than previously assumed. Mind-to-Face es-tablishes a new paradigm for neural-driven avatars, enabling personal-ized, emotion-aware telepresence and cognitive interaction in immersive