General Speech Reconstruction for Voice Calls Using Smartphone's Built-in Accelerometer

摘要

Voice calls are a critical communication mode of interpersonal interaction. However, one of its most significant limitations is susceptibility to ambient noise interference. Current approaches to mitigate noise typically rely on advanced algorithmic processing or supplementary modal signals to enhance the original speech. Nevertheless, when environmental noise reduces the signal-to-noise ratio (SNR) of the original speech below a certain threshold, these methods become entirely ineffective. In this paper, we propose AccVoice, a novel approach that relies solely on the smartphone's builtin accelerometer to capture facial vibration signals, enabling a general speech reconstruction system for voice calls. We first propose a triaxial fusion method to eliminate variations in accelerometer data caused by user-induced micro-motion noise and enhance the feature representation of the signal. Then, we introduce for the first time, to the best of our knowledge, the concept of removing voice timbre to enable cross-user general speech reconstruction capabilities of accelerometer signals. Finally, by employing a large-scale pre-trained speech model combined with a spectrum-separation-based reconstruction approach, we achieve high-quality speech generation. Besides, we use the one-shot sample to re-infuse the timbre into the reconstructed speech to maintain the authenticity of voice calls. We conduct extensive experiments in a cross-user scenario. The results demonstrate that AccVoice achieves high-quality speech reconstruction across different impact factors.