Whence the Voice? Self-supervised Dual-source Audio-Visual Localisation via Selective Convergence

摘要

Localising multiple sound sources in visual scenes remains afundamental challenge in multimodal perception due to an inherent cir-cular dependency: separating mixed audio requires knowing source loca-tions, while identifying sound-producing regions requires separated audiosignals. In this paper, we focus on the dual-source setting and discover aselective convergence in self-supervised audio-visual learning: when pre-sented with multiple sound sources, contrastive models naturally con-verge to the most salient audio-visual correspondence rather than at-tempting to represent all sources equally. This emergent phenomenon,analogous to human selective auditory attention, enables us to breakthe above circular dependency through a progressive two-stage frame-work: first, leveraging selective convergence to identify dominant sources,and then exploiting these learned priors to uncover remaining sources.Our self-supervised approach achieves the best performance among self-supervised methods on dual-source benchmarks without requiring anymanual annotations, and even surpasses some weakly-supervised ap-proaches on certain metrics. Furthermore, we identify a fundamentalevaluation inconsistency in existing benchmarks: comparing continuouslocalisation heatmaps against bounding-box annotations creates system-atic biases, particularly for non-axis-aligned objects where the boundingbox includes substantial background regions. To address this, we intro-duce pixel-level segmentation masks to the existing benchmark, enablingspatially-aligned evaluation. Together, these results suggest that embrac-ing rather than suppressing selectivity offers a scalable, annotation-freeroute to multi-source localisation.