DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

Abstract

Multimodal deception detection aims to identify deceptivebehavior by analyzing audiovisual cues for forensics and security. Inthese high-stakes settings, investigators need verifiable evidence con-necting audiovisual cues to final decisions, along with reliable general-ization across domains and cultural contexts. However, existing bench-marks provide only binary labels without intermediate reasoning cues.Datasets are also small with limited scenario coverage, leading to short-cut learning. We address these issues through three contributions. First,we construct reasoning datasets by augmenting existing benchmarks withstructured cue-level descriptions and reasoning chains, enabling modelsto output auditable reports. Second, we release T4-Deception, a mul-ticultural dataset based on the unified “To Tell the Truth” televisionformat implemented across four countries. With 1695 samples, it is thelargest non-laboratory deception detection dataset. Third, we proposetwo modules for robust learning under small-data conditions. StabilizedIndividuality-Commonality Synergy (SICS) refines multimodal represen-tations by combining learnable global priors with sample-adaptive resid-uals and applying polarity-aware recalibration. Distilled Modality Con-sistency (DMC) aligns modality-specific predictions with the fused mul-timodal predictions via knowledge distillation to prevent unimodal short-cut learning. Experiments on three established benchmarks and our noveldataset demonstrate that our method achieves state-of-the-art perfor-mance in both in-domain and cross-domain scenarios, while exhibitingsuperior transferability across diverse cultural contexts. The datasets andcode are available at this link.