OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation

Abstract

In this work, we study **Human-Object Interaction Video Generation (HOIVG)**, which aims to synthesize high-quality human-object interaction videos conditioned on text, reference images, audio, and pose. We present **OmniShow**, the first all-in-one model tailored for this practical yet challenging task, capable of harmonizing multimodal conditions and delivering industry-grade performance. To overcome the trade-off between controllability and quality, we introduce **Unified Channel-wise Conditioning** for efficient image and pose injection, and **Gated Local-Context Attention** to ensure precise audio-visual synchronization. To effectively address data scarcity, we develop a **Decoupled-Then-Joint Training** strategy that leverages a multi-stage training process with model merging to efficiently harness heterogeneous sub-task datasets. Furthermore, to fill the evaluation gap in this field, we establish HOIVG-Bench, a dedicated and comprehensive benchmark for HOIVG. Extensive experiments demonstrate that OmniShow achieves overall state-of-the-art performance across various multimodal conditioning settings, setting a solid standard for the emerging HOIVG task.