RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion

摘要

Humans learn locomotion through visual observation, inter-preting visual content first before imitating actions. However, state-of-the-art humanoid locomotion systems rely on either curated motion capturetrajectories or sparse text commands, leaving a critical gap between visualunderstanding and control. Text-to-motion methods suffer from semanticsparsity and staged pipeline errors, while video-based approaches onlyperform mechanical pose mimicry without genuine visual understanding.We propose RoboMirror, the first retargeting-free video-to-locomotionframework embodying “understand before you imitate”. Leveraging VLMs,it distills raw egocentric/third-person videos into visual motion intents,which directly condition a diffusion-based policy to generate physicallyplausible, semantically aligned locomotion without explicit pose recon-struction or retargeting. Extensive experiments validate RoboMirror’seffectiveness, it enables telepresence via egocentric videos, drasticallyreduces third-person control latency by 80%, and achieves a 3.7% highertask success rate than baselines. By reframing humanoid control aroundvideo understanding, we bridge the visual understanding and action gap.