XSemanticFlow: Cross Object Semantic Alignment for Zero-shot Manipulation

摘要

Establishing reliable correspondences across diverse objectinstances is fundamental to robust 3D understanding and generalizablemanipulation. However, existing methods are often inconsistent acrossgeometries or vulnerable to non-canonicalized poses, making correspon-dence transfer unreliable. To address this, we propose XSemanticFlow,which learns correspondences from pure semantic features. Given per-object semantic feature _x001C_elds, XSemanticFlow applies hierarchical intra-object self-attention and inter-object cross-attention to predict soft cor-respondence maps. By restricting cross-attention strictly to semantic to-kens, XSemanticFlow avoids geometric over_x001C_tting, enabling it to predictgeneralizable correspondences across di_x001B_erent topologies and producealigned feature _x001C_elds. This enables XSemanticFlow to learn e_x001B_ectivelythrough self-supervised cross-pose alignment on large-scale unlabeledshapes, followed by supervised cross-instance _x001C_netuning on a cosegmen-tation dataset containing labeled object pairs from PartNet. We eval-uate our method on both 3D understanding and manipulation tasks.For 3D understanding, XSemanticFlow improves self-segmentation withSE(3) augmentations by +19.1 mIoU and +20.6 accuracy, and cross-instance cosegmentation by +10.9 mIoU and +12.5 accuracy over base-lines, demonstrating stronger transform consistency and semantic align-ment. For manipulation, XSemanticFlow provides a reliable alternativeto recent video-generation-based pipelines for zero-shot manipulation.Rather than relying on synthesized visual plans, we directly transfer con-tact regions and end-e_x001B_ector trajectories from a single reference demon-stration to novel instances, bypassing the brittle video generation failurescommon in contact-critical tasks.