AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion

Abstract

Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback, a capability that remains challenging to replicate in robotic systems. Existing robotic grasping approaches predominantly rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact, limiting robustness and generalization. To address this challenge, we propose a uni(cid:28)ed visuo-tactilefusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive re(cid:28)nement. At its core, our method introduces an e(cid:30)cient visuo-tactile representation that tightly fuses object geometry with tactile feedback by associating tactile signals with (cid:28)nger identities. This uni(cid:28)ed representation supports contact-aware grasp pose generation during planning and tactile-guided re(cid:28)nement after contact, enabling the system to reason about (cid:28)ne-grained (cid:28)nger-object interactions and adjust grasps dynamically. Comprehensive experiments in both simulation and real-world environments demonstrate that our approach signi(cid:28)cantly enhances grasp success rates and generalization across diverse objects. • •