Conditional StyleGAN for Grasp Generation

Abstract

We present an approach based on conditional generative adversarial networks (GANs) to generate grasps directly and in a feed-forward manner from a raw depth image input. Building on the recently introduced StyleGAN architecture we extend results from an earlier proof-of-concept paper [1] and demonstrate successful sim2real transfer of grasp outputs for a robot arm with a Shadow Dexterous Hand. We find that the GAN model, which was only trained on a limited set of primitive objects, was able to generalize to a range of everyday real-world objects that differed significantly from the primitive objects used in simulation training. In contrast to discriminative models, the approach learns a latent representation in the set of feasible grasps that can be used for navigation in grasp space and thus allows smooth integration with other motion planning tools.