ICRA 2018 Papers

Simulations are attractive environments for training agents as they provide an abundant source of data and alleviate certain safety concerns during the training process. But the behaviours developed by agents in simulation are often specific to the characteristics of the simulator. Due to modeling error, strategies that are successful in simulation may not transfer to their real world counterparts. In this paper, we demonstrate a simple method to bridge this “reality gap”. By randomizing the dynamics of the simulator during training, we are able to develop policies that are capable of adapting to very different dynamics, including ones that differ significantly from the dynamics on which the policies were trained. This adaptivity enables the policies to generalize to the dynamics of the real world without any training on the physical system. Our approach is demonstrated on an object pushing task using a robotic arm. Despite being trained exclusively in simulation, our policies are able to maintain a similar level of performance when deployed on a real robot, reliably moving an object to a desired location from random initial configurations. We explore the impact of various design decisions and show that the resulting policies are robust to significant calibration error.

We present the Signature of Topologically Persistent Points (STPP), a global descriptor that encodes topological invariants of 3D point cloud data. These topological invariants include the zeroth and first homology groups and are computed using persistent homology, a method for finding the features of a topological space at different spatial resolutions. STPP is a competitive 3D point cloud descriptor when compared to the state of art and is resilient to noisy sensor data. We demonstrate experimentally on a publicly available RGB-D dataset that STPP can be used as a distinctive signature, thus allowing for 3D point cloud processing tasks such as object detection and classification.

Learning instead of designing robot controllers can greatly reduce engineering effort required, while also emphasizing robustness. Despite considerable progress in simulation, applying learning directly in hardware is still challenging, in part due to the necessity to explore potentially unstable parameters. We explore the concept of shaping the reward landscape with training wheels; temporary modifications of the physical hardware that facilitate learning. We demonstrate the concept with a robot leg mounted on a boom learning to hop fast. This proof of concept embodies typical challenges such as instability and contact, while being simple enough to empirically map out and visualize the reward landscape. Based on our results we propose three criteria for designing effective training wheels for learning in robotics. A video synopsis can be found at https://youtu.be/6iH5E3LrYh8.

This paper considers the problem of completing assemblies of passive objects in nonconvex environments, cluttered with convex obstacles of unknown position, shape and size that satisfy a specific separation assumption. A differential drive robot equipped with a gripper and a LIDAR sensor, capable of perceiving its environment only locally, is used to position the passive objects in a desired configuration. The method combines the virtues of a deliberative planner generating high-level, symbolic commands, with the formal guarantees of convergence and obstacle avoidance of a reactive planner that requires little onboard computation and is used online. The validity of the proposed method is verified both with formal proofs and numerical simulations.

Micro Aerial Vehicles have the potential to assist humans in real life tasks involving applications such as smart homes, search and rescue, and architecture construction. To enhance autonomous navigation capabilities these vehicles need to be able to create dense 3D maps of the environment, while concurrently estimating their own motion. In this paper, we are particularly interested in small vehicles that can navigate cluttered indoor environments. We address the problem of visual inertial state estimation, control and 3D mapping on platforms with Size, Weight, And Power (SWAP) constraints. The proposed approach is validated through experimental results on a 250 g, 22 cm diameter quadrotor equipped only with a stereo camera and an IMU with a computationally-limited CPU showing the ability to autonomously navigate, while concurrently creating a 3D map of the environment.

This paper reports a robotic laparoscopic surgery system performing electro-surgery on porcine cadaver kidney, and evaluates its accuracy in an open loop control scheme to conduct targeting and cutting tasks guided by a novel 3D endoscope. We describe the design and integration of the novel laparoscopic imaging system that is capable of reconstructing the surgical field using structured light. A targeting task is first performed to determine the average positioning error of the system as guided by the laparoscopic camera. The imaging system is then used to reconstruct the surface of a porcine cadaver kidney, and generate a cutting trajectory with consistent depth. The paper concludes by using the robotic system in open loop control to cut this trajectory using a multi degree of freedom electro-surgical tool. It is demonstrated that for a cutting depth of 3 mm, the robotic surgical system follows the trajectory with an average depth of 2.44 mm and standard deviation of 0.34 mm. The average positional accuracy of the system was 2.74±0.99 mm.

We present our approach for robotic perception in cluttered scenes that led to winning the recent Amazon Robotics Challenge (ARC) 2017. Next to small objects with shiny and transparent surfaces, the biggest challenge of the 2017 competition was the introduction of unseen categories. In contrast to traditional approaches which require large collections of annotated data and many hours of training, the task here was to obtain a robust perception pipeline with only few minutes of data acquisition and training time. To that end, we present two strategies that we explored. One is a deep metric learning approach that works in three separate steps: semantic-agnostic boundary detection, patch classification and pixel-wise voting. The other is a fully-supervised semantic segmentation approach with efficient dataset collection. We conduct an extensive analysis of the two methods on our ARC 2017 dataset. Interestingly, only few examples of each class are sufficient to fine-tune even very deep convolutional neural networks for this specific task.

We present the Semantic Robot Programming (SRP) paradigm as a convergence of robot programming by demonstration and semantic mapping. In SRP, a user can directly program a robot manipulator by demonstrating a snapshot of their intended goal scene in workspace. The robot then parses this goal as a scene graph comprised of object poses and inter-object relations, assuming known object geometries. Task and motion planning is then used to realize the user's goal from an arbitrary initial scene configuration. Even when faced with different initial scene configurations, SRP enables the robot to seamlessly adapt to reach the user's demonstrated goal. For scene perception, we propose the Discriminatively-Informed Generative Estimation of Scenes and Transforms (DIGEST) method to infer the initial and goal states of the world from RGBD images. The efficacy of SRP with DIGEST perception is demonstrated for the task of tray-setting with a Michigan Progress Fetch robot. Scene perception and task execution are evaluated with a public household occlusion dataset and our cluttered scene dataset.

This paper presents a purely visual semantic mapping framework using omnidirectional images. The approach rests upon the robust segmentation of the robot's local free space, replacing conventional range sensors for the generation of occupancy grid maps. The perceptions are mapped into a bird's eye view allowing an inverse sensor model directly by removing the non-linear distortions of the omnidirectional camera mirror. The system relies on a place category classifier to label the navigation relevant categories: room, corridor, doorway, and open room. Each place class maintains a separated grid map that are fused with the range-based occupancy grid for building a dense semantic map.

We present an approach to automatically assign semantic labels to rooms reconstructed from 3D RGB maps of apartments. Evidence for the room types is generated using state-of-the-art deep-learning techniques for scene classification and object detection based on automatically generated virtual RGB views, as well as from a geometric analysis of the map's 3D structure. The evidence is merged in a conditional random field, using statistics mined from different datasets of indoor environments. We evaluate our approach qualitatively and quantitatively and compare it to related methods.

Modern unmanned aerial vehicles (UAVs) rely on constant periodic sensor measurements to detect and avoid obstacles. However, constant checking and replanning are time and energy consuming and are often not necessary especially in situations in which the UAV can safely fly in uncluttered environments without entering unsafe states. Thus, in this paper, we propose a self-triggered framework that leverages reachability analysis to schedule the next time to check sensor measurements and perform replanning while guaranteeing safety under noise and disturbance effects. Further, we relax sensor checking and motion replanning operations by leveraging a risk-based analysis that determines the likelihood to reach undesired states over a certain time horizon. We also propose an online speed adaptation policy based on the planned trajectory curvature to minimize drift from the desired path due to the system dynamics. Finally, we validate the proposed approach with simulations and experiments for a quadrotor UAV motion planning case study in a cluttered environment.

Enabling robots to autonomously navigate complex environments is essential for real-world deployment. Prior methods approach this problem by having the robot maintain an internal map of the world, and then use a localization and planning method to navigate through the internal map. However, these approaches often include a variety of assumptions, are computationally intensive, and do not learn from failures. In contrast, learning-based methods improve as the robot acts in the environment, but are difficult to deploy in the real-world due to their high sample complexity. To address the need to learn complex policies with few samples, we propose a generalized computation graph that subsumes value-based model-free methods and model-based methods, with specific instantiations interpolating between model-free and model-based. We then instantiate this graph to form a navigation model that learns from raw images and is sample efficient. Our simulated car experiments explore the design decisions of our navigation model, and show our approach outperforms single-step and N-step double Q-learning. We also evaluate our approach on a real-world RC car and show it can learn to navigate through a complex indoor environment with a few hours of fully autonomous, self-supervised training. Videos of the experiments and code can be found at github.com/gkahn13/gcg.

Due to high impact forces and low duty cycles, monopedal jumping robots are particularly susceptible to failure from a slipping foot. Spines provide a solution to reduce slip, but there has been little research on how to effectively engage them into a surface with a dynamic jumping robot. Previous robots utilizing spines operate in different regimes of surface approach speed and cycle time. For a penetrable substrate, spines must be directed into the surface at suitable holding angles, then extracted before the foot leaves the ground. We accomplished this by designing a gripper mechanism for the robot Salto that pushes in angled spines along their length and is kinematically constrained to engage/disengage with leg crouch/extension. The resulting mechanism introduces no new actuators, enables jumping on penetrable inclines up to 60°, and enables static adhesion to hold 7.5 times the robot's weight from a ceiling.

We present a novel approach for mobile manipulator self-calibration using contact information. Our method, based on point cloud registration, is applied to estimate the extrinsic transform between a fixed vision sensor mounted on a mobile base and an end effector. Beyond sensor calibration, we demonstrate that the method can be extended to include manipulator kinematic model parameters, which involves a nonrigid registration process. Our procedure uses on-board sensing exclusively and does not rely on any external measurement devices, fiducial markers, or calibration rigs. Further, it is fully automatic in the general case. We experimentally validate the proposed method on a custom mobile manipulator platform, and demonstrate centimetre-level post-calibration accuracy in positioning of the end effector using visual guidance only. We also discuss the stability properties of the registration algorithm, in order to determine the conditions under which calibration is possible.

In the field of autonomous robotics, Simultaneous Localization and Mapping (SLAM) is still a challenging problem. With cheap visual sensors attracting more and more attention, various solutions to the SLAM problem using visual cues have been proposed. However, current visual SLAM systems are still computationally demanding, especially on embedded devices. In addition, collaborative SLAM approaches emerge using visual information acquired from multiple robots simultaneously to build a joint map. In order to address both challenges, we present an approach for remote visual SLAM where local binary features are extracted at the robot, compressed and sent over a network to a centralized powerful processing node running the visual SLAM algorithm. To this end, we propose a new feature coding scheme including a feature selection stage which ensures that only relevant information is transmitted. We demonstrate the effectiveness of our approach on well-known datasets. With the proposed approach, it is possible to build an accurate map while limiting the data rate to 75 kbits/frame.

How does a person work out their location using a floorplan? It is probably safe to say that we do not explicitly measure depths to every visible surface and try to match them against different pose estimates in the floorplan. And yet, this is exactly how most robotic scan-matching algorithms operate. Similarly, we do not extrude the 2D geometry present in the floorplan into 3D and try to align it to the real-world. And yet, this is how most vision-based approaches localise. Humans do the exact opposite. Instead of depth, we use high level semantic cues. Instead of extruding the floorplan up into the third dimension, we collapse the 3D world into a 2D representation. Evidence of this is that many of the floorplans we use in everyday life are not accurate, opting instead for high levels of discriminative landmarks. In this work, we use this insight to present a global localisation approach that relies solely on the semantic labels present in the floorplan and extracted from RGB images. While our approach is able to use range measurements if available, we demonstrate that they are unnecessary as we can achieve results comparable to state-of-the-art without them.

Screw-Propelled Vehicles (SPV's) have been widely used for terrestrial applications such as transportation over mud, snow, and amphibious environments. Similar vehicles have also been applied to industrial processes such as dewatering. Typical designs rely on a large pontoon shaft and relatively small blades to prevent unwanted sinkage or blade damage. These types of vehicles were considered during the design of the first lunar rover, given their success in aqueous and arctic media and simplicity compared to tracked vehicles. Studies have looked at the mobility of SPV's on the surface of granular media but there are not any computational and experimental studies on propulsive buried screws. Understanding the role of screw design and its angular velocity on thrust force is key to the advancement and control of SPV's. This study presents experimental and computational results of a submerged, double-helix Archimedes screw generating propulsive force against a bed of soda-lime glass beads. Thus, this research forms the basis for design of a future miniaturized exploration vehicle for space applications. In our study, we used two different screw designs (5 cm radius, 10 cm length, 63 and 44 degrees helix angle corresponding to 4 cm and 8 cm pitch, respectively) submerged in 2mm glass beads (90% roundness with sizes 1.8 mm to 2.2 mm), For both screws, a similar trend is observed between rotational speed and thrust force. We used EDEM, a Discrete Element Modeling (DEM) software for computational studies of the screw interactions with granular media. There is 5-20% discrepancy between our computational and experimental results. We will discuss possible sources of error and the potential for using DEM as a design tool for SPV's.

This paper presents SceneCut, a novel approach to jointly discover previously unseen objects and non-object surfaces using a single RGB-D image. SceneCut's joint reasoning over scene semantics and geometry allows a robot to detect and segment object instances in complex scenes where modern deep learning-based methods either fail to separate object instances, or fail to detect objects that were not seen during training. SceneCut automatically decomposes a scene into meaningful regions which either represent objects or scene surfaces. The decomposition is qualified by an unified energy function over objectness and geometric fitting. We show how this energy function can be optimized efficiently by utilizing hierarchical segmentation trees. Moreover, we leverage a pre-trained convolutional oriented boundary network to predict accurate boundaries from images, which are used to construct high-quality region hierarchies. We evaluate SceneCut on several different indoor environments, and the results show that SceneCut significantly outperforms all the existing methods.

Environment understanding, object detection and recognition are crucial skills for robots operating in the real world. In this paper, we propose a Convolutional Neural Network with multi-task objectives: object detection and scene classification in one unified architecture. The proposed network reasons globally about an image to understand the scene, hypothesize object locations, and encodes global scene features with regional object features to improve object recognition. We evaluate our network on the standard SUN RGBD dataset. Experiments show that our approach outperforms state-of-the-arts. Network predictions are further transformed into continuous robot beliefs to ensure temporal coherence and extended to 3D space for robotics applications. We embed the whole framework in Robot Operating System, and evaluate its performance on a real robot for semantic mapping and grasp detection.