ICRA 2022 Papers

The recent successes in deep reinforcement learning largely rely on the capabilities of generating masses of data, which in turn implies the use of a simulator. In particular, current progress in multi body dynamic simulators are under-pinning the implementation of reinforcement learning for end-to-end control of robotic systems. Yet simulators are mostly considered as black boxes while we have the knowledge to make them produce a richer information. In this paper, we are proposing to use the derivatives of the simulator to help with the convergence of the learning. For that, we combine model-based trajectory optimization to produce informative trials using 1st- and 2nd-order simulation derivatives. These locally-optimal runs give fair estimates of the value function and its derivatives, that we use to accelerate the convergence of the critics using Sobolev learning. We empirically demonstrate that the algorithm leads to a faster and more accurate estimation of the value function. The resulting value estimate is used in model-predictive controller as a proxy for shortening the preview horizon. We believe that it is also a first step toward superlinear reinforcement learning algorithm using simulation derivatives, that we need for end-to-end legged locomotion.

A practical approach to learning robot skills, often termed sim2real, is to train control policies in simulation and then deploy them on a real robot. Popular sim2real techniques build on domain randomization (DR) - training the policy on diverse randomly generated domains for better generalization to the real world. Due to the large number of hyper-parameters in both the policy learning and DR algorithms, one often ends up with a large number of trained policies, where choosing the best policy among them demands costly evaluation on the real robot. In this work we ask - can we rank the policies without running them in the real world? Our main idea is that a predefined set of real world data can be used to evaluate all policies, using out-of-distribution detection (OOD) techniques. In a sense, this approach can be seen as a ‘unit test’ to evaluate policies before any real world execution. However, we find that by itself, the OOD score can be inaccurate and very sensitive to the particular OOD method. Our main contribution is a simple-yet-effective policy score that combines OOD with an evaluation in simulation. We show that our score - VSDR - can significantly improve the accuracy of policy ranking without requiring additional real world data. We evaluate the effectiveness of VSDR on sim2real transfer in a robotic grasping task with image inputs. We extensively evaluate different DR parameters and OOD methods, and show that VSDR improves policy selection across the board. More importantly, our method achieves significantly better ranking, and uses significantly less data compared to baselines. Project website is at https://sites.google.com/view/vsdr/home

While imitation learning for vision-based au-tonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state-action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or-worse yet-the demonstrator does not have access to the platform at all? Is imitation learning for vision-based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent ideas from the Imitation from Observation (IfO) literature can be brought to bear such that a robot can learn to navigate using only ego-centric video collected by a demonstrator, even in the presence of viewpoint mismatch. To this end, we introduce a new algorithm, Visual-Observation-only Imitation Learning for Autonomous navigation (VOILA), that can successfully learn navigation policies from a single video demonstration collected from a physically different agent. We evaluate VOILA in the AirSim simulator and show that VOILA not only successfully imitates the expert, but that it also learns navigation policies that can generalize to novel environments. Further, we demonstrate the effectiveness of VOILA in a real-world setting by showing that it allows a wheeled Jackal robot to successfully imitate a human walking in an environment while recording video with a handheld mobile phone camera.

Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios. However, the poor photorealism and lack of diverse sensor modalities of existing simulation engines remain key hurdles towards realizing this potential. Here, we present VISTA††Full code release for the VISTA data-driven simulation engine is available here: vista.csail.mit.edu., an open source, data-driven simulator that integrates multiple types of sensors for autonomous vehicles. Using high fidelity, real-world datasets, VISTA represents and simulates RGB cameras, 3D LiDAR, and event-based cameras, enabling the rapid generation of novel viewpoints in simulation and thereby enriching the data available for policy learning with corner cases that are difficult to capture in the physical world. Using VISTA, we demonstrate the ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle. The policies learned in VISTA exhibit sim-to-real transfer without modification and greater robustness than those trained exclusively on real-world data.

Deformable object manipulation requires computationally efficient representations that are compatible with robotic sensing modalities. In this paper, we present VIRDO: an implicit, multi-modal, and continuous representation for deformable-elastic objects. VIRDO operates directly on visual (point cloud) and tactile (reaction forces) modalities and learns rich latent embeddings of contact locations and forces to predict object deformations subject to external contacts. Here, we demonstrate VIRDOs ability to: i) produce high-fidelity cross-modal reconstructions with dense unsupervised correspondences, ii) generalize to unseen contact formations, and iii) state-estimation with partial visio-tactile feedback. https://github.com/MMintLab/VIRDO

In this paper, we propose a tightly-coupled SLAM system fused with RGB, Depth, IMU and structured plane information. Traditional sparse points based SLAM systems always maintain a mass of map points to model the environment. Huge number of map points bring us a high computational complexity, making it difficult to be deployed on mobile devices. On the other hand, planes are common structures in man-made environment especially in indoor environments. We usually can use a small number of planes to represent a large scene. So the main purpose of this article is to decrease the high complexity of sparse points based SLAM. We build a lightweight back-end map which consists of a few planes and map points to achieve efficient bundle adjustment (BA) with an equal or better accuracy. We use homography constraints to eliminate the parameters of numerous plane points in the optimization and reduce the complexity of BA. We separate the parameters and measurements in homography and point-to-plane constraints and compress the measurements part to further effectively im-prove the speed of BA. We also integrate the plane information into the whole system to realize robust planar feature extraction, data association, and global consistent planar reconstruction. Finally, we perform an ablation study and compare our method with similar methods in simulation and real environment data. Our system achieves obvious advantages in accuracy and efficiency. Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures. The global bundle adjustment is nearly 2 times faster than the sparse points based SLAM algorithm.

Direct communication between humans and autonomous underwater vehicles (AUVs) is a relatively under-explored area in human-robot interaction research, although many tasks (e.g., surveillance, inspection, and search-and-rescue) require close diver-robot collaboration. Suboptimal AUV positioning relative to its human collaborators can lead to poor quality interaction and lead to excessive cognitive and physical load for divers. In this paper, we introduce a novel method for AUVs to autonomously navigate and achieve diver-relative positioning to begin interaction. Our method is based only on monocular vision, requires no global localization, and is computationally efficient. We present our algorithm and its implementation on board a physical AUV, performing extensive evaluations in the form of closed-water tests in a controlled pool. Our results show that the proposed monocular vision-based algorithm performs reliably and efficiently, operating entirely on-board the AUV.

Robots that process navigation instructions in large outdoor environments will need to operate at different levels of abstraction. For example, a land-surveying aerial robot receiving the instruction “go to Boston and go through the state forest on the way” must reason about a long-range goal like “go to Boston” while also processing a finer-grained constraint like “go through the state forest.” Existing approaches struggle to plan such commands because of the immense number of locations and constraints that can be expressed in language. We introduce a hierarchical representation of outdoor environments and a planning approach that dynamically compacts the robot's state space to enable tractable planning in city and state-scale environments. Our approach leverages natural abstractions in real-world map data, coupled with abstractions generated from users' instructions, to generate filtered environment views that accelerate planning while supporting a robot's ability to obey complex temporal goals and constraints at different levels of abstraction. We evaluate our approach on seven templates of LTLJ formulas and in an 80 kilometer-radius environment containing over 250,000 locations downloaded from OpenStreetMap. The results show our approach enables planning in seconds or minutes in a large outdoor environment while still satisfying the task specification.

Collaborative robots that operate alongside humans require the ability to understand their intent and forecast their pose. Among the various indicators of intent, the eye gaze is particularly important as it signals action towards the gazed object. By observing a person's gaze, one can effectively predict the object of interest and subsequently, forecast the person's pose. We leverage this and present a method that forecasts the human pose using gaze information for everyday pick and place actions in a home environment. Our method first attends to fixations to locate the coordinates of the object of interest before inputting said coordinates to a pose forecasting network. Experiments on the MoGaze dataset show that our gaze network lowers the errors of existing pose forecasting methods and that incorporating prior in the form of textual instructions further lowers the errors by a significant amount. Furthermore, the use of eye gaze now allows a simple multilayer perceptron network to directly forecast the keypose.aaCode available at www.imperial.ac.uk/personal-robotics/software

This work investigates how arm swing movements measured by Inertial Motion Unit (IMU) sensors can be used to identify and maintain the walking state in a self-balanced lower-limb exoskeleton for medical use. When an exoskeleton is in a dynamical state during gait, short patterns in IMU signals (e.g. a braking movement) can be hard to extract. Therefore, by relying on a threshold-based classifier constructed upon descriptive features of actively maintained arm swing movements, it is possible to build a gait termination detection method in which the transition between the walking and standstill states occurs whenever arm movements cease, and the corresponding patterns in the IMU signals disappear. Analysis of arm IMU signals were used to identify three amplitude and coordination-based features for the classification architecture. An online implementation of this novel detection interface for maintaining the walking state was validated with 11 unimpaired participants using the Atalante exoskeleton, leading to high accuracy with less than 2% of false negatives when the arms were swinging at a high amplitude, and less than 15% when they were swinging at a medium amplitude.

Haptic sensing has recently been used effectively for legged robot localization in extreme scenarios where cam-eras and LiDAR might fail, such as dusty mines and foggy sewers. However, existing haptic sensing mainly relies on supervised classification, with training and evaluation executed over explicit terrain classes. Defining classes is a significant limitation to real-world applications, where prior labelling and handcrafted classes are often impractical. This paper proposes a novel haptic localization system based on a fully unsupervised terrain representation learned solely from the force/torque sensors located in the quadruped robot's feet. Instead of using the detected terrain class for localization, we propose an improved autoencoder architecture to generate a sparse map of encodings on the first run and to localize against this sparse map during subsequent runs. We compare our approach to a haptic localization system based on supervised terrain classification, showing that the unsupervised method has comparable or better performance than the supervised one for the same trajectories while clearly outperforming the proprioceptive odometry estimator available on the robot. Therefore, the proposed approach is well-suited for a routine maintenance application, increasing the platform's robustness.

In this paper, we focus on a less explored, but more realistic and complex problem of domain adaptation in LiDAR semantic segmentation. There is a significant drop in performance of an existing segmentation model when training (source domain) and testing (target domain) data originate from different LiDAR sensors. To overcome this shortcoming, we propose an unsupervised domain adaptation framework that leverages unlabeled target domain data for self-supervision, coupled with an unpaired mask transfer strategy to mitigate the impact of domain shifts. Furthermore, we introduce the gated adapter module with a small number of parameters into the network to account for target domain-specific information. Experiments adapting from both real-to-real and synthetic-to-real LiDAR semantic segmentation benchmarks demonstrate the significant improvement over prior arts.

Depth information is considered valuable as it describes geometric structures, which benefits various robotic tasks. However, the depth acquired by RGB-D sensors still suffers from two deficiencies, i.e., incompletion and noises. Previous methods complete depth by exploring hand-tuned models or raising surface assumptions, while nowadays, deep approaches intend to solve this problem with rendered image pairs. For depth denoising, as a consequence of different sensor mechanisms, most methods can only work under specific devices. With existing methods, three challenges emerge: the onerous training set collecting process, the mismatch between existing models and present RGB-D sensors, and the non-real-time computation. In this paper, we first state depth completion and denoising are inherently different and without the need to collect or render complete and noiseless ground truths. We address all mentioned challenges with two separate un-supervised learning procedures. The completion network takes color and incomplete depth as input and predicts values to the unobserved area, which combines prior knowledge and color-depth correlations. The denoising step exploits image sequences to construct noise models in a self-supervised manner with the ability to cater to different sensors. Experimental comparisons and ablation studies demonstrate that even without human-labeled ground truths, the proposed method could produce better completion results and also reduce noises in real-time.

Instance-aware segmentation of unseen objects is essential for a robotic system in an unstructured environment. Although previous works achieved encouraging results, they were limited to segmenting the only visible regions of unseen objects. For robotic manipulation in a cluttered scene, amodal perception is required to handle the occluded objects behind others. This paper addresses Unseen Object Amodal Instance Segmentation (UOAIS) to detect 1) visible masks, 2) amodal masks, and 3) occlusions on unseen object instances. For this, we propose a Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order. We evaluated our method on three benchmarks (tabletop, indoors, and bin environments) and achieved state-of-the-art (SOTA) performance. Robot demos for picking up occluded objects, codes, and datasets are available at https://sites.google.com/view/uoais.

In Simultaneous Localization And Mapping (SLAM) problems, high-level landmarks have the potential to build compact and informative maps compared to traditional point-based landmarks. In this work, we focus on the param-eterization of frequently used geometric primitives including points, lines, planes, ellipsoids, cylinders, and cones. We first present a unified representation based on quadrics, an algebraic representation of quadratic surfaces in 3D. Then we propose a decomposed model of quadrics that discloses the symmetry and degeneration properties of a primitive. Based on the decomposition, we develop geometrically meaningful quadrics factors for the graph-SLAM problem. Then in simulation, it is shown that the decomposed formulation has better efficiency and robustness to observation noises than baseline parame-terizations. Finally, in real-world experiments, the proposed back-end framework is demonstrated to be capable of building compact and regularized maps.

Visual-inertial sensors have a wide range of applications in robotics. However, good performance often requires different sophisticated motion routines to accurately calibrate camera intrinsics and inter-sensor extrinsics. This work presents a novel formulation to learn a motion policy to be executed on a robot arm for automatic data collection for calibrating intrinsics and extrinsics jointly. Our approach models the calibration process compactly using model-free deep reinforcement learning to derive a policy that guides the motions of a robotic arm holding the sensor to efficiently collect measurements that can be used for both camera intrinsic calibration and camera-IMU extrinsic calibration. Given the current pose and collected measurements, the learned policy generates the subsequent transformation that optimizes sensor calibration accuracy. The evaluations in simulation and on a real robotic system show that our learned policy generates favorable motion trajectories and collects enough measurements efficiently that yield the desired intrinsics and extrinsics with short path lengths. In simulation, we are able to perform calibrations 10× faster than hand-crafted policies, which transfers to a real-world speed up of 3× over a human expert. The code of this work is publicly available at: https://github.com/ethz-asl/Learn-to-Calibrate.

Large efforts have focused on ensuring that the controllers for mobile service robots follow proxemics and other social rules to ensure both safe and socially acceptable distance to pedestrians. Nonetheless, involuntary contact may be unavoidable when the robot travels in crowded areas or when encountering adversarial pedestrians. Freezing the robot in response to contact might be detrimental to bystanders' safety and prevents it from achieving its task. Unavoidable contacts must hence be controlled to ensure the safe and smooth travelling of robots in pedestrian alleys. We present a force-limited and obstacle avoidance controller integrated into a time-invariant dynamical system (DS) in a closed-loop force controller that let the robot react instantaneously to contact or to the sudden appearance of pedestrians. Mitigating the risk of collision is done by modulating the velocity commands upon detecting a contact and by absorbing part of the contact force through active compliant control when the robot bumps inad-vertently against a pedestrian. We evaluated our method with a personal mobility robot -Qolo- showing contact mitigation with passive and active compliance. We showed the robot able to overcome an adversarial pedestrian within 9 N of the set limit contact force for speeds under 1 m/s. Moreover, we evaluated integrated obstacle avoidance proving the ability to advance without Incurring any other collision.

Autonomous Underwater Vehicles (AUVs) are a vital element for ocean exploration in various applications; however, energy sustainability still limits long-term operations. An option to overcome this problem is using underwater docking for power and data transfer. To robustly guide an AUV into a docking station, we propose an underwater vision algorithm for short-distance detection. In this paper, we present a Convolutional Neural Network architecture to accurately estimate the dock position during the terminal homing stage of the docking. Additionally, to alleviate the lack of available underwater datasets, two methods are proposed to generate synthetic datasets, one using a CycleGAN network, and another using Artistic Style transfer network. Both methods are used to train the same CNN architecture to compare the results. Finally, implementation details of the CNN are presented under the backseat architecture and ROS framework, running on an IVER3 AUV.

The Xacro XML macro language can be used to augment the Universal Robot Description Format (URDF) and is part of a critical toolchain from geometric representations to simulation, visualization, and system execution. However, mem-bers of the robotics community, especially newcomers, struggle to troubleshoot and understand the interplay between systems and the Xacro preprocessing pipeline. To better understand how system developers struggle with Xacros, we manually examine 712 Xacro-related questions from the question and answer site answers.ros.org and find Xacro misunderstandings fit into eight key categories using a systematic, qualitative approach called Open Coding. By examining the 'tags' applied to questions, we further find that Xacro problems manifest in a befuddlingly broad set of contexts. This hinders onboarding and complicates system developers' understanding of representations and tools in the Robot Operating System. We aim to provide an empirical grounding that identifies and prioritizes impediments to users of open robotics systems, so that tool designers, teachers, and robotics practitioners can devise ways of improving robot software tooling and education.