ICRA 2022 Papers

While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainties do not faithfully represent the true underlying uncertainties (process noise). Our key insight is that calibration is only achieved by imposing constraints across multiple examples, such as those in a mini-batch; as opposed to existing approaches which only impose constraints per-sample, often leading to overconfident (thus miscalibrated) uncertainty estimates. By enforcing the distribution of outputs of a neural network to resemble a target distribution by minimizing an ff -divergence, we obtain significantly better-calibrated models compared to prior approaches. Our approach, f-Cal, outperforms existing uncertainty calibration approaches on robot perception tasks such as object detection and monocular depth estimation over multiple real-world benchmarks.

Compliant actuation bestows robots with the ability to cope with unstructured environments, move with agility, and interact safely with humans at the expense of reduced tracking accuracy. The inclusion of dampening components aims to reduce oscillatory dynamics and partially restore precision without sacrificing the previously obtained characteristics. This paper introduces the concept and design of a novel damped compliant actuator suitable for building multi-degree of freedom systems. The proposed unit has a unique actuator topology that has never been seen before in the literature. The gearbox is used as a differential component, allowing the design of compact units without giving up safety and accuracy enhancements. We present and analyze the actuator's model and experimentally characterize the actuator prototype and the elastic and damping component.

This paper introduces a novel approach for whole-body motion planning and dynamic occlusion avoidance. The proposed approach reformulates the visibility constraint as a likelihood maximization of visibility probability. In this formulation, we augment the primary cost function of a whole-body model predictive control scheme through a relaxed log barrier function yielding a relaxed log-likelihood maximization formulation of visibility probability. The visibility probability is computed through a probabilistic shadow field that quantifies point light source occlusions. We provide the necessary algorithms to obtain such a field for both 2D and 3D cases. We demonstrate 2D implementations of this field in simulation and 3D implementations through real-time hardware experiments. We show that due to the linear complexity of our shadow field algorithm to the map size, we can achieve high update rates, which facilitates onboard execution on mobile platforms with limited computational power. Lastly, we evaluate the performance of the proposed MPC reformulation in simulation for a quadrupedal mobile manipulator.

Parallel mechanisms are becoming increasingly popular as subsystems in various robots due to their superior stiffness, payload-to-weight ratio, and dynamic properties. The serial connection of parallel subsystems leads to series-parallel hybrid robots, which are more difficult to model and control than serial or tree-type systems. At the same time, Whole-Body Control (WBC) has become the method of choice in the control of robots with redundant degrees of freedom, e.g., legged robots. However, most state-of-the-art WBC frameworks can only deal with serial or tree-type robot topologies. In this paper, we describe a computationally efficient framework for Whole-Body Control of series-parallel hybrid robots subjected to a large number of holonomic constraints. In contrast to existing WBC frameworks, our approach describes the optimization problem in the actuation space of a series-parallel robot, which provides better exploitation of the feasible workspace, higher accuracy, and more transparent behavior near singularities. We evaluate the proposed framework on two different humanoids with series-parallel architecture and compare its performance to a WBC approach for tree-type robots.

Search missions require motion planning and navigation methods for information gathering that continuously replan based on new observations of the robot's surroundings. Current methods for information gathering, such as Monte Carlo Tree Search, are capable of reasoning over long horizons, but they are computationally expensive. An alternative for fast online execution is to train, offline, an information gathering policy, which indirectly reasons about the information value of new observations. However, these policies lack safety guarantees and do not account for the robot dynamics. To overcome these limitations we train an information-aware policy via deep reinforcement learning, that guides a receding-horizon trajectory optimization planner. In particular, the policy continuously recommends a reference viewpoint to the local planner, such that the resulting dynamically feasible and collision-free trajectories lead to observations that maximize the information gain and reduce the uncertainty about the environment. In simulation tests in previously unseen environments, our method consistently outperforms greedy next-best-view policies and achieves competitive performance compared to Monte Carlo Tree Search, in terms of information gains and coverage time, with a reduction in execution time by three orders of magnitude.

Flying robots are usually rather delicate and require protective enclosures when facing the risk of collision, while high complexity and reduced payload are recurrent problems with collision-resilient flying robots. Inspired by arthropods' exoskeletons, we design a simple, open source, easily manufactured, semi-rigid structure with soft joints that can withstand high-velocity impacts. With an exoskeleton, the protective shell becomes part of the main robot structure, thereby minimizing its loss in payload capacity. Our design is simple to build and customize using cheap components (e.g. bamboo skewers) and consumer-grade 3D printers. The result is CogniFly, a sub-250 g autonomous quadcopter that survives multiple collisions at speeds up to 7 m s−1. In addition to its collision-resilience, CogniFly carries sensors that allow it to fly for approx. 17 min without the need of GPS or an external motion capture system, and it has enough computing power to run deep neural network models on-board. This structure becomes an ideal platform for high-risk activities, such as flying in a cluttered environment or reinforcement learning training, by dramatically reducing the risks of damaging its own hardware or the environment. Source code, 3D files, instructions and videos are available (open source license) through the project's website: https://thecognifly.github.io.

Correspondence learning is a fundamental problem in robotics, which aims to learn a mapping between state, action pairs of agents of different dynamics or embodiments. However, current correspondence learning methods either leverage strictly paired data-which are often difficult to collect-or learn in an unsupervised fashion from unpaired data using regularization techniques such as cycle-consistency-which suffer from severe misalignment issues. We propose a weakly supervised correspondence learning approach that trades off between strong supervision over strictly paired data and unsupervised learning with a regularizer over unpaired data. Our idea is to leverage two types of weak supervision: i) temporal ordering of states and actions to reduce the compounding error, and ii) paired abstractions, instead of paired data, to alleviate the misalignment problem and learn a more accurate correspondence. The two types of weak supervision are easy to access in real-world applications, which simultaneously reduces the high cost of annotating strictly paired data and improves the quality of the learned correspondence. We show the videos of the experiments on our website.

Existing state-of-the-art 3D point clouds understanding methods only perform well in a fully supervised manner. To the best of our knowledge, there exists no unified framework which simultaneously solves the downstream high-level understanding tasks, especially when labels are extremely limited. This work presents a general and simple framework to tackle point clouds understanding when labels are limited. We propose a novel unsupervised region expansion based clustering method for generating clusters. More importantly, we innovatively propose to learn to merge the over-divided clusters based on the local low-level geometric property similarities and the learned high-level feature similarities supervised by weak labels. Hence, the true weak labels guide pseudo labels merging taking both geometric and semantic feature correlations into consideration. Finally, the self-supervised data augmentation optimization module is proposed to guide the propagation of labels among semantically similar points within a scene. Experimental Results demonstrate that our framework has the best performance among the three most important weakly supervised point clouds understanding tasks including semantic segmentation, instance segmentation, and object detection even when limited points are labeled.

In this paper, we revisit the problem of learning a stabilizing controller from a finite number of demonstrations by an expert. By focusing on feedback linearizable systems, we show how to combine expert demonstrations into a stabilizing controller, provided that demonstrations are sufficiently long and there are at least n+1n+1 of them, where nn is the number of states of the system being controlled. The results are experimentally demonstrated on a CrazyFlie 2.0 quadrotor.

Manipulating objects with dexterity requires timely feedback that simultaneously leverages the senses of vision and touch. In this paper, we focus on the problem setting where both visual and tactile sensors provide pixel-level feedback for Visuotactile reinforcement learning agents. We investigate the challenges associated with multimodal learning and propose several improvements to existing RL methods; including tactile gating, tactile data augmentation, and visual degradation. When compared with visual-only and tactile-only baselines, our Visuotactile-RL agents showcase (1) significant improvements in contact-rich tasks; (2) improved robustness to visual changes (lighting/camera view) in the workspace; and (3) resilience to physical changes in the task environment (weight/friction of objects).

This paper emphasizes the importance of a robot's ability to refer to its task history, especially when it exe-cutes a series of pick-and-place manipulations by following language instructions given one by one. The advantage of referring to the manipulation history can be categorized into two folds: (1) the language instructions omitting details but using expressions referring to the past can be interpreted, and (2) the visual information of objects occluded by previous manipulations can be inferred. For this, we introduce a history-dependent manipulation task which objective is to visually ground a series of language instructions for proper pick-and-place manipulations by referring to the past. We also suggest a relevant dataset and model which can be a baseline, and show that our model trained with the proposed dataset can also be applied to the real world based on the CycleGAN. Our dataset and code are publicly available on the project website: https://sites.google.com/view/history-dependent-manipulation.

Task and motion planning (TAMP) algorithms aim to help robots achieve task-level goals, while maintaining motion-level feasibility. This paper focuses on TAMP domains that involve robot behaviors that take extended periods of time (e.g., long-distance navigation). In this paper, we develop a visual grounding approach to help robots probabilistically evaluate action feasibility, and introduce a TAMP algorithm, called GROP, that optimizes both feasibility and efficiency. We have collected a dataset that includes 96, 000 simulated trials of a robot conducting mobile manipulation tasks, and then used the dataset to learn to ground symbolic spatial relationships for action feasibility evaluation. Compared with competitive TAMP baselines, GROP exhibited a higher task-completion rate while maintaining lower or comparable action costs. In addition to these extensive experiments in simulation, GROP is fully implemented and tested on a real robot system.

Autonomous mobile robots deployed in outdoor environments must reason about different types of terrain for both safety (e.g., prefer dirt over mud) and deployer preferences (e.g., prefer dirt path over flower beds). Most existing solutions to this preference-aware path planning problem use semantic segmentation to classify terrain types from camera images, and then ascribe costs to each type. Unfortunately, there are three key limitations of such approaches - they 1) require preenumeration of the discrete terrain types, 2) are unable to handle hybrid terrain types (e.g., grassy dirt), and 3) require expensive labelled data to train visual semantic segmentation. We introduce Visual Representation Learning for Preference-Aware Path Planning (VRL-PAP), an alternative approach that overcomes all three limitations: VRL-PAP leverages un-labelled human demonstrations of navigation to autonomously generate triplets for learning visual representations of terrain that are viewpoint invariant and encode terrain types in a continuous representation space. The learned representations are then used along with the same unlabelled human navigation demonstrations to learn a mapping from the representation space to terrain costs. At run time, VRL-PAP maps from images to representations and then representations to costs to perform preference-aware path planning. We present empirical results from challenging outdoor settings that demonstrate VRL-PAP 1) is successfully able to pick paths that reflect demonstrated preferences, 2) is comparable in execution to geometric navigation with a highly detailed manually annotated map (without requiring such annotations), 3) is able to generalize to novel terrain types with minimal additional unlabeled demonstrations.

This paper presents results from a study on the perception of robot attributes by human observers in their peripheral field of view depending on types of eye-movements of the latter. A between-subjects design is used, where a picture of a robot head is presented on a screen in the peripheral field of view of the participants with two conditions of eye-movements: static and pursuit. The two conditions are realized by a dot to be tracked by the participants, which is placed either in the center of the screen or moving linearly between left and right screen margins. As a subjective measure for appearance attributes, anthropomorphism is used using the multi-component Human-Robot Interaction Evaluation Scale (HRIES). Significant differences are revealed in the sociability and agency sub-scales of HRIES with respect to eye-movement behaviors. The results show, that humans perceive robot appearance attributes differently depending on whether or not pursuit eye-movements are conducted supporting the main hypothesis. Further, for pursuit movements, the scale center is rated for these sub-scales, which may indicate, that the particular robot characteristic is perceived less distinctively. No significant interactions are found for animacy and disturbance, which may be due to less applicability of these sub-scales to static images. The findings may have an impact on close interaction between humans and robots and potential anthropomorphism-related influences on task accomplishment to be considered in interaction design.

Drawing inspiration from biology, we describe the way in which visual sensing with a monocular camera can provide a reliable signal for navigation of mobile robots. The work takes inspiration from the classic paper [3] which described a behavioral strategy pursued by diving sea birds based on a visual cue called time-to-contact. A closely related concept of time-to-transit, \tau\tau, is defined, and it is shown that steering laws based on monocular camera perceptions of \tau\tau can reliably steer a mobile vehicle. The contribution of the paper is two-fold. It provides a simple theory of robust vision-based steering control. It goes on to show how the theory guides the implementation of robust visual navigation using ROS-Gazebo simulations as well as deployment and experiments with a camera-equipped Jackal robot. As will be noted, there is an extensive literature on how animals use optical flow to guide their movements. The novelty of the work below is the introduction of the concepts of Eulerian optical flow and time-to-transit, \tau\tau and the demonstration that control laws based on the \tau\tau-values associated with an aggregated set of features in the field of view can be used to reliably steer a laboratory robot.

Robots capable of traversing flights of stairs play an important role in both indoor and outdoor applications. The capability of accurately identifying a staircase is one of the vital technical functions in these robots. This paper presents a vision-based ascending stair detection algorithm using RGB-Depth (RGB-D) data based on an interpretable model. The method follows the four steps: 1) pre-processing of RGB images for line extraction by applying the dilatation and Canny filters followed by the probabilistic Hough line transform, 2) defining the regions of interests (ROIs) via K- mean clustering, 3) training the initial model based on a support vector machine (SVM) using three extracted features (i.e., gradient, continuity factor, and deviation cost), and 4) building an interpretable model for stair classification by determining the decision boundary conditions. The developed method was evaluated for its performance using our dataset, and the results showed 85% sensitivity and 94% specificity. When the same model was tested on a different test set, the sensitivity and specificity slightly decreased to 80% and 90%, respectively. By shifting the boundary conditions using only a small subset of the new dataset without rebuilding the model, performance was improved to 90% sensitivity and 96% specificity. The presented method is also compared with existing SVM- and neural- network- based methods.

In this paper, we present a complete pipeline for 3D semantic mapping solely based on a stereo camera system. The pipeline comprises a direct sparse visual odometry frontend as well as a back-end for global optimization including GNSS integration, and semantic 3D point cloud labeling. We propose a simple but effective temporal voting scheme which improves the quality and consistency of the 3D point labels. Qualitative and quantitative evaluations of our pipeline are performed on the KITTI-360 dataset. The results show the effectiveness of our proposed voting scheme and the capability of our pipeline for efficient large-scale 3D semantic mapping. The large-scale mapping capabilities of our pipeline is furthermore demonstrated by presenting a very large-scale semantic map covering 8000 km of roads generated from data collected by a fleet of vehicles.

In this paper, we present a framework rooted in control and planning that enables quadrupedal robots to traverse challenging terrains with discrete footholds using visual feedback. Navigating discrete terrain is challenging for quadrupeds because the motion of the robot can be aperiodic, highly dynamic, and blind for the hind legs of the robot. Additionally, the robot needs to reason over both the feasible footholds as well as the base velocity in order to speed up or slow down at different parts of the discrete terrain. To address these challenges, we build an offline library of periodic gaits which span two trotting steps, and switch between different motion primitives to achieve aperiodic motions of different step lengths on a quadrupedal robot. The motion library is used to provide targets to a geometric model predictive controller which outputs the contact forces at the stance feet. To incorporate visual feedback, we use terrain mapping tools and a forward facing depth camera to build a local height map of the terrain around the robot, and extract feasible foothold locations around both the front and hind legs of the robot. Our experiments show a small scale quadruped robot navigating multiple unknown, challenging and discrete terrains in the real world.

Stable contact and safe responses to the collision have been studied to develop interactive robots such as service and collaborative robots. Stable and safe interactions are usually achieved through the inherent compliance of a motion controller with external torque estimation. However, a fixed control gain would sacrifice either compliance or position tracking performance. Additionally, external torque estimation is susceptible to model errors. In this study, a novel variable stiffness control approach is proposed to achieve a high position tracking performance in free motion and compliant behavior in the contact state. For this purpose, a precise estimation of the external torque and control gains that change based on the external torque are required. To estimate the external torque precisely, a collision detecting learning algorithm that uses long short-term memory (LSTM) is adopted. Although this method uses only proprioceptive sensors, its torque estimation capability is comparable to that of methods that use additional sensors. Then, the stiffness of a motion controller is adjusted based on the external torque in the stable region. Moreover, by adopting the Operational Space Formulation considering joint elasticity for a motion controller, high position tracking performance can be achieved with only proprioceptive sensors. The performance of the proposed method was validated through comparative experiments with two degrees of freedom (DoF) manipulator.