ICRA 2021 Papers

This paper addresses the safe path planning problem for autonomous mobility with multi-modal perception uncertainties. Specifically, we assume that different sensor inputs lead to different Gaussian process regulated perception uncertainties (named as multi-modal perception uncertainties). We implement a Bayesian inference algorithm, which merges the multi-modal GP-regulated uncertainties into a unified one and translates the unified uncertainty into a dynamic risk map. With the safe path planner taking the risk map as input, we are able to plan a safe path for the autonomous vehicle to follow. Experimental results on an autonomous golf cart testbed validate the applicability and efficiency of the proposed algorithm.

This paper presents an integrated methodology and experimental validation of an autonomous framework for unmanned underwater vehicles (UUVs) merely equipped with a conventional monocular camera and a pressure sensor to accomplish high-performance autonomy. Optimal pose of the UUV is solved iteratively by Levenburg-Marquardt optimization for the Perspective-n-Point (PnP) problem. To guarantee a consistent localization system, a properly-tuned EKF with extra outlier removal approaches including applying Chi-square tests of innovations adequately removes measurement noises, mean-while provides unknown navigation state estimations. A classic adaptive controller is developed to enable autonomous mobility. Real-time experiments are designed to demonstrate underwater autonomous performance with a miniature commercial UUV, BlueROV2 Heavy.

In this paper, we propose a novel deep reinforcement learning (DRL) system for the autonomous navigation of mobile robots that consists of three modules: map navigation, multi-view perception and multi-branch control. Our DRL system takes as the input a routed map provided by a global planner and three RGB images captured by a multi-camera setup to gather global and local information, respectively. In particular, we present a multi-view perception module based on an attention mechanism to filter out redundant information caused by multi-camera sensing. We also replace raw RGB images with low-dimensional representations via a specifically designed network, which benefits a more robust sim2real transfer learning. Extensive experiments in both simulated and real-world scenarios demonstrate that our system outperforms state-of-the-art approaches.

We propose an approach enabling an Unmanned Aerial Vehicle (UAV) to autonomously enter a target building through an open window. We use a fusion of depth camera and 2D Light Detection and Ranging (LiDAR) data for window detection and continuous estimation of its position, orientation, and size. The proposed algorithms are capable of running both with and without available a priori information. The obtained detections are utilized for planning collision-free trajectories through the target window. We use a sensor fusion algorithm for robust altitude estimation from laser rangefinder data while flying over ground with inconsistent elevation. Particular focus is given to the transition between outdoor and indoor environments and vice-versa to achieve the required reliability of UAV state estimation. The proposed approach has been verified in multiple real-world experiments, where the UAV was able to successfully enter and leave the target building both under normal conditions and under decreased visibility conditions in a smoke-filled environment.

This paper contributes a method designed to enable autonomous distributed 3D nuclear radiation field mapping. The algorithm uses a single radiation sensor and a sequence of spatially distributed and robotically acquired radiation measurements across a discretized 3D grid to derive a radiation gradient. The derived gradient is probabilistically propagated to unknown components of the map to further guide a curiosity-driven path planner by identifying the next most radiologically informative point given available information. To demonstrate the method, we develop a resilient micro flying robot capable of autonomous GPS-denied navigation that integrates a Thallium–doped Cesium Iodide (CsI(Tl)) scintillator and Silicon Photomultiplier (SiPm) combined with custom–built pulse counting circuitry. A set of experimental studies is presented inside an indoor facility within which actual radioactive uranium ore sources have been distributed.

In this work, we focus on the autonomous navigation of snake robots in densely-cluttered environments, where collisions between the robot and obstacles are frequent, which could happen often in disaster scenarios, underground caves, or grassland/forest environments. This work takes the view that obstacles are not to be avoided, but rather exploited to support and direct the motion of the snake robot. We build upon a decentralized state-of-the-art compliant controller for serpenoid locomotion, and develop a bi-stable dynamical system that relies on inertial feedback to continuously steer the robot toward a desired direction. We experimentally show that this controller allows the robot to autonomously navigate dense environments by consistently locomoting along a given, global direction of travel in the world, which could be selected by a human operator or a higher level planner. We further equip the robot with an onboard vision system, allowing the robot to autonomously select its own direction of travel, based on the obstacle distribution ahead of its position (i.e., enacting feedforward control). In those additional experiments on hardware, we show how such an exteroceptive sensor can allow the robot to steer before hitting obstacles and to preemptively avoid challenging regions where proprioception-only (i.e., torque and inertial) feedback control would not suffice.

Planetary robotics navigation has attracted the great attention of many researchers in recent years. Localization is one of the most important problems for robots on another planet in the lack of GPS. The robots need to be able to know their location and the surrounding map in the environment concurrently, to work and communicate together on another planet. In the current work, a novel algorithm is designed to cooperatively localize a team of robots on another planet. Consequently, a robust algorithm is developed for cooperative Visual Odometry (VO) to localize each robot in a planetary environment while detecting both intra-loop closure and inter-loop closures using previously observed area by the robot and shared area from other robots, respectively. To validate the proposed algorithm, a comparison is provided between the proposed cooperative VO and the single version of VO. Accordingly, a planetary analogue real dataset is employed to investigate the accuracy of the proposed algorithm. The results promise the concept of cooperative VO to significantly increase the accuracy of localization.

The compact flocking of relatively localized Un-manned Aerial Vehicles (UAVs) in high obstacle density areas is discussed in this paper. The presented work tackles realistic scenarios in which the environment map is not known apriori and the use of a global localization system and communication infrastructure is difficult due to the presence of obstacles. To achieve flocking in such a constrained environment, we propose a fully decentralized, bio-inspired control law that uses only onboard sensor data for safe flocking through the environment without any communication with other agents. In the proposed approach, each UAV agent uses onboard sensors to self-localize and estimate the relative position of other agents in its local reference frame. The usability and performance of the proposed approach were verified and evaluated using various experiments in a realistic robotic simulator and a natural forest. The presented experiments also validate the utility of onboard relative localization for autonomous multi-UAV applications in the absence of global localization information and communication.

The tabletop robot Haru, used for affective telepresence research, enables a teleoperator to communicate affects from a distance. The robot’s expressiveness offers myriad ways of communicating affects through the execution of emotive routines. The teleoperator reacts to input modalities such as the user’s facial expression, gestures and speech-based intent as perceived by the robot’s perception system. However, due to the sheer number of routines to select from, the task of choosing the appropriate or the most preferred routine is becoming cumbersome. In this paper, we propose a human-in-the-loop reinforcement learning mechanism in which an agent learns the teleoperator’s selection preference as a function of the input modalities and aids the routine selection process by narrowing it to n-best optimal choices. Our experimental results show that with only a few number of interactions from the teleoperator, the system can learn to recommend optimal routine behaviors for all perceived modalities, which greatly reduces the workload of the teleoperator.

Model predictive control (MPC) is a powerful feedback technique that is often used in data-driven robotics. The performance of data-driven MPC depends on the accuracy of the model, which often requires careful tuning. Furthermore, specifying the task with an objective function and synthesizing a feedback policy are not straightforward and typically lead to suboptimal solutions driven by trial and error. To address these challenges, we present a method to jointly optimize the data-driven system identification, task specification, and control synthesis of unknown dynamical systems. We use our method to develop AutoMPC3, a software package designed to automate and optimize data-driven MPC. Empirical evaluation on the pendulum swing-up, cart-pole swing-up, and half-cheetah running demonstrates that our method finds data-driven control policies that outperform offline reinforcement learning, without any hand-tuning.

While the automatic creation of maps for localization is a widely tackled problem, the automatic inference of higher layers of HD maps is not. Additionally, approaches that learn from maps require richer and more precise landmarks than currently available.In this work, we fuse semantic detections from a monocular camera with depth and orientation estimation from lidar to automatically detect, track and map parametric, semantic map elements. We propose the use of tailored representations that are minimal in the number of parameters, making the map compact and the estimation robust and precise enough to enable map inference even from single frame detections. As examples, we map traffic signs, traffic lights and poles using upright rectangles and cylinders.After robust multi-view optimization, traffic lights and signs have a mean absolute position error of below 10 cm, extent estimates are below 5 cm and orientation MAE is below 6◦. This proves the suitability as automatically generated, pixel-accurate ground truth, reducing the task of ground truth generation from tedious 3D annotation to a post-processing of misdetections.

LiDAR odometry algorithms are complex and involve a number of hyper-parameters. The choice of hyper-parameters can substantively affect the performance of odometry estimation, and it is necessary to carefully fine-tune the hyper-parameters depending on the sensor, environment, and algorithm to achieve the best estimation results. While odometry estimation algorithms are often tuned manually, this is time-consuming and may also result in a sub-optimal parameter set. This paper presents an automatic hyper-parameter tuning approach for LiDAR odometry estimation. By taking advantage of the sequential model-based optimization (SMBO) approach, we automatically optimize the hyper-parameter set of a black-box odometry estimation algorithm without detailed knowledge of the algorithm. In addition, a LiDAR data augmentation approach is also proposed to prevent overfitting. Through evaluation, we show that the combination of SMBO-based parameter exploration and data augmentation enables us to efficiently and robustly optimize the hyper-parameter set for several different odometry estimation algorithms. We also demonstrate that the optimized parameter set exhibits superior performance with respect to KITTI dataset and in a real use scenario.

The purpose of this paper is the robotic hanging manipulation of an object of various shapes that is not limited to a specific category. To achieve this, we propose a method that allows the estimator to learn many different shapes with hanging points without any manual annotation. A random shape generator using GAN solves the limitation of the number of 3D models and can handle objects of various shapes. In addition, hanging is repeated in the dynamics simulation, and hanging points are automatically generated. A large amount of training data is generated by rendering random-textured objects with hanging points in the random simulation environment. A deep neural network trained with these data was able to estimate hanging points of an unknown category object in the real world and achieved hanging manipulation by a robot.

The increasing frequency of robot interactions with humans and complex environments motivates the design and control of soft robotic systems made of compliant structures. Additive manufacturing processes enable the creation of new robotic systems and mechanisms that gain their compliance extrinsically through their design, rather than simply their material. With the ability to rapidly and affordably manufacture such mechanisms, the interest in producing custom task-specific underactuated mechanisms has grown. These mechanisms may be useful for highly repetitive tasks or complex situations where the simplicity of control outweighs the versatility of a more traditional robotic system. In the course of this work, new underactuated monolithic soft robotic mechanisms are presented along with automated design processes. The structures can reach multiple arbitrary end effector poses and are controlled with as few actuators as possible. Two methods for the automated design of such structures via inverse kinematics and two concepts for the mechanical realization are presented. Several functional models were created to illustrate possible application scenarios and validate the mechanical concepts. A novel contact surface was also developed and shown to improve the accuracy performance of these structures. These automated design methods contribute toward the general accessibility and usability of multi-pose underactuated extrinsically soft mechanisms.

In this paper we present a system capable of collecting and annotating, human performed, robot understandable, everyday activities from virtual environments. The human movements are mapped in the simulated world using off-the-shelf virtual reality devices with full body, and eye tracking capabilities. All the interactions in the virtual world are physically simulated, thus movements and their effects are closely relatable to the real world. During the activity execution, a subsymbolic data logger is recording the environment and the human gaze on a per-frame basis, enabling offline scene reproduction and replays. Coupled with the physics engine, online monitors (symbolic data loggers) are parsing (using various grammars) and recording events, actions, and their effects in the simulated world.

The initial design of a robotic workcell layout has a large impact on the feasibility and performance of the intended robotic tasks. We define this layout design as a constrained nonlinear optimization problem that aims to optimize the placement of workcell components by minimizing the distance traveled between task sequences while maximizing the robot’s manipulability. Suitable constraints guarantee the reachability as well as the absence of collisions. We solve this optimization problem via a genetic algorithm, and demonstrate it in three scenarios for a dual-arm robotic system that assembles product variants out of aluminum profiles.

Malaria is a worldwide scourge, and the broad deployment of an effective vaccine would improve the lives of millions of people. A vaccine based on Plasmodium falciparum (PfSPZ) sporozoites extracted from the salivary glands of infected mosquitoes shows significant promise. However, the large-scale industrial production of PfSPZ-based vaccines will benefit from automation of the key step of extracting sporozoites from mosquito salivary glands that is currently performed by manual microdissection. In this work, we demonstrate a robotic system prototype for extracting salivary glands from mosquitoes to streamline vaccine production and reduce the need for operators. In the proposed system, mosquitoes are decapitated in an automated robotic pick-place-decapitate process, then a squeezer apparatus extracts mosquito salivary glands from the body. Mosquito detection and body part localization are performed by computer vision methods. The software allows system operation in simulation and on the robotic hardware, which facilitates subsystem development and integration. Experiments show encouraging results with success rates of 93% in robotic mosquito manipulation and 87.1% in salivary gland extraction. The system has the potential to improve the efficiency of PfSPZ vaccine production with significant gains in throughput and reduction in training times for a highly deskilled initial manual step. Further, this system is expected to pave the way for a more mature future system.

In this paper, a new method for offline programming part assembly operations with tight fittings is presented. More specifically, an assembly process trajectory generator with self programming capabilities is developed where the user needs to provide only very sparse and intuitive input. The presented system is added to the existing skill based robot software package VEROSIM. In VEROSIM, the trajectory generator is applied to an industrial test platform for assembling insulin injection devices at the Danish pharmaceutical company Novo Nordisk, where it is shown that the trajectories as expected are executable. Hence, the method is a strong alternative to online approaches such as programming by demonstration.

This paper proposes an automatic and accuracy- enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects from massive point clouds. To obviate user interventions, we propose an automated planar board detection from LiDAR range images. To extract a target completely, we suppress outliers and restore rejected inliers of the target board by introducing a target completion method. We empirically find that range measurements of various LiDARs are mainly skewed by constant offset values. To compensate for this, we suggest a range offset model for each laser channel in calibration procedures. The relative pose between LiDARs and range offsets are jointly estimated by minimizing bi-directional point- to-board distances within the iterative re-weighted least squares (IRLS) framework. To verify the suggested range offset model, we obtain and analyze extensive real-world measurements. By conducting experiments using the various sensor configurations and shapes of boards, we quantitatively and qualitatively confirm accuracy and versatility of the proposed method by comparing with the state-of-the-art LiDAR calibration methods. All the source code and data used in the paper are available at : https://github.com/JunhaAgu/AutoL2LCalib.