ICRA 2021 Papers

We propose a novel objective for penalizing geometric inconsistencies and improving the depth and pose estimation performance of monocular camera images. Our objective is designed using the Wasserstein distance between two point clouds, estimated from images with different camera poses. The Wasserstein distance can impose a soft and symmetric coupling between two point clouds, which suitably maintains geometric constraints and results in a differentiable objective. By adding our objective to those of other state-of-the-art methods, we can effectively penalize geometric inconsistencies and obtain highly accurate depth and pose estimations. Our proposed method was evaluated using the KITTI dataset.

Image guided depth completion is the task of generating a dense depth map from a sparse depth map and a high quality image. In this task, how to fuse the color and depth modalities plays an important role in achieving good performance. This paper proposes a two-branch backbone that consists of a color-dominant branch and a depth-dominant branch to exploit and fuse two modalities thoroughly. More specifically, one branch inputs a color image and a sparse depth map to predict a dense depth map. The other branch takes as inputs the sparse depth map and the previously predicted depth map, and outputs a dense depth map as well. The depth maps predicted from two branches are complimentary to each other and therefore they are adaptively fused. In addition, we also propose a simple geometric convolutional layer to encode 3D geometric cues. The geometric encoded backbone conducts the fusion of different modalities at multiple stages, leading to good depth completion results. We further implement a dilated and accelerated CSPN++ to refine the fused depth map efficiently. The proposed full model ranks 1st in the KITTI depth completion online leaderboard at the time of submission. It also infers much faster than most of the top ranked methods. The code of this work is available at https://github.com/JUGGHM/PENet_ICRA2021.

In this paper, we address the Perception– Constrained Model Predictive Control (PCMPC) and state estimation problems for quadrotors with cable suspended payloads using a single camera and Inertial Measurement Unit (IMU). We design a receding–horizon control strategy for cable suspended payloads directly formulated on the system manifold configuration space SE (3) ×S2. The approach considers the system dynamics, actuator limits and the camera’s Field Of View (FOV) constraint to guarantee the payload’s visibility during motion. The monocular camera, IMU, and vehicle’s motor speeds are combined to provide estimation of the vehicle’s states in 3D space, the payload’s states, the cable’s direction and velocity. The proposed control and state estimation solution runs in real-time at 500 Hz on a small quadrotor equipped with a limited computational unit. The approach is validated through experimental results considering a cable suspended payload trajectory tracking problem at different speeds.

We present PATHoBot an autonomous crop surveying and intervention robot for glasshouse environments. The aim of this platform is to autonomously gather high quality data and also estimate key phenotypic parameters. To achieve this we retro-fit an off-the-shelf pipe-rail trolley with an array of multi-modal cameras, navigation sensors and a robotic arm for close surveying tasks and intervention. In this paper we describe PATHoBot design choices made to ensure proper operation in a commercial glasshouse environment. As a surveying platform we collect a number of datasets which include both sweet pepper and tomatoes. We show how PATHoBot enables novel surveillance approaches by first improving our previous work on fruit counting by incorporating wheel odometry and depth information. We find that by introducing re-projection and depth information we are able to achieve an absolute improvement of 20 points over the baseline technique in an "in the wild" situation. Finally, we present a 3D mapping case study, further showcasing PATHoBot’s crop surveying capabilities.

Breast cancer affects one out of eight women. Ultrasound (US) plays an important role in the diagnostic workflow, especially during the biopsy phase, in which tissue is extracted from the lesion for further analysis. The extension from 2D to 3D US acquisitions has multiple benefits including enhanced lesion localization and improved registration with MRI data. Current commercial 3D US systems lack the ability to preserve the breast’s original shape. Robotic US scanners follow tailored trajectories and produce high quality volumes by accurate localization of 2D slices captured with a conventional linear probe. Current methods require a patient specific model to plan the scanning trajectory.In this study we investigate how to change the direction of the scanning trajectory based on US feedback, such that no patient specific model is required to perform a scan. In our method, the scanning trajectory is kept tangent to the breast based on confidence maps of the US images and an estimation of current radius of curvature of the surface. We evaluated our approach on a realistic breast phantom. The robot revolves around the breast without prior knowledge of its shape. In ten scans, the RMS error between the probe’s scanning plane and the breast’s surface normal is 12.6° out-of-plane, and 4.3° in-plane. A 3D US reconstruction shows the acquired data. This is a step forward to fully autonomous, high quality robotic US volume acquisitions.

Accurate state and uncertainty estimation is imperative for mobile robots and self driving vehicles to achieve safe navigation in pedestrian rich environments. A critical component of state and uncertainty estimation for robot navigation is to perform robustly under out-of-distribution noise. Traditional methods of state estimation decouple perception and state estimation making it difficult to operate on noisy, high dimensional data. Here, we describe an approach that combines the expressiveness of deep neural networks with principled approaches to uncertainty estimation found in recursive filters. We particularly focus on techniques that provide better robustness to out-of-distribution noise and demonstrate applicability of our approach on two scenarios: a simple noisy pendulum state estimation problem and real world pedestrian localization using the nuScenes dataset [1]. We show that our approach improves state and uncertainty estimation compared to baselines while achieving approximately 3× improvement in computational efficiency.

Origami, the art of folding paper, can impart useful design inspirations to the creation of mechanical structures and mechanisms. Bistability is a useful property for origami designs, which can help compartmentalize different actuations and stiffness tuning regimes. Given the benefits of bistability, we investigated origami designs used to build robots and deployable structures. We show snap-through bistable designs applied to parallel and curved mechanisms, which are of value to robotic mechanical design. The designs proposed were investigated through geometry analysis and stress-strain experiments. The origami designs were modified to show that the mechanical properties of our bistable designs can be modulated. Initial actuation utilized magnetic and tendon-driven mechanisms for the parallel and curved structures, respectively. We anticipate that these bistable snap-through designs can contribute to deployable mechanisms and give such devices additional capabilities in mechanical response tuning.

Colorectal cancer incidence has been steadily rising worldwide. Magnetic colonoscopes provide new approaches to conduct colon inspection and treatment. This paper presents a novel electromagnetically actuated soft-tethered colonoscope to achieve precise and stable orientation control. An inflated balloon is designed to eliminate the unpredictable disturbance of the floating tether. A 2OR Pseudo-Rigid-Body (PRB) model of the soft tether is developed to analyze the relationship between the tether deflection and applied force and torque. A closed-loop control framework is constructed with visual position feedback. Experiments are first conducted to validate the assumption of the PRB model and the efficacy of the magnetic field model. Then, trajectory tracking tasks and disturbance rejection tests are performed to validate the feasibility of the proposed solution and closed-loop control. Results show that the colonoscope can stably and accurately orient to the desired orientation with an absolute mean position error of less than 0.5 mm and an average velocity of 3.5 mm/s. The distal tip can quickly re-stabilize to the desired orientation even when a large disturbance exists.

Prior work on generating explanations in a planning context has focused on providing the rationale behind an AI agent’s decision-making. While these methods offer the right explanations, they fail to heed the cognitive requirement of understanding an explanation from the explainee or human’s perspective. In this work, we set out to address this issue by considering the order for communicating information in an explanation, or the progressiveness of making explanations. Progression is the notion of building complex concepts on simpler ones, which is known to benefit learning. In this work, we investigate a similar effect when an explanation is composed of multiple parts that are communicated sequentially. The challenge here lies in determining the order for receiving different parts of an explanation that would assist in understanding. Given the sequential nature, a formulation based on goal-based MDP is presented. The reward function of this MDP is learned via inverse reinforcement learning based on training data. We evaluated our approach in an escape-room domain to demonstrate its effectiveness. Upon analyzing the results, it revealed that the desired order arises strongly from both domain-dependent and independence features. This result confirmed our expectation that the process of understanding an explanation for planning tasks was progressive and context dependent. We also showed that the explanations generated using the learned rewards achieved better task performance and simultaneously reduced cognitive load. These results shed light on designing explainable robots across various domains.

Robotic manipulators are increasingly being used to perform additive manufacturing. The accuracy of a built part is dependent on the trajectory execution error of the manipulator. For articulated manipulators, the trajectory execution error and achievable build accuracy vary considerably over the workspace. Therefore, the build accuracy depends on where the part is placed in the manipulator workspace. If the part is small compared to the manipulator workspace, its placement can be optimized to improve the accuracy. This paper provides experimental evidence that the placement of the parts changes its build accuracy. We model the trajectory execution error of the manipulator for additive manufacturing. We validate these errors by comparing the predicted errors with the experimental errors. Finally, we present an algorithm to optimize the part placement for improving built part accuracy during robot-based additive manufacturing.

We introduce an optimization method for recent approaches on keypoint-based pose estimation of robotic manipulators utilizing monocular images. The method takes into account the segmented shape of the robot using Convolutional Neural Networks and a keypoint refinement through a set of score values. To this end, the primal 2D keypoint detection is exploited as an initial guess for further shape-based keypoint adjustments. Afterwards, the overall methods incorporates a perspective-n-point algorithm using 3D point correspondences that are derived by forward kinematics. We hereby complement an existing public dataset with annotated segmentations of a Universal Robot UR5 manipulator. The evaluation of the optimization approach shows clearly that noise on the initial key-point detection can be suppressed and minimized. Furthermore, the overall success rate of the perspective transformation can be enhanced towards more than 90%. Thus, the overall methods is applicable for single-shot pose estimation. The evaluation results also show a significant reduction of the standard deviation of the resulting pose estimation. Consequently, the proposed optimization positively affects applicability and precision.

Although existing cellular network base stations are typically immobile, the recent development of small form factor base stations and self driving cars has enabled the possibility of deploying a team of continuously moving base stations that can reorganize the network infrastructure to adapt to changing network traffic usage patterns. Given such a system of mobile base stations (MBSes) that can freely move on the road, how should their path be planned in an effort to optimize the experience of the users? This paper addresses this question by modeling the problem as a Markov Decision Process where the actions correspond to the MBSes deciding which direction to go at traffic intersections; states corresponds to the position of MBSes; and rewards correspond to minimization of packet loss in the network. A Monte Carlo Tree Search (MCTS)-based anytime algorithm that produces path plans for multiple base stations while optimizing expected packet loss is proposed. Simulated experiments in the city of Verdun, QC, Canada with varying user equipment (UE) densities and random initial conditions show that the proposed approach consistently outperforms myopic planners, and is able to achieve near-optimal performance.

A method for planning and controlling the somersault motion of a quadruped robot is proposed in this paper. The method divides the somersault motion into 5 stages according to intuitive understanding. Based on the simplified dynamic model, the linear programming method is used to obtain the maximum ground reaction force under the constraints of joint torque and friction cone, and then the optimal leg thrusting trajectory is obtained by double integration of the acceleration. In order to achieve the buffered landing of the robot after somersault, a whole body controller based on null space projection is used to obtain the optimal joint torque under the constraints of the robot’s foot position, torso position and torso posture. The somersault motion control method proposed in this paper has been verified by the dynamics simulation software Webots and quadruped robot platform Yobogo. The results show that the robot can complete stable front flip and back flip under the constraints of joint output torque and foot motion space constraints.

UV radiation has been used as a disinfection strategy to deactivate a wide range of pathogens, but existing irradiation strategies do not ensure sufficient exposure of all environmental surfaces and/or require long disinfection times. We present a near-optimal coverage planner for mobile UV disinfection robots. The formulation optimizes the irradiation time efficiency, while ensuring that a sufficient dosage of radiation is received by each surface. The trajectory and dosage plan are optimized taking collision and light occlusion constraints into account. We propose a two-stage scheme to approximate the solution of the induced NP-hard optimization, and, for efficiency, perform key irradiance and occlusion calculations on a GPU. Empirical results show that our technique achieves more coverage for the same exposure time as strategies for existing UV robots, can be used to compare UV robot designs, and produces near-optimal plans.

Keyhole neurosurgery is challenging, due to the complex anatomy of the brain and the inherent risk of damaging vital structures while reaching the surgical target. This paper presents a path planner for safe and effective neurosurgical interventions. The strengths of the proposed framework lay in the integration of multiple risk structures combined into a deductive method for fast and intuitive user interaction, and a modular architecture. The tool is intended to support neurosurgeons at quickly determining the most appropriate surgical trajectory through the brain matter with minimized risk; the user interface guides the user through the decision making process and helps save planning time of neurosurgical interventions. Risk structures and trajectories can be visualized in an intuitive way, thanks to a 3D brain surgery simulator developed with Unity. A qualitative evaluation with clinical experts shows the practical relevance, while a quantitative performance and functionality analysis proves the robustness and effectiveness of the system with respect to literature.

This paper presents a non-linear optimization method for trajectory planning of tethered aerial robots. Particularly, the paper addresses the planning problem of an unmanned aerial vehicle (UAV) linked to an unmanned ground vehicle (UGV) by means of a tether. The result is a collision-free trajectory for UAV and tether, assuming the UGV position is static. The optimizer takes into account constraints related to the UAV, UGV and tether positions, obstacles and temporal aspects of the motion such as limited robot velocities and accelerations, and finally the tether state, which is not required to be tense. The problem is formulated in a weighted multi-objective optimization framework. Results from simulated scenarios demonstrate that the approach is able to generate obstacle-free and smooth trajectories for the UAV and tether.

For robots to leave the safety of the laboratory and explore the world, maneuverability will need to be mastered. However, transient motions, such as rapid acceleration and deceleration, have received little attention in the literature. This is mainly due to the complexity of analyzing these high dimensional systems that have no closed-form solution which makes controller design a non-trivial task. One method is to utilize heuristic control inspired by animal locomotion (or intuition), but these may not be optimal for a given task. Here, we take the novel approach and leverage trajectory optimization methods to enable us to identify heuristic controllers for the task of transient locomotion. Specifically, we investigate acceleration to a steady-state gait as well as decelerating from a steady-state gait to rest. These identified heuristic controllers were then validated on a hybrid pneumatic-electric monopod robot. Our initial results indicate that a Raibert controller is in fact the energy optimal policy for transient maneuvers.

Unlike loose coupling approaches and the EKF-based approaches in the literature, we propose an optimization-based visual-inertial SLAM tightly coupled with raw Global Navigation Satellite System (GNSS) measurements, a first attempt of this kind in the literature to our knowledge. More specifically, reprojection error, IMU pre-integration error and raw GNSS measurement error are jointly minimized within a sliding window, in which the asynchronism between images and raw GNSS measurements is accounted for. In addition, issues such as marginalization, noisy measurements removal, as well as tackling vulnerable situations are also addressed. Experimental results on public dataset in complex urban scenes show that our proposed approach outperforms state-of-the-art visual-inertial SLAM, GNSS single point positioning, as well as a loose coupling approach, including scenes mainly containing low-rise buildings and those containing urban canyons.

We propose a safety control approach based on the online optimal scaling of the size of bounding volumes used as dynamic safety zones for collaborative robotics. Intersection tests between bounding volumes surrounding robot and human allow the safety controller to identify possible collisions. Our proposed approach optimizes online smooth stop trajectories, to be engaged if a potential collision is detected. Unlike other approaches, the robot dynamics and its torque constraints are here considered to plan optimal trajectories that minimize the size of safety zones surrounding the robot. Simulation results on a validated model of a robot with seven degrees-of-freedom verify the feasibility of the proposed approach.