IROS 2020 Papers

Mobile robotic gas distribution mapping (GDM) is a useful tool for hazardous scene assessment where a quick and accurate representation of gas concentration levels is required throughout a staging area. However, research in robotic path planning for GDM has primarily focused on mapping in open spaces or estimating the source term in dispersion models. Whilst this may be appropriate for environment monitoring in general, the vast majority of GDM applications involve obstacles, and path planning for autonomous robots must account for this. This paper aims to tackle this challenge by integrating a GDM function with an informative path planning framework. Several GDM methods are explored for their suitability in cluttered environments and the GMRF method is chosen due to its ability to account for obstacle interactions within the plume. Based on the outputs of the GMRF, several reward functions are proposed for the informative path planner. These functions are compared to a lawnmower sweep in a high fidelity simulation, where the RMSE of the modelled gas distribution is recorded over time. It is found that informing the robot with uncertainty, normalised concentration and time cost, significantly reduces the time required for a single robot to achieve an accurate map in a large-scale, urban environment. In the context of a hazardous gas release scenario, this time reduction could save lives as well as further gas ingress.

Multi-modal estimation systems have the advantage of increased accuracy and robustness. To achieve accurate sensor fusion with these types of systems, a reliable extrinsic calibration between each sensor pair is critical. This paper presents a novel self-calibration framework for lidar-inertial systems. The key idea of this work is to use an informative path planner to find the admissible path that produces the most accurate calibration of such systems in an unknown environment within a given time budget. This is embedded into a simultaneous localization, mapping and calibration lidar-inertial system, which involves challenges in dealing with agile motions for excitation and large amount of data. Our approach has two stages: firstly, the environment is explored and mapped following a pre-defined path; secondly, the map is exploited to find a continuous and differentiable path that maximises the information gain within a sampling-based planner. We evaluate the proposed self-calibration method in a simulated environment and benchmark it with standard predefined paths to show its performance.

The availability of real-world datasets is the prerequisite for developing object detection methods for autonomous driving. While ambiguity exists in object labels due to error-prone annotation process or sensor observation noises, current object detection datasets only provide deterministic annotations without considering their uncertainty. This precludes an in-depth evaluation among different object detection methods, especially for those that explicitly model predictive probability. In this work, we propose a generative model to estimate bounding box label uncertainties from LiDAR point clouds, and define a new representation of the probabilistic bounding box through spatial distribution. Comprehensive experiments show that the proposed model represents uncertainties commonly seen in driving scenarios. Based on the spatial distribution, we further propose an extension of IoU, called the Jaccard IoU (JIoU), as a new evaluation metric that incorporates label uncertainty. Experiments on the KITTI and the Waymo Open Datasets show that JIoU is superior to IoU when evaluating probabilistic object detectors.

In this paper, we propose three novel Hardware-in-the-loop simulation (HLS) methods for a fully-actuated orbital robot in the presence of external interactions using On-Ground Facility Manipulators (OGFM). In particular, a fixed-base and a vehicle-driven manipulator are considered in the analyses. The key idea is to describe the orbital robot's dynamics using the Lagrange-Poincaré(LP) equations, which reveal a block-diagonalized inertia. The resulting advantage is that noisy joint acceleration/torque measurements are avoided in the computation of the spacecraft motion due to manipulator interaction even while considering external forces. The proposed methods are a consequence of two facilitating theorems, which are proved herein. These theorems result in two actuation maps between the simulated orbital robot and the physical OGFM. The chief advantage of the proposed methods is physical consistency without level-set assumptions on the momentum map. We validate this through experiments on both types of OGFM in the presence of external forces. Finally, the effectiveness of our approach is validated through a HLS of a fully-actuated orbital robot while interacting with the environment.

Recognising in what type of environment one is located is an important perception task. For instance, for a robot operating indoors it is helpful to be aware whether it is in a kitchen, a hallway or a bedroom. Existing approaches attempt to classify the scene based on 2D images or 2.5D range images. Here, we study scene recognition from 3D point cloud (or voxel) data, and show that it greatly outperforms methods based on 2D birds-eye views. Moreover, we advocate multi-task learning as a way to improve scene recognition, building on the fact that the scene type is highly correlated with the objects in the scene, and therefore with its semantic segmentation into different object classes. In a series of ablation studies, we show that successful scene recognition is not just the recognition of individual objects unique to some scene type (such as a bathtub), but depends on several different cues, including coarse 3D geometry, colour, and the (implicit) distribution of object categories. Moreover, we demonstrate that surprisingly sparse 3D data is sufficient to classify indoor scenes with good accuracy.

We present a robotic grasping system that uses a single external monocular RGB camera as input. The object-to-robot pose is computed indirectly by combining the output of two neural networks: one that estimates the object-to-camera pose, and another that estimates the robot-to-camera pose. Both networks are trained entirely on synthetic data, relying on domain randomization to bridge the sim-to-real gap. Because the latter network performs online camera calibration, the camera can be moved freely during execution without affecting the quality of the grasp. Experimental results analyze the effect of camera placement, image resolution, and pose refinement in the context of grasping several household objects. We also present results on a new set of 28 textured household toy grocery objects, which have been selected to be accessible to other researchers. To aid reproducibility of the research, we offer 3D scanned textured models, along with pre-trained weights for pose estimation.

Global navigation satellite system (GNSS) has been considered as a panacea for positioning and tracking since the last decade. However, it suffers from severe limitations in terms of accuracy, particularly in highly cluttered and indoor environments. Though real-time kinematics (RTK) supported GNSS promises extremely accurate localisation, employing such services are expensive, fail in occluded environments and are unavailable in areas where cellular base stations are not accessible. It is, therefore, necessary that the GNSS data is to be filtered if high accuracy is required. Thus, this article presents a GNSS-based particle filter that exploits the spatial constraints imposed by the environment. In the proposed setup, the state prediction of the sample set follows a restricted motion according to the topological map of the environment. This results in the transition of the samples getting confined between specific discrete points, called the topological nodes, defined by a topological map. This is followed by a refinement stage where the full set of predicted samples goes through weighting and resampling, where the weight is proportional to the predicted particle's proximity with the GNSS measurement. Thus, a discrete space continuous-time Bayesian filter is proposed, called the Topological Particle Filter (TPF).The proposed TPF is put to test by localising and tracking fruit pickers inside polytunnels. Fruit pickers inside polytunnels can only follow specific paths according to the topology of the tunnel. These paths are defined in the topological map of the polytunnels and are fed to TPF to tracks fruit pickers. Extensive datasets are collected to demonstrate the improved discrete tracking of strawberry pickers inside polytunnels thanks to the exploitation of the environmental constraints.

Limited flight range is a common problem for multicopters. To alleviate this problem, we propose a method for finding the optimal speed and heading of a multicopter when flying a given path to achieve the longest flight range. Based on a novel multivariable extremum seeking controller with adaptive step size, the method (a) does not require any power consumption model of the vehicle, (b) can adapt to unknown disturbances, (c) can be executed online, and (d) converges faster than the standard extremum seeking controller with constant step size. We conducted indoor experiments to validate the effectiveness of this method under different payloads and initial conditions, and showed that it is able to converge more than 30% faster than the standard extremum seeking controller. This method is especially useful for applications such as package delivery, where the size and weight of the payload differ for different deliveries and the power consumption of the vehicle is hard to model.

In the paper, a pair of auto-tuning methods for fixed-parameter controllers is presented, in application to multirotor unmanned aerial vehicles (UAVs) control. In both cases, the automatized process of searching the best altitude controller parameters is carried out with the use of a modified golden-search method, for a selected cost function, during the flight of a pair of UAVs. All the calculations are performed in real-time in the iterative manner using only basic sensory information available concerning current altitude information for a pair of UAVs. The auto-tuning process of the controller is characterized by neglectfully low computational demand, and the parameters are obtained rapidly with no dynamic model of a UAV needed. In both methods, by using a pair of UAVs in tuning process, the level of control performance can be increased, what has been proved by means of multiple outdoor experiments. The first method increases precision of the obtained controller parameters by averaging sensory information over a pair of UAVs, whereas in the second, by exchanging measurement information between the units, the search space is explored faster. The latter is of special importance when seeking the best controller parameters, what is especially expected when a limited experiment duration of multirotor UAVs is taken into account.

This paper describes a method that corrects errors of a VSLAM-estimated trajectory for cars driving in GPS-denied environments, by applying constraints from public databases of geo-tagged images (Google Street View, Mapillary, etc). The method, dubbed Appearance-based Geo-Alignment for Simultaneous Localisation and Mapping (AGA-SLAM), encodes the available image database as an appearance map, which represents the space with a compact holistic descriptor for each image plus its associated geo-tag. The VSLAM trajectory is corrected on-line by incorporating constraints from the recognized places along the trajectory into a position-based optimization framework. The paper presents a seamless formulation to combine local and absolute metric observations with associations from Visual Place Recognition. The robustness of the holistic image descriptor to changes due to weather or illumination variations ensures a long-term consistent method to improve car localization. The proposed method has been extensively evaluated on more than 70 sequences from 4 different datasets, proving out its effectiveness and endurance to appearance challenges.

Robots perceive their environment using various sensor modalities, e.g., vision, depth, sound or touch. Each modality provides complementary information for perception. However, while it can be assumed that all modalities are available for training, when deploying the robot in real-world scenarios the sensor setup often varies. In order to gain flexibility with respect to the deployed sensor setup we propose a new multimodal approach within the framework of contrastive learning. In particular, we consider the case of learning from RGB-D images while testing with one modality available, i.e., exclusively RGB or depth. We leverage contrastive learning to capture high-level information between different modalities in a compact feature embedding. We extensively evaluate our multimodal contrastive learning method on the Falling Things dataset and learn representations that outperform prior methods for RGB-D object recognition on the NYU-D dataset. Our code and details on the used datasets are available at: https://github.com/meyerjo/MultiModalContrastiveLearning.

In this paper, an improved motion planning scheme is proposed for surgical robot control with multiple active constraints, including joint constraints, joint velocity constraints and remote center of motion constraints. It introduces an improved recurrent neural network (RNN) to optimize the online motion planning respect to multiple constraints. The demonstrated surgical operation trajectory is derived using teaching by demonstration. An improved motion planning scheme using the novel recurrent neural network is then designed to achieve the accurate task tracking under the multiple constraints. The general quadratic performance index is adopted to represent the constraints. Finally, the effectiveness of the proposed algorithm is demonstrated using KUKA LWR4+ robot in a lab setup environment.

This paper describes a novel low-level controller for the lower-limb exoskeleton Atalante. The controller implemented on the commercialized product Atalante works under the assumption of full rigidity, performing position control through decentralized joint PIDs. However, this controller is unable to tackle the presence of flexibilities in the system, which cause static errors and undesired oscillations. We modify this controller by leveraging estimations of the position and velocity of the flexibilities, readily available on Atalante through the use of strapdown IMUs. Instead of considering feedback on the motor position only, we perform feedback on both the joint position and the flexibility angle, keeping a decentralized approach. This enables compensation of both the static error present at rest, and rapid damping of the oscillations. To tune the gains of the proposed controller, we use a linearized model of an elastic joint to which we apply a steady-state LQR, which creates desirable robustness to the flexible model. The proposed controller is experimentally validated through various single support experiments on Atalante, either empty or with a user. In all cases, the proposed controller outperforms the state-of- the-art controller, providing improved trajectory tracking and disturbance rejection.

This work studies redundant actuation for both trajectory tracking and disturbance rejection on flexible cable-driven parallel robots (CDPR). High dynamics/bandwidth unidirectional force generators, like air propellers, are used in combination with conventional but slower cable winding winches. To optimally balance the action of the two types of actuation within their saturation constraints, a model predictive controller is used. Experiments show the added value of on-board propulsion units with respect to winch-only control in order to improve the overall CDPR dynamic behavior.

Autonomous round trip missions arise as an interesting topic since ESA and NASA agreed to bring soil samples from Mars. This work proposes a new method to improve autonomous rover guidance for this kind of missions. It is focused on the use of dynamically updated cost maps that are used to plan the rover path for a round-trip. The main advantage of the proposed method is the use of gathered information by the rover during the traverse. So, the cost map is updated in two ways, the first one, related to the encountered obstacles, which are included within the cost map; and the second one that uses terrain features to assign different costs to different patches of a segmented terrain. The generated cost map is then used to plan the return path based on the previously obtained information from the rover. The proposed method has been validated first in a simulation environment with Software in the Loop. The implementation is finally validated by in-field testing in a Mars-like environment. Results show the return traverse is improved by means of the proposed method.

In this paper, we propose the stabilization strategy for a soft landing in a biped walking using impedance control and the optimization-based whole-body control framework. Even though proper contact forces and desired trajectories of the robot are given, the robot can be unstable easily if unexpected forces are applied to the robot or impulsive contact force is produced in the landing state while the robot is walking. Therefore, the impedance control approach using contact forces is performed to obtain the modified references that regulate the modified desired position, velocity and acceleration of the swing foot, and improves the walking stability. Moreover, we perform a whole-body control using quadratic programming (QP) that tracks the modified trajectories constrained with the centroidal momentum dynamics. To validate the algorithm, a walking task on uneven terrain using a humanoid robot is shown.

We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, stochastic, nonlinear dynamics. Our approach extends the Normalizing Flows framework to learn stable Stochastic Differential Equations. We prove the Lyapunov stability for a class of Stochastic Differential Equations and we propose a learning algorithm to learn them from a set of demonstrated trajectories. Our model extends the set of stable dynamical systems that can be represented by state-of-the-art approaches, eliminates the Gaussian assumption on the demonstrations, and outperforms the previous algorithms in terms of representation accuracy. We show the effectiveness of our method with both standard datasets and a real robot experiment.

A method to identify the kinematics model of a human hand that less suffers from the skin artifact is proposed based on a fact that the movements of nails with respect to the corresponding fingertip bones are much smaller than that of skin. It consists of two stages. In the first (individual) stage, the most likely combination of joint assignments and angles of each finger is identified through a dual-phase least squares method (LSM), where the joint angles are estimated in the inner LSM and the joint assignments in the outer LSM, from the movement of the hand dorsum with respect to the base coordinate frame attached to each nail. In the second (merging) stage, kinematic models of each finger are merged so as to compromise the estimated movements of the hand dorsum by them also through the dual-phase LSM. It is shown that the identified joint assignments have an advantage over several existing anthropomorphic robot hands based on the distribution of pinchability (DOP), which is also proposed in this paper as a novel index to evaluate the ability of in-hand manipulation.