IROS 2019 Papers

Despite the impressive progress being made in autonomous vehicles, human drivers will remain ubiquitous in the imminent years. Therefore, intelligent hybrid vehicular systems must be aware of the interactions between humans and the environment (e.g., sound, vibration, speed, etc.). In this paper, we evaluate the effect of acoustic annoyance on drivers in a real-world driving study. We found significant differences in driving styles elicited by annoying acoustics and present an online classifier that uses onboard inertial measurement unit measurements to distinguish whether a driver is annoyed with 77% accuracy. Moreover, we directly measured the forces applied on the passenger with a pressure mat lined on the car seat, and empirically confirm that our proposed passenger dynamics model is reasonable. However, due to our acoustically induced driving styles not being polarizing enough, we were unable to show that passengers' self-reported ride comfort changed with acoustic annoyance.

In this article, the adhesion modeling and control case of a Vortex Climbing Robot (VCR) is investigated against a surface of variable orientations. The critical adhesion force exerted from the implemented Vortex Actuator (VA) and the VCR's achievable payload are analyzed under 3-DOF rotations of the test surface, while extracted from both geometrical analysis and dynamically-simulated numerical results. A model-based control scheme is later proposed, with the goal of achieving adhesion while the VCR remains immobilized, limiting the power consumption and compensating for disturbances (e.g. moving cables) leading to Center-of-Mass (CoM) changes. Finally, the model-based control scheme is experimentally evaluated, with the VCR prototype on a rotating and moving flat surface. The presented results support the use of the proposed methodology in climbing robots targeting inspection and maintenance of stationary surfaces (flat, curved etc.), as well as future robotic solutions operating on moving structures (e.g. ships, cranes, folding bridges).

Small, low-speed fixed-wing Unmanned Aerial Vehicles (UAVs) operating autonomously, beyond-visual-line-of-sight (BVLOS) will inevitably encounter winds rising to levels near or exceeding the vehicles' nominal airspeed. In this paper, we develop a nonlinear lateral-directional path following guidance law with explicit consideration of online wind estimates. Energy efficient airspeed reference compensation logic is developed for excess wind scenarios (i.e. when the wind speed rises above the airspeed), enabling either mitigation, prevention, or over-powering of excess wind induced run-away from a given path. The developed guidance law is demonstrated on a representative small, low-speed test UAV in two flight experiments conducted in mountainous regions of Switzerland with strong, turbulent wind conditions, gusts reaching up to 13 meters per second. We demonstrate track-keeping errors of less than 1 meter consistently maintained during a representative duration of gusting, excess winds and a mean ground speed undershoot of 0.5 meters per second from the commanded minimum forward ground speed demonstrated in over 5 minutes of the showcased flight results.

Ground surface detection in point cloud is widely used as a key module in autonomous driving systems. Different from previous approaches which are mostly developed for lidars with high beam resolution, e.g. Velodyne HDL-64, this paper proposes ground detection techniques applicable to much sparser point cloud captured by lidars with low beam resolution, e.g. Velodyne VLP-16. The approach is based on the RANSAC scheme of plane fitting. Inlier verification for plane hypotheses is enhanced by exploiting the point-wise tangent, which is a local feature available to compute regardless of the density of lidar beams. Ground surface which is not perfectly planar is fitted by multiple (specifically 4 in our implementation) disjoint plane regions. By assuming these plane regions to be rectanglar and exploiting the integral image technique, our approach approximately finds the optimal region partition and plane hypotheses under the RANSAC scheme with real-time computational complexity.

This article introduces and evaluates two decentralized data sharing algorithms for multi-robot visual-inertial simultaneous localization and mapping (VI-SLAM): Factor Sparsification for Visual-Inertial Packets (FS-VIP) and Min-K-Cover Selection for Visual-Inertial Packets (MKCS-VIP). Both methods make robots regularly build and exchange data packets which describe the successive portions of their map, but rely on distinct paradigms. While FS-VIP builds on consistent marginalization and sparsification techniques, MKCSVIP selects raw visual and inertial information which can best help to perform a faithful and consistent re-estimation while reducing the communication cost. Performances in terms of accuracy and communication loads are evaluated on multi-robot scenarios built on both available (EUROC) and custom datasets (SOTTEVILLE).

Pushing is a fundamental robotic skill. Existing work has shown how to exploit models of pushing to achieve a variety of tasks, including grasping under uncertainty, in-hand manipulation and clearing clutter. Such models, however, are approximate, which limits their applicability.Learning-based methods can reason directly from raw sensory data with accuracy, and have the potential to generalize to a wider diversity of scenarios. However, developing and testing such methods requires rich-enough datasets. In this paper we introduce Omnipush, a dataset with high variety of planar pushing behavior.In particular, we provide 250 pushes for each of 250 objects, all recorded with RGB-D and a high precision tracking system. The objects are constructed so as to systematically explore key factors that affect pushing-the shape of the object and its mass distribution-which have not been broadly explored in previous datasets, and allow to study generalization in model learning.Omnipush includes a benchmark for meta-learning dynamic models, which requires algorithms that make good predictions and estimate their own uncertainty. We also provide an RGB video prediction benchmark and propose other relevant tasks that can be suited with this dataset. Data and code are available at https://web.mit.edu/mcube/omnipush-dataset/.

Estimating the state of a deformable object is crucial for robotic manipulation, yet accurate tracking is challenging when the object is partially-occluded. To address this problem, we propose an occlusion-robust RGBD sequence tracking framework based on Coherent Point Drift (CPD). To mitigate the effects of occlusion, our method 1) Uses a combination of locally linear embedding and constrained optimization to regularize the output of CPD, thus enforcing topological consistency when occlusions create disconnected pieces of the object; 2) Reasons about the free-space visible by an RGBD sensor to better estimate the prior on point location and to detect tracking failures during occlusion; and 3) Uses shape descriptors to find the most relevant previous state of the object to use for tracking after a severe occlusion. Our method does not rely on physics simulation or a physical model of the object, which can be difficult to obtain in unstructured environments. Despite having no physical model, our experiments demonstrate that our method achieves improved accuracy in the presence of occlusion as compared to a physics-based CPD method while maintaining adequate run-time.

The paper describes a control framework using a capacitive skin for advanced and safe human-robot interactions. The skin consists of high density capacitive sensors which are able to detect objects at close distance. We propose a multitask kinematic control scheme which manages at once the control task of the end-effector and the obstacle avoidance task by using the capacitive skin measurements. To solve the multiobjective problem, a Weight-Prioritized solution based on a QP formalism is adopted. The robustness of the kinematic controller is enhanced by adding a local joint reconfiguration when the robot moves close to a singularity. Experimental results on manipulator arm equipped with the capacitive sensors are presented in realistic situations of human-robot interactions.

This study performs a nonlinear observability analysis on range assisted inertial navigation system (INS) for quadrotor micro-aerial vehicles (MAV). The INS is formulated incorporating the quadrotor dynamics with aerodynamic drag forces. The observability analysis is carried out for cases where three and two range measurements are available. The analysis facilitates the range assisted localization of MAVs when there are less than four range measurements are available. The primary objective of this study is to identify the conditions under which the INS becomes unobservable, and these conditions are validated through numerical simulation. The main contributions of this paper are as follows, 1. Nonlinear observability analysis of the range assisted INS for quadrotor MAVs. 2. Theoretical derivation and numerical validation of unobservable conditions for three and two range cases. 3. Experimental validations of estimator performance.

In this study, we aim at grasping a single target object in a cluttered environment using a robotic arm. While dexterous grasp for various shapes of objects is not considered in this work, we focus on developing the method to mitigate clutter near the target object as soon as quickly. For this purpose, we propose a method to generate nonlinear pushing motions for object singulation based on an off-the-shelf machine learning algorithm and a typical semantic segmentation algorithm. Through experiments, we show that the success rate of robotic grasping is considerably improved by the proposed pushing behavior. And notably, the nonlinear pushing trajectories allows the robot to perform singulation of the target object in a cluttered environment with fewer trials than linear pushing usually pursued in related works.

This paper considers the problem of rearrangement planning, i.e finding a sequence of manipulation actions that displace multiple objects from an initial configuration to a given goal configuration. Rearrangement is a critical skill for robots so that they can effectively operate in confined spaces that contain clutter. Examples of tasks that require rearrangement include packing objects inside a bin, wherein objects need to lay according to a predefined pattern. In tight bins, collision-free grasps are often unavailable. Nonprehensile actions, such as pushing and sliding, are preferred because they can be performed using minimalistic end-effectors that can easily be inserted in the bin. Rearrangement with nonprehensile actions is a challenging problem as it requires reasoning about object interactions in a combinatorially large configuration space of multiple objects. This work revisits several existing rearrangement planning techniques and introduces a new one that exploits nested nonprehensile actions by pushing several similar objects simultaneously along the same path, which removes the need to rearrange each object individually. Experiments in simulation and using a real Kuka robotic arm show the ability of the proposed approach to solve difficult rearrangement tasks while reducing the length of the end-effector's trajectories.

The recent emergence of object proposal algorithms in the computer vision community shows great promise to addressing difficult problems in robotic such as object discovery and salient object detection. However, it is difficult to determine how these algorithms actually perform for real-world robot vision applications, because the standard evaluation protocol uses datasets which do not adequately account for real-world noise (motion blur, occlusion, etc.). We evaluated several state-of-the-art object proposal algorithms using naturalistic datasets from the robotics community, and found a substantial performance drop across all algorithms. This suggests that many object proposal algorithms are not as generalizable as the computer vision literature purports, which can have a significant impact on how they are applied to robotics. We also conducted a study on how each algorithm is influenced by specific kinds of real-world robot vision challenges, including variable brightness, gamma correction, Gaussian blur, and Gaussian noise. Our results provide insight into certain weaknesses of object proposal algorithms, which can be used to gauge how they might be suitable for different robotics applications. It is our intent that this work will motivate future research about how to design more flexible and robust object proposal algorithms for the robotics community.

We address the problem of planning the placement of a rigid object with a dual-arm robot in a cluttered environment. In this task, we need to locate a collision-free pose for the object that a) facilitates the stable placement of the object, b) is reachable by the robot and c) optimizes a user-given placement objective. In addition, we need to select which robot arm to perform the placement with. To solve this task, we propose an anytime algorithm that integrates sampling-based motion planning with a novel hierarchical search for suitable placement poses. Our algorithm incrementally produces approach motions to stable placement poses, reaching placements with better objective as runtime progresses. We evaluate our approach for two different placement objectives, and observe its effectiveness even in challenging scenarios.

We introduce a novel method for oriented place recognition with 3D LiDAR scans. A Convolutional Neural Network is trained to extract compact descriptors from single 3D LiDAR scans. These can be used both to retrieve near-by place candidates from a map, and to estimate the yaw discrepancy needed for bootstrapping local registration methods. We employ a triplet loss function for training and use a hard-negative mining strategy to further increase the performance of our descriptor extractor. In an extensive evaluation on the NCLT and KITTI datasets, we demonstrate that our method outperforms related state-of-the-art approaches based on both data-driven and handcrafted data representation in challenging long-term outdoor conditions.

We propose ORBSLAM-Atlas, a system able to handle an unlimited number of disconnected sub-maps, that includes a robust map merging algorithm able to detect submaps with common regions and seamlessly fuse them. The outstanding robustness and accuracy of ORBSLAM are due to its ability to detect wide-baseline matches between keyframes, and to exploit them by means of non-linear optimization, however it only can handle a single map. ORBSLAM-Atlas brings the wide-baseline matching detection and exploitation to the multiple map arena. The result is a SLAM system significantly more general and robust, able to perform multisession mapping. If tracking is lost during exploration, instead of freezing the map, a new sub-map is launched, and it can be fused with the previous map when common parts are visited. Our criteria to declare the camera lost contrast with previous approaches that simply count the number of tracked points, we propose to discard also inaccurately estimated camera poses due to bad geometrical conditioning. As a result, the map is split into more accurate sub-maps, that are eventually merged in a more accurate global map, thanks to the multi-mapping capabilities.We provide extensive experimental validation in the EuRoC datasets, where ORBSLAM-Atlas obtains accurate monocular and stereo results in the difficult sequences where ORBSLAM failed. We also build global maps after multiple sessions in the same room, obtaining the best results to date, between 2 and 3 times more accurate than competing multi-map approaches. We also show the robustness and capability of our system to deal with dynamic scenes, quantitatively in the EuRoC datasets and qualitatively in a densely populated corridor where camera occlusions and tracking losses are frequent.

In this paper, a novel normal distribution mixture matching based model free object tracking algorithm using 2D LIDAR is proposed. Each target object is modeled as a normal distribution mixture that captures the distribution of the points scanned from the surface of the object. This novel representation enables normal distribution transform (NDT) to accurately estimate the motion of objects, even if the shape of the points differs depending on where it is observed. Our evaluation of the proposed algorithm shows good performance in practical applications. In addition, we provides an alternative way of segmentation and data association using occupancy grid map to avoid a problem that defines a distance metric between the mixture and the point cloud. As a result, the proposed algorithm works in real time in our experiments.

The modulation of step location and duration plays an important role in realizing robust bipedal walking. This paper formulates it as a nonlinear programming problem (NLP) and proposes a novel optimization approach to adjust step location and duration in real time. Based on state feedback, the Linear Inverted Pendulum dynamics is exploited to determine the optimal step parameters. Different from previous works, this work presents three main characteristics: i) the hyperbolic functions of step duration rather than the step duration itself are chosen to be optimization variables; ii) the approach can be switched from baseline two-steps-prediction optimization to one-step-prediction optimization through merely adding several equality constraints in problem formulation; iii) the approach can deal with relative step location tracking (velocity tracking) or absolute step location tracking (position tracking) via changing the reference step parameters. As a result, the first characteristic enables the NLP to be solved in a computational-efficient manner and the latter two endow the approach with versatility under different control modes. The effectiveness has been demonstrated by simulation experiments.

We extend the core operation of non-parametric belief propagation (NBP), also known as multi-scale sequential Gibbs sampling, to approximate products of kernel density estimated beliefs that reside on some manifold. The original algorithm, though multidimensional, implicitly assumes the beliefs to reside on the Euclidean \mathbb{R}^{d}\mathbb{R}^{d} space only. The proposed extension generalizes to any mixture of Riemannian manifolds, provided the primary operations—addition and subtraction— are defined. Our motivation is primarily focused on state-estimation using non-Gaussian factor graphs for multimodal simultaneous localization and mapping in robotics. The paper presents the method as well as simulation and experimental results for validation. Our implementation is publicly available and allows for expansion with user-defined manifold mixtures.

Remote sensing measurements can provide crucial information about the material properties of a planetary surface but their application is limited by their spatial resolution, typically tens of meters per pixel, when constituent materials are mixed at much finer scale. Consequently the orbital observations must be validated with in situ measurements from a spectrometer on the ground. In planetary exploration this means that a rover must visit selected locations that jointly improve a model of the environment and satisfy mobility and sampling constraints. Conventional planning methods used in this situation follow sub-optimal greedy strategies that are not scalable to large areas. We show how the problem can be effectively defined in a Markov Decision Process framework and propose a planning algorithm based on Monte Carlo Tree Search, which is efficient but devoid of these drawbacks thereby providing superior performance. We evaluate our approach using hyperspectral imagery of a well-studied geologic site in Cuprite, Nevada.