ICRA 2019 Papers

The parameter estimation of space manipulator systems on orbit is studied, whose manipulators are subject to joint flexibilities. To improve path planning and tracking capabilities, advanced control strategies that benefit from the knowledge of system parameters are required. These parameters include the system inertial parameters as well as the stiffness and damping parameters, which describe joint flexibilities. During operation some of these parameters may change or be unknown. Estimation methods based on the equations of motion are sensitive to noise, while methods based on the angular momentum conservation, while they are tolerant to noise, they cannot estimate the parameters that describe joint flexibilities. A parameter estimation method, based on the energy balance, applied during the motion of a space flexible-joint manipulator system in the free-floating mode, is developed. The method is tolerant to noise and can reconstruct the system full dynamics. It is shown that the parameters estimated by the proposed method can describe the system dynamics fully. The application of the developed method is valid for spatial systems; it is illustrated by a planar 7 degrees of freedom (DoF) example system.

Pedestrian detection is one of the most explored topics in computer vision and robotics. The use of deep learning methods allowed the development of new and highly competitive algorithms. Deep Reinforcement Learning has proved to be within the state-of-the-art in terms of both detection in perspective cameras and robotics applications. However, for detection in omnidirectional cameras, the literature is still scarce, mostly because of their high levels of distortion. This paper presents a novel and efficient technique for robust pedestrian detection in omnidirectional images. The proposed method uses deep Reinforcement Learning that takes advantage of the distortion in the image. By considering the 3D bounding boxes and their distorted projections into the image, our method is able to provide the pedestrian’s position in the world, in contrast to the image positions provided by most state-of-the-art methods for perspective cameras. Our method avoids the need of pre-processing steps to remove the distortion, which is computationally expensive. Beyond the novel solution, our method compares favorably with the state-of-the-art methodologies that do not consider the underlying distortion for the detection task.

Navigating surgical tools in the dynamic and tortuous anatomy of the lung's airways requires accurate, real-time localization of the tools with respect to the preoperative scan of the anatomy. Such localization can inform human operators or enable closed-loop control by autonomous agents, which would require accuracy not yet reported in the literature. In this paper, we introduce a deep learning architecture, called OffsetNet, to accurately localize a bronchoscope in the lung in real-time. After training on only 30 minutes of recorded camera images in conserved regions of a lung phantom, OffsetNet tracks the bronchoscope's motion on a held-out recording through these same regions at an update rate of 47 Hz and an average position error of 1.4 mm. Because this model performs poorly in less conserved regions, we augment the training dataset with simulated images from these regions. To bridge the gap between camera and simulated domains, we implement domain randomization and a generative adversarial network (GAN). After training on simulated images, OffsetNet tracks the bronchoscope's motion in less conserved regions at an average position error of 2.4 mm, which meets conservative thresholds required for successful tracking.

This paper aims to develop an optimal controller that can automatically provide personalized control of robotic knee prosthesis in order to best support gait of individual prosthesis wearers. We introduced a new reinforcement learning (RL) controller for this purpose based on the promising ability of RL controllers to solve optimal control problems through interactions with the environment without requiring an explicit system model. However, collecting data from a human-prosthesis system is expensive and thus the design of a RL controller has to take into account data and time efficiency. We therefore propose an offline policy iteration based reinforcement learning approach. Our solution is built on the finite state machine (FSM) impedance control framework, which is the most used prosthesis control method in commercial and prototypic robotic prosthesis. Under such a framework, we designed an approximate policy iteration algorithm to devise impedance parameter update rules for 12 prosthesis control parameters in order to meet individual users' needs. The goal of the reinforcement learning-based control was to reproduce near-normal knee kinematics during gait. We tested the RL controller obtained from offline learning in real time experiment involving the same able-bodied human subject wearing a robotic lower limb prosthesis. Our results showed that the RL control resulted in good convergent behavior in kinematic states, and the offline learning control policy successfully adjusted the prosthesis control parameters to produce near-normal knee kinematics in 10 updates of the impedance control parameters.

Target search with unmanned aerial vehicles (UAVs) is relevant problem to many scenarios, e.g., search and rescue (SaR). However, a key challenge is planning paths for maximal search efficiency given flight time constraints. To address this, we propose the Obstacle-aware Adaptive Informative Path Planning (OA-IPP) algorithm for target search in cluttered environments using UAVs. Our approach leverages a layered planning strategy using a Gaussian Process (GP)based model of target occupancy to generate informative paths in continuous 3D space. Within this framework, we introduce an adaptive replanning scheme which allows us to trade off between information gain, field coverage, sensor performance, and collision avoidance for efficient target detection. Extensive simulations show that our OA-IPP method performs better than state-of-the-art planners, and we demonstrate its application in a realistic urban SaR scenario.

Handing over objects is the foundation of many human-robot interaction and collaboration tasks. In the scenario where a human is handing over an object to a robot, the human chooses where the object needs to be transferred. The robot needs to accurately predict this point of transfer to reach out proactively, instead of waiting for the final position to be presented. This work presents an efficient method for predicting the Object Transfer Point (OTP), which synthesizes (1) an offline OTP calculated based on human preferences observed in a human-robot motion study with (2) a dynamic OTP predicted based on the observed human motion. Our proposed OTP predictor is implemented on a humanoid nursing robot and experimentally validated in human-robot handover tasks. Compared to only using static or dynamic OTP estimators, it has better accuracy at the earlier phase of handover (up to 45% of the handover motion) and can render fluent handovers with a reach-to-grasp response time (about 3.1 secs) close to natural human receiver's response. In addition, the OTP prediction accuracy is maintained across the robot's visible workspace by utilizing a user-adaptive reference frame.

In this paper, we focus on the robot grasping problem with parallel grippers using image data. For this task, we propose and implement an end-to-end approach. In order to detect the good grasping poses for a parallel gripper from RGB images, we have employed transfer learning for a Convolutional Neural Network (CNN) based object detection architecture. Our obtained results show that, the adapted network either outperforms or is on-par with the state-of-the art methods on a benchmark dataset. We also performed grasping experiments on a real robot platform to evaluate our method's real world performance.

For autonomous driving applications it is critical to know which type of road users and road side infrastructure are present to plan driving manoeuvres accordingly. Therefore autonomous cars are equipped with different sensor modalities to robustly perceive its environment. However, for classification modules based on machine learning techniques it is challenging to overcome unseen sensor noise. This work presents an object classification module operating on unsupervised learned multi-modal features with the ability to overcome gradual or total sensor failure. A two stage approach composed of an unsupervised feature training and a uni-modal and multimodal classifiers training is presented. We propose a simple but effective decision module switching between uni-modal and multi-modal classifiers based on the closeness in the feature space to the training data. Evaluations on the ModelNet 40 data set show that the proposed approach has a 14% accuracy gain compared to a late fusion approach operating on a noisy point cloud data and a 6% accuracy gain when operating on noisy image data.

This paper presents a novel approach for local 3D environment representation for autonomous unmanned ground vehicle (UGV) navigation called On Visible Point Clouds Mesh (OVPC Mesh). Our approach represents the surrounding of the robot as a watertight 3D mesh generated from local point cloud data in order to represent the free space surrounding the robot. It is a conservative estimation of the free space and provides a desirable trade-off between representation precision and computational efficiency, without having to discretize the environment into a fixed grid size. Our experiments analyze the usability of the approach for UGV navigation in rough terrain, both in simulation and in a fully integrated real-world system. Additionally, we compare our approach to well-known state-of the-art solutions, such as Octomap and Elevation Mapping and show that OVPC Mesh can provide reliable 3D information for trajectory planning while fulfilling real-time constraints.

Path tracking controllers for an autonomous vehicle are often designed by using either a dynamic model or a kinematic one and some models are related to wheel-ground contact, that makes the efficiency of the controller highly dependent on the ground parameters estimation, especially for off-road mobile robots intended to navigate in open environments. This paper proposes a new nonlinear observer designed to estimate the front and rear contact cornering stiffnesses in real time, that are related both on tire and soil proprieties. The latter is estimated using steering angles as well as yaw rate and lateral velocity, which are provided by a preliminary Kalman-Bucy observer. The performance of the proposed nonlinear observer combined with the LQR controller is evaluated by both advanced simulations and experiments in real conditions at different speeds.

Soft robots are challenging to model due in large part to the nonlinear properties of soft materials. Fortunately, this softness makes it possible to safely observe their behavior under random control inputs, making them amenable to large-scale data collection and system identification. This paper implements and evaluates a system identification method based on Koopman operator theory in which models of nonlinear dynamical systems are constructed via linear regression of observed data by exploiting the fact that every nonlinear system has a linear representation in the infinite-dimensional space of real-valued functions called observables. The approach does not suffer from some of the shortcomings of other nonlinear system identification methods, which typically require the manual tuning of training parameters and have limited convergence guarantees. A dynamic model of a pneumatic soft robot arm is constructed via this method, and used to predict the behavior of the real system. The total normalized-root-mean-square error (NRMSE) of its predictions is lower than that of several other identified models including a neural network, NLARX, nonlinear Hammerstein-Wiener, and linear state space model.

Compliant fish-like robots are being developed as efficient and dependable underwater observation platforms with low impact on the observed environment. Orientation control is an essential building block to achieve autonomy for those vehicles. So far, the major focus has been on rigid tails or on flexible tails with a high degree of actuation. We present a novel control strategy for an underactuated robotic fish with a flexible tail optimized for cruising. The basis for our approach is the generation of asymmetric velocity profiles of the robot's tail beats. To achieve such velocity profiles, the usual sinusoidal tail actuation is replaced with skewed triangle waves. We provide a simple formulation for such waves, where their skew is dependent on only one variable which we define as skew factor. Furthermore, a nonlinear control law is derived to achieve the desired turning motions. We implement the controller on a compliant fish-like robot with a simple actuation mechanism. The control scheme is experimentally validated, and its robustness is tested in field trials.

Unstructured environments impose several challenges when robots are required to perform different tasks and adapt to unseen situations. In this context, a relevant problem arises: how can robots learn to perform various tasks and adapt to different conditions? A potential solution is to endow robots with learning capabilities. In this line, imitation learning emerges as an intuitive way to teach robots different motor skills. This learning approach typically mimics human demonstrations by extracting invariant motion patterns and subsequently applies these patterns to new situations. In this paper, we propose a novel kernel treatment of imitation learning, which endows the robot with imitative and adaptive capabilities. In particular, due to the kernel treatment, the proposed approach is capable of learning human skills associated with high-dimensional inputs. Furthermore, we study a new concept of correlation-adaptive imitation learning, which allows for the adaptation of correlations exhibited in high-dimensional demonstrated skills. Several toy examples and a collaborative task with a real robot are provided to verify the effectiveness of our approach.

We propose an ultra-wideband-based (UWB) localization system that achieves high accuracy through nonparametric estimation of measurement probability densities and explicit modeling of antenna delays. This problem is difficult because non-line-of-sight conditions give rise to multimodal errors, which make linear estimation methods ineffective. The primary contribution in this paper is an approach for both characterizing these errors in situ and an optimization framework that recovers both positions and antenna delays. We evaluate our system with a network of 8 nodes based on the DecaWave DWM1000 and achieve accuracies from 3 cm RMSE in line-of-sight conditions to 30 cm RMSE in non-line-of-sight conditions. Collecting measurements and localizing the network in this manner requires less than a minute, after which the realized network may be used for dynamic real-time tracking.

Efficient exploration of the search space is crucial for faster convergence in sampling-based motion planning. An effective sampling method must first concentrate on quickly finding a good initial solution and then focus the search on regions that can potentially improve the current best solution. In this paper, we propose a non-parametric exploration technique that addresses these challenges. The proposed algorithm prioritizes search by utilizing heuristics. After an initial solution is found, the method generates samples in the “L_{2} -L_{2} -informed set”, while leveraging collision data to reduce the number of samples in the obstacle space. We demonstrate the efficiency of the proposed approach with several benchmarking experiments.

Synthetic Aperture Sonar (SAS) is a technique to improve the spatial resolution from a moving set of receivers by extending the array in time, increasing the effective array length and aperture. This technique is limited by the accuracy of the receiver position estimates, necessitating highly accurate, typically expensive aided-inertial navigation systems for submerged platforms. We leverage simultaneous localization and mapping to fuse acoustic and navigational measurements and obtain accurate pose estimates even without the benefit of absolute positioning for lengthy underwater missions. We demonstrate a method of formulating the well-known SAS problem in a SLAM framework, using acoustic data from hydrophones to simultaneously estimate platform and beacon position. An empirical probability distribution is computed from a conventional beamformer to correctly account for uncertainty in the acoustic measurements. The non-parametric method relieves the familiar Gaussian-only assumption currently used in the localization and mapping discipline and fits effectively into a factor graph formulation with conventional factors such as ground-truth priors and odometry. We present results from field experiments performed on the Charles River with an autonomous surface vehicle which demonstrate simultaneous localization of an unknown acoustic beacon and vehicle positioning, and provide comparison to GPS ground truths.

To match the ever increasing standards of fresh products, and the need to reduce waste, we devise an alternative to the destructive and highly variable fruit ripeness estimation by a penetrometer. We propose a fully automatic method to assess the ripeness of mango which is non-destructive, allows the user to test multiple surface areas with a single touch and is capable of dissociating between ripe and non-ripe fruits. A custom-made gripper equipped with a capacitive tactile sensor array is used to palpate the fruit. The ripeness is estimated as mango stiffness extracted through a simplified spring model. We test the framework on a set of 25 mangoes of the Keitt variety, and compare the results to penetrometer measurements. We show it is possible to correctly classify 88% of the mango without removing the skin of the fruit. The method can be a valuable substitute for non-destructive fruit ripeness testing. To the authors knowledge, this is the first robotics ripeness estimation system based on capacitive tactile sensing technology.

A hybrid manufacturing process combining abrasive jet and laser micromaching enables the creation of living hinges in nitinol that retain the superelastic properties of the bulk material. The former selectively etches through the thickness of a workpiece and the latter defines the part's final geometry. Because the majority of the material removal is done with the room-temperature mechanical etching procedure, thermal damage to the part is minimized. Processing parameters to achieve desired geometries are described, a bending stiffness model for the living hinges is provided, and validation experiments are presented. Lastly, to demonstrate the usefulness of these components to millimeter-sized robotic systems and medical devices, we show their integration in two prototype devices: an endoscopic camera wrist and a simple laser beam steering system.

Visual localization is a key step in many robotics pipelines, allowing the robot to (approximately) determine its position and orientation in the world. An efficient and scalable approach to visual localization is to use image retrieval techniques. These approaches identify the image most similar to a query photo in a database of geo-tagged images and approximate the query's pose via the pose of the retrieved database image. However, image retrieval across drastically different illumination conditions, e.g. day and night, is still a problem with unsatisfactory results, even in this age of powerful neural models. This is due to a lack of a suitably diverse dataset with true correspondences to perform end-to-end learning. A recent class of neural models allows for realistic translation of images among visual domains with relatively little training data and, most importantly, without ground-truth pairings.In this paper, we explore the task of accurately localizing images captured from two traversals of the same area in both day and night. We propose ToDayGAN - a modified image-translation model to alter nighttime driving images to a more useful daytime representation. We then compare the daytime and translated night images to obtain a pose estimate for the night image using the known 6-DOF position of the closest day image. Our approach improves localization performance by over 250% compared the current state-of-the-art, in the context of standard metrics in multiple categories.