ICRA 2019 论文列表

Precise near-ground trajectory control is difficult for multi-rotor drones, due to the complex aerodynamic effects caused by interactions between multi-rotor airflow and the environment. Conventional control methods often fail to properly account for these complex effects and fall short in accomplishing smooth landing. In this paper, we present a novel deep-learning-based robust nonlinear controller (Neural-Lander) that improves control performance of a quadrotor during landing. Our approach combines a nominal dynamics model with a Deep Neural Network (DNN) that learns high-order interactions. We apply spectral normalization (SN) to constrain the Lipschitz constant of the DNN. Leveraging this Lipschitz property, we design a nonlinear feedback linearization controller using the learned model and prove system stability with disturbance rejection. To the best of our knowledge, this is the first DNN-based nonlinear feedback controller with stability guarantees that can utilize arbitrarily large neural nets. Experimental results demonstrate that the proposed controller significantly outperforms a Baseline Nonlinear Tracking Controller in both landing and cross-table trajectory tracking cases. We also empirically show that the DNN generalizes well to unseen data outside the training domain.

End-to-end learning for autonomous navigation has received substantial attention recently as a promising method for reducing modeling error. However, its data complexity, especially around generalization to unseen environments, is high. We introduce a novel image-based autonomous navigation technique that leverages in policy structure using the Riemannian Motion Policy (RMP) framework for deep learning of vehicular control. We design a deep neural network to predict control point RMPs of the vehicle from visual images, from which the optimal control commands can be computed analytically. We show that our network trained in the Gibson environment can be used for indoor obstacle avoidance and navigation on a real RC car, and our RMP representation generalizes better to unseen environments than predicting local geometry or predicting control commands directly.

This study addresses an operation of unmanned aerial vehicles (UAVs) in a network environment where there is time-varying network delay. The network delay entails undesirable effects on the stability of the UAV control system due to delayed state feedback and outdated control input. Although several networked control algorithms have been proposed to deal with the network delay, most existing studies have assumed that the plant dynamics is known and simple, or the network delay is constant. These assumptions are improper to multirotor-type UAVs because of their nonlinearity and time-sensitive characteristics. To deal with these problems, we propose a networked control system using model predictive control (MPC) designed under the consideration of multirotor characteristics. We also apply a Gaussian process (GP) to learn an unknown nonlinear model, which increases the accuracy of path planning and state estimation. Flight experiments show that the proposed algorithm successfully compensates the network delay and Gaussian process learning improves the UAV's path tracking performance.

Subretinal injection is known to be a complicated task for ophthalmologists to perform, the main sources of difficulties are the fine anatomy of the retina, insufficient visual feedback, and high surgical precision. Image guided robot-assisted surgery is one of the promising solutions that bring significant surgical enhancement in treatment outcome and reduces the physical limitations of human surgeons. In this paper, we demonstrate a robust framework for needle detection and localization in subretinal injection using microscope-integrated Optical Coherence Tomography (MI-OCT) based on deep learning. The proposed method consists of two main steps: a) the preprocessing of OCT volumetric images; b) needle localization in the processed images. The first step is to coarsely localize the needle position based on the needle information above the retinal surface and crop the original image into a small region of interest (ROI). Afterward, the cropped small image is fed into a well trained network for detection and localization of the needle segment. The entire framework is extensively validated in ex-vivo pig eye experiments with robotic subretinal injection. The results show that the proposed method can localize the needle accurately with a confidence of 99.2%.

To enable autonomous robot navigation among unknown dynamic obstacles, a real-time adaptive motion planner (RAMP) plans the robot motion online based on sensing the environment as the robot moves with sensors mounted on the robot. However, the sensed environmental data from the robot's local view is usually incomplete due to occlusions from obstacles and limited sensing range.This paper incorporates learning about the environment into the RAMP framework by leveraging the Hilbert Maps framework to generate a probabilistic model of occupancy of the unknown dynamic environment based on past observations. Utilizing this probabilistic model enables RAMP to reason about trajectory fitness when sensing information is partial and incomplete. This allows the RAMP robot to take advantage of what it has experienced from being in the dynamic environment before to inform its subsequent executions even though the dynamic environment changes in unknown ways. The effectiveness of incorporating such learned probabilistic data into RAMP is shown in both simulation and real experiments.

This paper proposes a novel method to estimate the scale online for monocular SLAM. Unlike conventional scale estimation methods that require a metric sensor such as an IMU and apriori knowledge of its biases, this approach estimates the scale online by solely using the monocular camera and multirotor dynamics model in an extended Kalman Filter framework. Further, we discuss the observability of scale and theoretically show that the scale becomes observable when multirotor dynamics model and monocular vision are used in conjunction. We validate our proposition with extensive experimentation on the local as well as on the standard datasets and compare the same with other state of the art methods.

Autonomous driving presents one of the largest problems that the robotics and artificial intelligence communities are facing at the moment, both in terms of difficulty and potential societal impact. Self-driving vehicles (SDVs) are expected to prevent road accidents and save millions of lives while improving the livelihood and life quality of many more. However, despite large interest and a number of industry players working in the autonomous domain, there still remains more to be done in order to develop a system capable of operating at a level comparable to best human drivers. One reason for this is high uncertainty of traffic behavior and large number of situations that an SDV may encounter on the roads, making it very difficult to create a fully generalizable system. To ensure safe and efficient operations, an autonomous vehicle is required to account for this uncertainty and to anticipate a multitude of possible behaviors of traffic actors in its surrounding. We address this critical problem and present a method to predict multiple possible trajectories of actors while also estimating their probabilities. The method encodes each actor's surrounding context into a raster image, used as input by deep convolutional networks to automatically derive relevant features for the task. Following extensive offline evaluation and comparison to state-of-the-art baselines, the method was successfully tested on SDVs in closed-course tests.

We study the end-to-end steering problem using visual input data from an onboard vehicle camera. An empirical comparison between spatial, spatio-temporal and multimodal models is performed assessing each concept's performance from two points of evaluation. First, how close the model is in predicting and imitating a real-life driver's behavior, second, the smoothness of the predicted steering command. The latter is a newly proposed metric. Building on our results, we propose a new recurrent multimodal model. The suggested model has been tested on a custom dataset recorded by BMW, as well as the public dataset provided by Udacity. Results show that it outperforms previously released scores. Further, a steering correction concept from off-lane driving through the inclusion of correction frames is presented. We show that our suggestion leads to promising results empirically.

The recent success of object detection systems motivates object-based representations for robot navigation; i.e. semantic simultaneous localization and mapping (SLAM). The semantic SLAM problem can be decomposed into a discrete inference problem: determining object class labels and measurement-landmark correspondences (the data association problem), and a continuous inference problem: obtaining the set of robot poses and object locations in the environment. A solution to the semantic SLAM problem necessarily addresses this joint inference, but under ambiguous data associations this is in general a non-Gaussian inference problem, while the majority of previous work focuses on Gaussian inference. Previous solutions to data association either produce solutions between potential hypotheses or maintain multiple explicit hypotheses for each association. We propose a solution that represents hypotheses as multiple modes of an equivalent non-Gaussian sensor model. We then solve the resulting non-Gaussian inference problem using nonparametric belief propagation. We validate our approach in a simulated hallway environment under a variety of sensor noise characteristics, as well as using real data from the KITTI dataset, demonstrating improved robustness to perceptual aliasing and odometry uncertainty.

In this paper, we present a novel policy search reinforcement learning algorithm that can deal with multimodality in control policies based on Gaussian processes. Our approach employs Overlapping Mixtures of Gaussian Processes (OMGPs) for a control policy, in which all the GPs in the mixture are global and overlapped in the input space. We first extend the OMGPs by combing sparse pseudo-input GPs as OMSGPs to reduce its computational cost of learning and prediction suitable for policy search. Then, we derive a novel multimodal policy search algorithm based on variational Bayesian inference by placing the OMSGPs as the prior of the multimodal control policy. To validate the effectiveness of our algorithm, we applied it to two typical robotic tasks in simulation: 1) object grasping and 2) table-sweep tasks since they both require the multimodality in the optimal policies. Simulation results demonstrate that our algorithm can efficiently learn multimodal policies even with high dimensional observations.

This paper presents a robot control architecture comprised of tactile and vision sensors incorporated into an off-the-shelf bin picking system. The proposed architecture facilitates flexible and reliable handling of objects and materials by employing a tactile grasp validation system and in-hand object monitoring. Two industrial grippers, specifically a magnetic gripper and a flexible vacuum gripper, are equipped with a compliant custom tactile sensor array. The algorithms used are for each specific gripper and respective sensor, however are transferrable to other grippers as well. The tactile sensing augments vision-based picking approaches, thus supporting in-hand object recognition [1] that is particularly useful for hard-to-recognize objects such as objects with transparent or shiny surfaces. Algorithms for tactile-based object pose estimation in the gripper and for grasp monitoring, including grasp validation, complete the approach.

Reliable detection and reconstruction of wires is one of the hardest problems in the UAV community, with a wide ranging impact in the industry in terms of wire avoidance capabilities and powerline corridor inspection. In this work, we introduce a real-time, model-based, multi-view algorithm to reconstruct wires from a set of images with known camera poses, while exploiting their natural shape - the catenary curve. Using a model-based approach helps us deal with partial wire detections in images, which may occur due to natural occlusion and false negatives. In addition, using a parsimonious model makes our algorithm efficient as we only need to optimize for 5 model parameters, as opposed to hundreds of 3D points in bundle-adjustment approaches. Our algorithm obviates the need for pixel correspondences by computing the reprojection error via the distance transform of binarized wire segmentation images. Further, we make our algorithm robust to arbitrary initializations by introducing an on-demand, approximate extrapolation of the distance transform based objective. We demonstrate the effectiveness of our algorithm against false negatives and random initializations in simulation, and show qualitative results with real data collected from a small UAV.

We consider the problem of information collection from a polygonal environment using a multi-robot system, subject to continuous connectivity constraints. In particular, the robots, having a common radius of communication range, must remain connected throughout the exploration maximizing the information collection. The information gained through the exploration of the terrain is wirelessly transmitted to a base station. The base station performs the centralized planning of informative paths for the robots based on the information collected by them and thereafter, the robots follow these paths. This paper formulates the problem of multi-robot informative path planning under continuous connectivity constraints as an integer program leveraging the ideas of bipartite graph matching and minimal node separators. Theoretical analysis of the proposed solution proves that the informative paths will be collision-free and will be free of both livelock and deadlock. Experimental results demonstrate the low computational requirements of our algorithm for planning the informative paths, taking only about 0.75 sec. for planning a joint set of collision-free informative locations for 10 robots.

Camera viewpoint selection is an important aspect of visual grasp detection, especially in clutter where many occlusions are present. Where other approaches use a static camera position or fixed data collection routines, our Multi-View Picking (MVP) controller uses an active perception approach to choose informative viewpoints based directly on a distribution of grasp pose estimates in real time, reducing uncertainty in the grasp poses caused by clutter and occlusions. In trials of grasping 20 objects from clutter, our MVP controller achieves 80% grasp success, outperforming a single-viewpoint grasp detector by 12%. We also show that our approach is both more accurate and more efficient than approaches which consider multiple fixed viewpoints. Code is available at https://github.com/dougsm/mvp_grasp.

This paper addresses the problem of planning time-optimal trajectories for multiple cooperative agents along specified paths through a static road network. Vehicle interactions at intersections create non-trivial decisions, with complex flow-on effects for subsequent interactions. A globally optimal, minimum time trajectory is found for all vehicles using Mixed Integer Linear Programming (MILP). Computational performance is improved by minimising binary variables using iteratively applied targeted collision constraints, and efficient goal constraints. Simulation results in an open-pit mining scenario compare the proposed method against a fast heuristic method and a reactive approach based on site practices. The heuristic is found to scale better with problem size while the MILP is able to avoid local minima.

This paper introduces the Close Enough Orienteering Problem (CEOP) for planning missions with multi-rotor aerial vehicles considering their maximal velocity and acceleration limits. The addressed problem stands to select the most rewarding target locations and sequence to visit them in the given limited travel budget. The reward is collected within a non-zero range from a particular target location that allows saving the travel cost, and thus collect more rewards. Hence, we are searching for the fastest trajectories to collect the most valuable rewards such that the motion constraints are not violated, and the travel budget is satisfied. We leverage on existing trajectory parametrization based on Bézier curves recently deployed in surveillance planning using unsupervised learning, and we propose to employ the learning in a solution of the introduced multi-vehicle CEOP. Feasibility of the proposed approach is supported by empirical evaluation and experimental deployment using multi-rotor vehicles.

Template matching has been shown to accurately estimate the pose of a new object given a limited number of samples. However, pose estimation of occluded objects is still challenging. Furthermore, many robot application domains encounter texture-less objects for which depth images are more suitable than color images. In this paper, we propose a novel framework, Multi-Task Template Matching (MTTM), that finds the nearest template of a target object from a depth image while predicting segmentation masks and a pose transformation between the template and a detected object in the scene using the same feature map of the object region. The proposed feature comparison network computes segmentation masks and pose predictions by comparing feature maps of templates and cropped features of a scene. The segmentation result from this network improves the robustness of the pose estimation by excluding points that do not belong to the object. Experimental results show that MTTM outperforms baseline methods for segmentation and pose estimation of occluded objects despite using only depth images.

The space of all possible deformations of soft robotic actuators is extremely large. It is impossible to explicitly measure each internal degree of freedom, regardless of the number and types of sensors. It is, however, possible to measure a smaller subset of task-relevant deformations using only a few well-placed sensors. But for a different task, the soft actuator's deformation behavior might differ significantly. Instead of finding a new sensor placement for the new task, which would result in a separate hand for every task, we propose a method that maintains the original sensors and uses prior knowledge about each task to extend the applicability of the existing sensorized actuators to new tasks. We demonstrate our approach by the example of a PneuFlex actuator of the RBO Hand 2. When sensorizing the actuator for a single task, the sensor model does not transfer well to other tasks. Using our multi-task method, we train new sensor models that use prior knowledge about the tasks. The new models improve measurement accuracy for the new tasks without having to change the sensor hardware.

We consider the problem of reconstructing regions of interest of a scene using multiple robot arms and RGB-D sensors. This problem is motivated by a variety of applications, such as precision agriculture and infrastructure inspection. A viewpoint evaluation function is presented that exploits predicted observations and the geometry of the scene. A recently proposed non-myopic planning algorithm, Decentralised Monte Carlo tree search, is used to coordinate the actions of the robot arms. Motion planning is performed over a navigation graph that considers the high-dimensional configuration space of the robot arms. Extensive simulated experiments are carried out using real sensor data and then validated on hardware with two robot arms. Our proposed targeted information gain planner is compared to state-of-the-art baselines and outperforms them in every measured metric. The robots quickly observe and accurately detect fruit in a trellis structure, demonstrating the viability of the approach for real-world applications.