ICRA 2019 论文列表

Predicting the motion of a driver’s vehicle is crucial for advanced driving systems, enabling detection of potential risks towards shared control between the driver and automation systems. In this paper, we propose a variational neural network approach that predicts future driver trajectory distributions for the vehicle based on multiple sensors.Our predictor generates both a conditional variational distribution of future trajectories, as well as a confidence estimate for different time horizons. Our approach allows us to handle inherently uncertain situations, and reason about information gain from each input, as well as combine our model with additional predictors, creating a mixture of experts.We show how to augment the variational predictor with a physics-based predictor, and based on their confidence estimations, improve overall system performance. The resulting combined model is aware of the uncertainty associated with its predictions, which can help the vehicle autonomy to make decisions with more confidence. The model is validated on real-world urban driving data collected in multiple locations. This validation demonstrates that our approach improves the prediction error of a physics-based model by 25% while successfully identifying the uncertain cases with 82% accuracy.

Estimating statistical uncertainties allows autonomous agents to communicate their confidence during task execution and is important for applications in safety-critical domains such as autonomous driving. In this work, we present the uncertainty-aware imitation learning (UAIL) algorithm for improving end-to-end control systems via data aggregation. UAIL applies Monte Carlo Dropout to estimate uncertainty in the control output of end-to-end systems, using states where it is uncertain to selectively acquire new training data. In contrast to prior data aggregation algorithms that force human experts to visit sub-optimal states at random, UAIL can anticipate its own mistakes and switch control to the expert in order to prevent visiting a series of sub-optimal states. Our experimental results from simulated driving tasks demonstrate that our proposed uncertainty estimation method can be leveraged to reliably predict infractions. Our analysis shows that UAIL outperforms existing data aggregation algorithms on a series of benchmark tasks.

Combining multiple sensors for advanced perception is a crucial requirement for autonomous vehicle navigation. Heterogeneous sensors are used to obtain rich information about the surrounding environment. The combination of the camera and lidar sensors enables precise range information that can be projected onto the visual image data. This gives a high level understanding of the scene which can be used to enable context based algorithms such as collision avoidance and navigation. The main challenge when combining these sensors is aligning the data into a common domain. This can be difficult due to the errors in the intrinsic calibration of the camera, extrinsic calibration between the camera and the lidar and errors resulting from the motion of the platform. In this paper, we examine the algorithms required to provide motion correction for scanning lidar sensors. The error resulting from the projection of the lidar measurements into a consistent odometry frame is not possible to remove entirely, and as such it is essential to incorporate the uncertainty of this projection when combining the two different sensor frames. This work proposes a novel framework for the prediction of the uncertainty of lidar measurements (in 3D) projected in to the image frame (in 2D) for moving platforms. The proposed approach fuses the uncertainty of the motion correction with uncertainty resulting from errors in the extrinsic and intrinsic calibration. By incorporating the main components of the projection error, the uncertainty of the estimation process is better represented. Experimental results for our motion correction algorithm and the proposed extended uncertainty model are demonstrated using real-world data collected on an electric vehicle equipped with wide-angle cameras covering a 180-degree field of view and a 16-beam scanning lidar.

Diversity of environments is a key challenge that causes learned robotic controllers to fail due to the discrepancies between the training and evaluation conditions. Training from demonstrations in various conditions can mitigate - but not completely prevent - such failures. Learned controllers such as neural networks typically do not have a notion of uncertainty that allows to diagnose an offset between training and testing conditions, and potentially intervene. In this work, we propose to use Bayesian Neural Networks, which have such a notion of uncertainty. We show that uncertainty can be leveraged to consistently detect situations in high-dimensional simulated and real robotic domains in which the performance of the learned controller would be sub-par. Also, we show that such an uncertainty based solution allows making an informed decision about when to invoke a fallback strategy. One fallback strategy is to request more data. We empirically show that providing data only when requested results in increased data-efficiency.

Stereo cameras are widely used for sensing and navigation of underwater robotic systems. They can provide high resolution color views of a scene; the constrained camera geometry enables metrically accurate depth estimation; they are also relatively cost-effective. Traditional stereo vision algorithms rely on feature detection and matching to enable triangulation of points for estimating disparity. However, for underwater applications, the effects of underwater light propagation lead to image degradation, reducing image quality and contrast. This makes it especially challenging to detect and match features, especially from varying viewpoints. Recently, deep learning has shown success in end-to-end learning of dense disparity maps from stereo images. Still, many state-of-the-art methods are supervised and require ground truth depth or disparity, which is challenging to gather in subsea environments. Simultaneously, deep learning has also been applied to the problem of underwater image restoration. Again, it is difficult or impossible to gather real ground truth data for this problem. In this work, we present an unsupervised deep neural network (DNN) that takes input raw color underwater stereo imagery and outputs dense depth maps and color corrected imagery of underwater scenes. We leverage a model of the process of underwater image formation, image processing techniques, as well as the geometric constraints inherent to the stereo vision problem to develop a modular network that outperforms existing methods.

We equip an ultra-wideband (UWB) node and a 2D LiDAR sensor a.k.a. 2D laser rangefinder on a mobile robot, and place UWB beacon nodes at unknown locations in an unknown environment. All UWB nodes can do ranging with each other thus forming a cooperative sensor network. We propose to fuse the peer-to-peer ranges measured between UWB nodes and laser scanning information, i.e., range measured between robot and nearby objects/obstacles, for simultaneous localization of the robot, all UWB beacons and LiDAR mapping. The fusion is inspired by two facts: 1) LiDAR may improve UWB-only localization accuracy as it gives a more precise and comprehensive picture of the surrounding environment; 2) on the other hand, UWB ranging measurements may remove the error accumulated in the LiDAR-based SLAM algorithm. Our experiments demonstrate that UWB/LiDAR fusion enables drift-free SLAM in real-time based on ranging measurements only.

This paper proposes a novel cooperative system for an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) which utilizes the UAV not only as a flying sensor but also as a tether attachment device. Two robots are connected with a tether, allowing the UAV to anchor the tether to a structure located at the top of a steep terrain, impossible to reach for UGVs. Thus, enhancing the poor traversability of the UGV by not only providing a wider range of scanning and mapping from the air, but also by allowing the UGV to climb steep terrains with the winding of the tether. In addition, we present an autonomous framework for the collaborative navigation and tether attachment in an unknown environment. The UAV employs visual inertial navigation with 3D voxel mapping and obstacle avoidance planning. The UGV makes use of the voxel map and generates an elevation map to execute path planning based on a traversability analysis. Furthermore, we compared the pros and cons of possible methods for the tether anchoring from multiple points of view. To increase the probability of successful anchoring, we evaluated the anchoring strategy with an experiment. Finally, the feasibility and capability of our proposed system were demonstrated by an autonomous mission experiment in the field with an obstacle and a cliff.

We propose an image-based cross-view geolocalization method that estimates the global pose of a UAV with the aid of georeferenced satellite imagery. Our method consists of two Siamese neural networks that extract relevant features despite large differences in viewpoints. The input to our method is an aerial UAV image and nearby satellite images, and the output is the weighted global pose estimate of the UAV camera. We also present a framework to integrate our crossview geolocalization output with visual odometry through a Kalman filter. We build a dataset of simulated UAV images and satellite imagery to train and test our networks. We show that our method performs better than previous camera pose estimation methods, and we demonstrate our networks ability to generalize well to test datasets with unseen images. Finally, we show that integrating our method with visual odometry significantly reduces trajectory estimation errors.

Collaborative robots are becoming more common on factory floors as well as regular environments, however, their safety still is not a fully solved issue. Collision detection does not always perform as expected and collision avoidance is still an active research area. Collision avoidance works well for fixed robot-camera setups, however, if they are shifted around, Eye-to-Hand calibration becomes invalid making it difficult to accurately run many of the existing collision avoidance algorithms. We approach the problem by presenting a stand-alone system capable of detecting the robot and estimating its position, including individual joints, by using a simple 2D colour image as an input, where no Eye-to-Hand calibration is needed. As an extension of previous work, a two-stage transfer learning approach is used to re-train a multi-objective convolutional neural network (CNN) to allow it to be used with heterogeneous robot arms. Our method is capable of detecting the robot in real-time and new robot types can be added by having significantly smaller training datasets compared to the requirements of a fully trained network. We present data collection approach, the structure of the multi-objective CNN, the two-stage transfer learning training and test results by using real robots from Universal Robots, Kuka, and Franka Emika. Eventually, we analyse possible application areas of our method together with the possible improvements.

We study the problem of computing shortest collision-free Dubins paths when turning a corner. We present a sufficient condition for a closed-form solution. Specifically, consider S as the set consisting of paths of the form RSRSR, RSRSL, LSRSR and LSRSL that pass through the interior corner, where sub-paths RSR, RSL, and LSR are elementary Dubins paths composed of segments which are either straight (S) or turning left (L) or right (R). We find the closed-form optimal path around a corner when S is nonempty. Our solution can be used in an efficient path planner, for example, when navigating corridors. It can also be used as a subroutine for planners such as RRTs.

Multirobot coverage is the problem of planning paths for several identical robots such that the combined regions traced out by the robots completely cover their environment. We consider the problem of multirobot coverage with the objective of minimizing the mission time, which depends on the number of turns taken by the robots. To solve this problem, we first partition the environment into ranks which are long thin rectangles the width of the robot's coverage tool. Our novel partitioning heuristic produces a set of ranks which minimizes the number of turns. Next, we solve a variant of the multiple travelling salesperson problem (m-TSP) on the set of ranks to minimize the robots' mission time. The resulting coverage plan is guaranteed to cover the entire environment. We present coverage plans for a robotic vacuum using real maps of 25 indoor environments and compare the solutions to paths planned without the objective of minimizing turns. Turn minimization reduced the number of turns by 6.7% and coverage time by 3.8% on average for teams of 1-5 robots.

Guaranteeing safe constraint satisfaction in nonlinear control systems with uncertainty remains a major challenge for control. The most successful control method handling constraints under uncertainty has without doubt been model predictive control (MPC). In particular, recent sampling-based MPC methods have shown success in controlling stochastic systems with complex, nonlinear dynamics. The sampling-based schemes are appealing since they do not need strong assumptions on the underlying model, except that it can be forward simulated. At the same time, the lack of major assumptions on the models make the statement of safety or robustness guarantees difficult. However, the samples drawn during the control process inherently contain probabilistic information about these properties. In this paper, we formally describe the problem that results by adding chance constraints to a sampling-based MPC scheme. Furthermore, based on a variant of the Chernoff bound, we derive trust regions, in which the sampling based estimation of the safety constraint satisfies a specified quality. Finally, we present a case study in the navigation domain to demonstrate the applicability of the proposed approach.

We introduce and study the problem of planning a trajectory for an agent to carry out a reconnaissance mission while avoiding being detected by an adversarial guard. This introduces a multi-objective version of classical visibility-based target search and pursuit-evasion problem. In our formulation, the agent receives a positive reward for increasing its visibility (by exploring new regions) and a negative penalty every time it is detected by the guard. The objective is to find a finite-horizon path for the agent that balances the trade off between maximizing visibility and minimizing detectability.We model this problem as a discrete, sequential, two-player, zero-sum game. We use two types of game tree search algorithms to solve this problem: minimax search tree and Monte-Carlo search tree. Both search trees can yield the optimal policy but may require possibly exponential computational time and space. We propose several pruning techniques to reduce the computational cost while still preserving optimality guarantees. Simulation results show that the proposed strategy prunes approximately three orders of magnitude nodes as compared to the brute-force strategy. We also find that the Monte-Carlo search tree saves approximately one order of computational time as compared to the minimax search tree.

We present a new approach to transfer grasp configurations from prior example objects to novel objects. We assume the novel and example objects have the same topology and similar shapes. We perform 3D segmentation on these objects using geometric and semantic shape characteristics. We compute a grasp space for each part of the example object using active learning. We build bijective contact mapping between these model parts and compute the corresponding grasps for novel objects. Finally, we assemble the individual parts and use local replanning to adjust grasp configurations while maintaining its stability and physical constraints. Our approach is general, can handle all kind of objects represented using mesh or point cloud and a variety of robotic hands.

In this paper, we present a novel approach for surgical task segmentation. A segmentation policy learns the correlations between features and segmentation points from manually labeled data. The most correlated features and rules for segmenting them are identified and learned. These form a complete set of segmentation policy. The proposed approach is developed to segment new but similar tasks through transfer learning. It is verified through applying the segmentation rule learned from the labeled data to segment other tasks. The performance of the proposed algorithm was evaluated by comparing the results against the ground truths. Experimental results demonstrate that our approach can achieve high segmentation rates with an accuracy of between 68.8% - 81.8%.

In this paper, we propose a trajectory-based reinforcement learning method named deep latent policy gradient (DLPG) for learning locomotion skills. We define the policy function as a probability distribution over trajectories and train the policy using a deep latent variable model to achieve sample efficient skill learning. We first evaluate the sample efficiency of DLPG compared to the state-of-the-art reinforcement learning methods in simulated environments. Then, we apply the proposed method to a four-legged walking robot named Snapbot to learn three basic locomotion skills of turn left, go straight, and turn right. We demonstrate that, by properly designing two reward functions for curriculum learning, Snapbot successfully learns the desired locomotion skills with moderate sample complexity.

Trajectory planning for a tractor-trailer vehicle is challenging because the vehicle kinematics consists of underactuated and nonholonomic constraints that are highly coupled. Prevalent sampling-based or search-based planners suitable for rigid-body vehicles are not capable of handling the tractor-trailer vehicle cases. This work aims to deal with generic n-trailer cases in the tiny environments. To this end, an optimal control problem is formulated, which is beneficial in being accurate, straightforward, and unified. An adaptively homotopic warm-starting approach is proposed to facilitate the numerical solution process of the formulated optimal control problem. Compared with the existing sequential warm starting strategies, our proposal can adaptively define the subproblems with the purpose of making the gaps between adjacent subproblems “pleasant” for the solver. Unification and efficiency of the proposed adaptively homotopic warm-starting approach have been investigated in several extremely tiny scenarios. Our planner finds solutions that other existing planners cannot. Online planning opportunities are briefly discussed as well.

Autonomous driving has harsh requirements of small model size and energy efficiency, in order to enable the embedded system to achieve real-time on-board object detection. Recent deep convolutional neural network based object detectors have achieved state-of-the-art accuracy. However, such models are trained with numerous parameters and their high computational costs and large storage prohibit the deployment to memory and computation resource limited systems. Low-precision neural networks are popular techniques for reducing the computation requirements and memory footprint. Among them, binary weight neural networks (BWNs) are the extreme case which quantizes the float-point into just 1 bit. BWNs are difficult to train and suffer from accuracy deprecation due to the extreme low-bit representation. To address this problem, we propose a knowledge transfer (KT) method to aid the training of BWN using a full-precision teacher network. We built DarkNet- and MobileNet-based binary weight YOLOv2 detectors and conduct experiments on KITTI benchmark for car, pedestrian and cyclist detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the model size of DarkNet-YOLO from 257 MB to 8.8 MB and MobileNet-YOLO from 193 MB to 7.9 MB.

Autonomous vehicles are undergoing a rapid development thanks to advances in perception, planning and control methods and technologies achieved in the last two decades. Moreover, the lowering costs of sensors and computing platforms are attracting industrial entities, empowering the integration and development of innovative solutions for civilian use. Still, the development of autonomous racing cars has been confined mainly to laboratory studies and small to middle scale vehicles. This paper tackles the development of a planning and control framework for an electric full scale autonomous racing car, which is an absolute novelty in the literature, upon which we report our preliminary experiments and perspectives on future work. Our system leverages real time Nonlinear Model Predictive Control to track a pre-planned racing line. We describe the whole control system architecture including the mapping and localization methods employed.