ICRA 2019 论文列表

For mobile robots to localize robustly, actively considering the perception requirement at the planning stage is essential. In this paper, we propose a novel representation for active visual localization. By formulating the Fisher information and sensor visibility carefully, we are able to summarize the localization information into a discrete grid, namely the Fisher information field. The information for arbitrary poses can then be computed from the field in constant time, without the need of costly iterating all the 3D landmarks. Experimental results on simulated and real-world data show the great potential of our method in efficient active localization and perception-aware planning. To benefit related research, we release our implementation of the information field to the public.

Accurate relative pose is one of the key components in visual odometry (VO) and simultaneous localization and mapping (SLAM). Recently, the self-supervised learning framework that jointly optimizes the relative pose and target image depth has attracted the attention of the community. Previous works rely on the photometric error generated from depths and poses between adjacent frames, which contains large systematic error under realistic scenes due to reflective surfaces and occlusions. In this paper, we bridge the gap between geometric loss and photometric loss by introducing the matching loss constrained by epipolar geometry in a self-supervised framework. Evaluated on the KITTI dataset, our method outperforms the state-of-the-art unsupervised egomotion estimation methods by a large margin. The code and data are available at https://github.com/hlzz/DeepMatchVO.

Accurate global localization is essential for planetary rovers to reach mission goals and mitigate operational risk. For initial exploration missions, it is inappropriate to deploy GPS or build other infrastructure for navigating. One way of determining global position is to use terrain relative navigation (TRN). TRN compares planetary rover-perspective images and 3D models to existing satellite orbital imagery and digital elevation models (DEMs) for absolute positioning. However, TRN is limited by the quality of orbital data and the presence and uniqueness of terrain features. This work presents a novel combination of belief space planning with terrain relative navigation. Additionally, we introduce a new method for increasing the robustness of belief space planning to noisy map data. The new algorithm provides a statistically significant reduction in localization uncertainty when tested on elevation data produced from lunar orbital imagery.

Autonomous micro aerial vehicles still struggle with fast and agile maneuvers, dynamic environments, imperfect sensing, and state estimation drift. Autonomous drone racing brings these challenges to the fore. Human pilots can fly a previously unseen track after a handful of practice runs. In contrast, state-of-the-art autonomous navigation algorithms require either a precise metric map of the environment or a large amount of training data collected in the track of interest. To bridge this gap, we propose an approach that can fly a new track in a previously unseen environment without a precise map or expensive data collection. Our approach represents the global track layout with coarse gate locations, which can be easily estimated from a single demonstration flight. At test time, a convolutional network predicts the poses of the closest gates along with their uncertainty. These predictions are incorporated by an extended Kalman filter to maintain optimal maximum-a-posteriori estimates of gate locations. This allows the framework to cope with misleading high-variance estimates that could stem from poor observability or lack of visible gates. Given the estimated gate poses, we use model predictive control to quickly and accurately navigate through the track. We conduct extensive experiments in the physical world, demonstrating agile and robust flight through complex and diverse previously-unseen race tracks. The presented approach was used to win the IROS 2018 Autonomous Drone Race Competition, outracing the second-placing team by a factor of two.

Recent human-in-the-loop (HIL) optimization studies using wearable devices have shown an improved average metabolic reduction by optimizing a small number of control parameters during short-duration walking experiments. However, the slow metabolic dynamics, high measurement noise, and experimental time constraints create challenges for increasing the number of control parameters to be optimized. Prior work applying gradient descent and Bayesian optimization to this problem have decoupled metabolic estimation and control parameter selection using fixed estimation intervals, which imposes a hard limit on the number of parameter evaluations possible in a given time budget. In this work, we take a different approach that couples estimation and parameter selection, allowing the algorithm to spend less time on refining the metabolic estimates for parameters that are unlikely to improve performance over the best observed values. Our approach uses a Kalman filter-based metabolic estimator to formulate an optimal stopping problem during the data acquisition step of standard Bayesian optimization. Performance was analyzed in numerical simulations and in pilot human subject testing with two subjects that involved optimizing six control parameters of a single-joint exosuit and four parameters of a multi-joint exosuit.

Sampling efficiency in a highly constrained environment has long been a major challenge for sampling-based planners. In this work, we propose Rapidly-exploring Random disjointed-Trees* (RRdT*), an incremental optimal multi-query planner. RRdT* uses multiple disjointed-trees to exploit local-connectivity of spaces via Markov Chain random sampling, which utilises neighbourhood information derived from previous successful and failed samples. To balance local exploitation, RRdT* actively explore unseen global spaces when local-connectivity exploitation is unsuccessful. The active trade-off between local exploitation and global exploration is formulated as a multi-armed bandit problem. We argue that the active balancing of global exploration and local exploitation is the key to improving sample efficient in sampling-based motion planners. We provide rigorous proofs of completeness and optimal convergence for this novel approach. Furthermore, we demonstrate experimentally the effectiveness of RRdT*'s locally exploring trees in granting improved visibility for planning. Consequently, RRdT* outperforms existing state-of-the-art incremental planners, especially in highly constrained environments.

This paper proposes an analysis of walking balance in terms of movements of stance and swing legs based on an inverted pendulum and a simple pendulum. Linearization, decoupling, and non-dimensionalization of a compass gait model enable to characterize the relationship of the trajectories between the stance and swing legs by only two parameters (energy ratio and phase difference). The energy ratio is defined by the ratio of the orbital energy between the pendulums. The phase difference represents the position of the stance leg in relation to the swing leg. This study considers an orbital energy conservation of a step transition and analyzes reachability of a desirable touchdown condition. If the time evolution from a current state is not reachable to the desired touchdown region, the state is labeled as a state in balance loss. By analyzing the reachability limits of the energy ratio and phase difference, we illustrate the balance loss and safe regions on the state space of the inverted pendulum, which is termed as balance map. We examined the effects of the simplification and linearization of the compass gait model by using computer simulations. Through the simulations of walking with perturbations, we confirmed that the balance map analysis could predict a future fall in an early phase for even trajectories derived by the nonlinear model.

Model-free Reinforcement Learning (RL) offers an attractive approach to learn control policies for high dimensional systems, but its relatively poor sample complexity often necessitates training in simulated environments. Even in simulation, goal-directed tasks whose natural reward function is sparse remain intractable for state-of-the-art model-free algorithms for continuous control. The bottleneck in these tasks is the prohibitive amount of exploration required to obtain a learning signal from the initial state of the system. In this work, we leverage physical priors in the form of an approximate system dynamics model to design a curriculum for a model-free policy optimization algorithm. Our Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance. BaRC is general, in that it can accelerate training of any model-free RL algorithm on a broad class of goal-directed continuous control MDPs. Its curriculum strategy is physically intuitive, easy-to-tune, and allows incorporating physical priors to accelerate training without hindering the performance, flexibility, and applicability of the model-free RL algorithm. We evaluate our approach on two representative dynamic robotic learning problems and find substantial performance improvement relative to previous curriculum generation techniques and naive exploration strategies.

In autonomous driving community, numerous benchmarks have been established to assist the tasks of 3D/2D object detection, stereo vision, semantic/instance segmentation. However, the more meaningful dynamic evolution of the surrounding objects of ego-vehicle is rarely exploited, and lacks a large-scale dataset platform. To address this, we introduce BLVD, a large-scale 5D semantics benchmark which does not concentrate on the static detection or semantic/instance segmentation tasks tackled adequately before. Instead, BLVD aims to provide a platform for the tasks of dynamic 4D (3D+temporal) tracking, 5D (4D+interactive) interactive event recognition and intention prediction. This benchmark will boost the deeper understanding of traffic scenes than ever before. We totally yield 249, 129 3D annotations, 4, 902 independent individuals for tracking with the length of overall 214, 922 points, 6, 004 valid fragments for 5D interactive event recognition, and 4, 900 individuals for 5D intention prediction. These tasks are contained in four kinds of scenarios depending on the object density (low and high) and light conditions (daytime and nighttime). The benchmark can be downloaded from our project site https://github.com/VCCIV/BLVD/.

The Elastic Inflatable Actuators (EIAs) has several advantages such as the inherent compliance due to the body comprised of a soft materials such as silicone. Among them, the soft fiber reinforced actuator is based on the principle that the expansion of enclosure and constraint of fiber pattern lead to a desired operation. While lots of researches on the actuator has been attributed to linear and bending motions, however, there are only few researches on rotary, or torsional, motions. In this paper, we propose a new actuator that causes azimuthal deformation due to restriction of anisotropically distributed fiber element along the radial direction and expansion of the hyper elastic material. Structure design of the actuator and a fabrication process of the actuator are presented. Subsequently, FEM simulation and experiment are executed to measure rotation angles of the actuators corresponding to the applied pressure.

Tissue manipulation is a frequently used fundamental subtask of any surgical procedures, and in some cases it may require the involvement of a surgeon's assistant. The complex dynamics of soft tissue as an unstructured environment is one of the main challenges in any attempt to automate the manipulation of it via a surgical robotic system. Two AI learning based model predictive control algorithms using vision strategies are proposed and studied: (1) reinforcement learning and (2) learning from demonstration. Comparison of the performance of these AI algorithms in a simulation setting indicated that the learning from demonstration algorithm can boost the learning policy by initializing the predicted dynamics with given demonstrations. Furthermore, the learning from demonstration algorithm is implemented on a Raven IV surgical robotic system and successfully demonstrated feasibility of the proposed algorithm using an experimental approach. This study is part of a profound vision in which the role of a surgeon will be redefined as a pure decision maker whereas the vast majority of the manipulation will be conducted autonomously by a surgical robotic system. A supplementary video can be found at: http://bionics.seas.ucla.edu/research/surgeryproject17.html.

Soil stabilization is a fundamental component of nearly all construction projects, ranging from commercial construction to environmental restoration projects. Previous work in autonomous construction has generally not considered these essential stabilization and anchoring tasks. In this work we present Romu, an autonomous robot capable of building continuous linear structures by using a vibratory hammer to drive interlocking sheet piles into soil. We report on hardware parameters and their effects on pile driving performance, and demonstrate autonomous operation in both controlled and natural environments. Finally, we present simulations in which a small swarm of robots build with sheet piles in example terrains, or apply an alternate spray-based stabilizing agent, and quantify the ability of each intervention to mitigate hydraulic erosion.

Autonomous robotic boats are devised to transport people and goods similar to self-driving cars. One of the attractive features specially applied in water environment is to dynamically link and join multiple boats into one unit in order to form floating infrastructure such as bridges, markets or concert stages, as well as autonomously self-detach to perform individual tasks.In this paper we present a novel latching system that enables robotic boats to create dynamic united floating infrastructure while overcoming water disturbances. The proposed latching mechanism is based on the spherical joint (ball and socket) that allows rotation and free movements in two planes at the same time. In this configuration, the latching system is capable to securely and efficiently assemble/disassemble floating structures. The vision-based robot controller guides the self-driving robotic boats to latch with high accuracy in the millimeter range. Moreover, in case the robotic boat fails to latch due to harsh weather, the autonomous latching system is capable to recompute and reposition to latch successfully. We present experimental results from latching and docking in indoor environments. Also, we present results in outdoor environments from latching a couple of robotic boats in open water with calm and turbulent currents.

Compared to open surgical techniques, laparoscopic surgical methods aim to reduce the collateral tissue damage and hence decrease the patient recovery time. However, constraints imposed by the laparoscopic surgery, i.e. the operation of surgical tools in limited spaces, turn simple surgical tasks such as suturing into time-consuming and inconsistent tasks for surgeons. In this paper, we develop an autonomous laparoscopic robotic suturing system. More specific, we expand our smart tissue anastomosis robot (STAR) by developing i) a new 3D imaging endoscope, ii) a novel actuated laparoscopic suturing tool, and iii) a suture planning strategy for the autonomous suturing. We experimentally test the accuracy and consistency of our developed system and compare it to sutures performed manually by surgeons. Our test results on suture pads indicate that STAR can reach 2.9 times better consistency in suture spacing compared to manual method and also eliminate suture repositioning and adjustments. Moreover, the consistency of suture bite sizes obtained by STAR matches with those obtained by manual suturing.

We propose a greedy and supervised learning approach for visibility-based exploration, reconstruction and surveillance. Using a level set representation, we train a convolutional neural network to determine vantage points that maximize visibility. We show that this method drastically reduces the on-line computational cost and determines a small set of vantage points that solve the problem. This enables us to efficiently produce highly-resolved and topologically accurate maps of complex 3D environments. Unlike traditional next-best-view and frontier-based strategies, the proposed method accounts for geometric priors while evaluating potential vantage points. While existing deep learning approaches focus on obstacle avoidance and local navigation, our method aims at finding near-optimal solutions to the more global exploration problem. We present realistic simulations on 2D and 3D urban environments.

In this paper, a method for online parameter estimation and automatic control of a system of rigidly attached quadcopters is introduced. First, the method performs an estimation of the physical structure attaching the quadcopters by relying solely on information from the quadcopters' Inertial Measurement Units (IMU). This information is obtained via simple and short online experiments, allowing their plug and play assembly without any human intervention. Then, given the estimated physical attachment's parameters, a stable operation of the quadcopters is achieved via an adaptive controller architecture, where the controller parameters are obtained using Reinforcement Learning. Finally, experimental results validate the proposed method, showing that a correct estimation of the physical structure is obtained allowing the autonomous flight of a pair of attached quadcopters.

Automatic targeting of plant cells to perform tasks like extraction of chloroplast is often desired in the study of plant biology. Hence, this paper proposes an improved cell segmentation method combined with a robust tracking algorithm for vision-guided micromanipulation in plant cells. The objective of this work is to develop an automatic plant cell detection and localization technique to complete the automated workflow for plant cell manipulation. The complex structural properties of plant cells make both segmentation of cells and visual tracking of the microneedle immensely challenging, unlike single animal cell applications. Thus, an improved version of watershed segmentation with adaptive thresholding is proposed to detect the plant cells without the need for staining of the cells or additional tedious preparations. To manipulate the needle to reach the identified centroid of the cells, tracking of the needle tip is required. Visual and motion information from two data sources namely, template tracking and projected manipulator trajectory are combined using score-based normalized weighted averaging to continuously track the microneedle. The selection of trackers is influenced by their complementary nature as the former and latter are individually robust against physical and visual uncertainties, respectively. Experimental results validate the effectiveness of the proposed method by detecting plant cell centroids accurately, tracking the microneedle constantly and reaching the plant cell of interest despite the presence of visual disturbances.

The recent increase in the use of aerial vehicles raises concerns about the safety and reliability of autonomous operations. There is a growing need for methods to monitor the status of these aircraft and report any faults and anomalies to the safety pilot or to the autopilot to deal with the emergency situation. In this paper, we present a real-time approach using the Recursive Least Squares method to detect anomalies in the behavior of an aircraft. The method models the relationship between correlated input-output pairs online and uses the model to detect the anomalies. The result is an easy-to-deploy anomaly detection method that does not assume a specific aircraft model and can detect many types of faults and anomalies in a wide range of autonomous aircraft. The experiments on this method show a precision of 88.23%, recall of 88.23% and 86.36% accuracy for over 22 flight tests. The other contribution is providing a new fault detection open dataset for autonomous aircraft, which contains complete data and the ground truth for 22 fixed-wing flights with eight different types of mid-flight actuator failures to help future fault detection research for aircraft.