IROS 2021 论文列表

This paper presents a self-supervised Learning from Learned Hallucination (LfLH) method to learn fast and reactive motion planners for ground and aerial robots to navigate through highly constrained environments. The recent Learning from Hallucination (LfH) paradigm for autonomous navigation executes motion plans by random exploration in completely safe obstacle-free spaces, uses hand-crafted hallucination techniques to add imaginary obstacles to the robot’s perception, and then learns motion planners to navigate in realistic, highly-constrained, dangerous spaces. However, current hand-crafted hallucination techniques need to be tailored for specific robot types (e.g., a differential drive ground vehicle), and use approximations heavily dependent on certain assumptions (e.g., a short planning horizon). In this work, instead of manually designing hallucination functions, LfLH learns to hallucinate obstacle configurations, where the motion plans from random exploration in open space are optimal, in a self-supervised manner. LfLH is robust to different robot types and does not make assumptions about the planning horizon. Evaluated in both simulated and physical environments with a ground and an aerial robot, LfLH outperforms or performs comparably to previous hallucination approaches, along with sampling- and optimization-based classical methods.

Aerobatic quadrotors have been a very active field of research for the last two decades. Their huge community boosted the development of computational light-weight planning and control algorithms. In contrast and despite recent progress, research on agile micro autonomous underwater vehicles (µAUV) is still in its infancy. Both vehicle classes share a close relationship. They achieve high speeds of multiple bodylengths per second. At the same time they are subject to limited onboard resources such as sensors and computing power.In this work, we explore and exploit the potential synergies between aerobatic drones and hydrobatic µAUVs. In order to demonstrate the possible transfer of concepts we build on a state-of-the-art quadrotor trajectory planning framework and extend it to incorporate hydrodynamic effects. Furthermore, we study in a series of experiments the performance of the transferred concepts and show that various quadrotor simplifications match well for hydrobatic µAUVs.

In this paper, we enable automated property verification of deliberative components in robot control architectures. We focus on formalizing the execution context of Behavior Trees (BTs) to provide a scalable, yet formally grounded, methodology to enable runtime verification and prevent unexpected robot behaviors. To this end, we consider a message-passing model that accommodates both synchronous and asynchronous composition of parallel components, in which BTs and other components execute and interact according to the communication patterns commonly adopted in robotic software architectures. We introduce a formal property specification language to encode requirements and build runtime monitors. We performed a set of experiments, both on simulations and on the real robot, demonstrating the feasibility of our approach in a realistic application and its integration in a typical robot software architecture. We also provide an OS-level virtualization environment to reproduce the experiments in the simulated scenario.

In this paper, we are interested in formalizing human trajectories in human-robot encounters. We consider a particular case where a human and a robot walk towards each other. A question that arises is whether, when, and how humans will deviate from their trajectory to avoid a collision. These human trajectories can then be used to generate socially acceptable robot trajectories. To model these trajectories, we propose a data-driven algorithm to extract a formal specification expressed in Signal Temporal Logic with probabilistic predicates. We evaluated our method on trajectories collected through an online study where participants had to avoid colliding with a robot in a shared environment. Further, we demonstrate that probabilistic STL is a suitable formalism to depict human behavior, choices and preferences in specific scenarios of social navigation.

This paper studies stabilizer design for position-controlled humanoid robots. Stabilizers are an essential part for position-controlled humanoids, whose primary objective is to adjust the control input sent to the robot to assist the tracking controller to better follow the planned reference trajectory. To achieve this goal, this paper develops a novel force-feedback based whole-body stabilizer that fully exploits the six-dimensional force measurement information and the whole-body dynamics to improve tracking performance. Relying on rigorous analysis of whole-body dynamics of position-controlled humanoids under unknown contact, the developed stabilizer leverages quadratic-programming based technique that allows cooperative consideration of both the center-of-mass tracking and contact force tracking. The effectiveness of the proposed stabilizer is demonstrated on the UBTECH Walker robot in the MuJoCo simulator. Simulation validations show a significant improvement in various scenarios as compared to commonly adopted stabilizers based on the zero-moment-point feedback and the linear inverted pendulum model.

Formation control of multi-robot systems has been largely studied due to its wide application domain. Several methods in the literature rely on explicit communication among the robots, which in realistic scenarios may lead to reduced performance or even instability due to delays and packet loss or corruption. Nonetheless, multi-robot coordination based solely on implicit communication has been proposed in cooperative manipulation problems. Taking inspiration from this, we propose a method to solve the formation control problem for a group of ground robots not relying on direct communication among them. Instead, the robots are physically constrained to a common object through elastic cables in order to exploit forces as a means of indirect communication. After deriving the dynamic equations, the control and planning approaches are explained, and the stability of the controlled system is discussed using Lyapunov’s stability theory. Numerical simulations are presented to support the method.

In robotic surgeries employing the master-slave operation scheme, various haptic devices have been adopted as master manipulators. The main challenge of the haptic device is to contribute to a sensitive reaction to external forces for operators. Since force perception on fingertips is impaired by unstable entire hand condition, stable hand condition helps to improve force perception. This study proposed a master manipulator that enables a force feedback at the gripping fingertips on a stable hand condition with a hand rest function. In this framework, the hand rest function was realized by switching the impedance value in admittance control for the translation section of the master manipulator and the displaying force was generated based on the pivot point, which depends on only the movement of the translation section. With master-slave operation, an 8-drawing experiment was performed under multiple force feedback and hand conditions after evaluating the effectiveness of a hand rest on sensitivity for external force in a preliminary experiment. The results showed that the hand rest function contributed to effective force feedback.

Robots can grasp, even manipulate, objects with different shape, weight and size thanks to the their end-effectors. These are mostly constituted by two fingers, and are known as grippers. However, despite being quite simple for human beings, manipulation is not so straightforward to carry out on robotic systems. One of the main obstacles is the lack of reliable control methods: this is especially true for pneumatic grippers. Such devices are often mounted on industrial robots, though their behavior does not go beyond basic fully-open or fully-closed operations. This happens also as a consequence of the incapability of taking into account frictional effects limiting the pneumatic gripper performance. In this article, a new control strategy is delivered to solve this issue. The proposed strategy allows controlling the grasping force of a pneumatic gripper, without performance degradation due to friction. A pneumatic gripper was built and instrumented with several sensors to experimentally validate the proposed control strategy. The gripper was mechanically connected to a robotic arm and tested with different desired force profiles upon a wide range of force.

Developing the flocking behavior for a dynamic squad of fixed-wing UAVs is still a challenge due to kinematic complexity and environmental uncertainty. In this paper, we deal with the decentralized flocking and collision avoidance problem through deep reinforcement learning (DRL). Specifically, we formulate a decentralized DRL-based decision making framework from the perspective of every follower, where a collision avoidance mechanism is integrated into the flocking controller. Then, we propose a novel reinforcement learning algorithm PS-CACER for training a shared control policy for all the followers. Besides, we design a plug-n-play embedding module based on convolutional neural networks and the attention mechanism. As a result, the variable-length system state can be encoded into a fixed-length embedding vector, which makes the learned DRL policy independent with the number and the order of followers. Finally, numerical simulation results demonstrate the effectiveness of the proposed method, and the learned policies can be directly transferred to semi-physical simulation without any parameter finetuning.

The future motion of traffic participants is inherently uncertain. To plan safely, therefore, an autonomous agent must take into account multiple possible trajectory outcomes and prioritize them. Recently, this problem has been addressed with generative neural networks. However, most generative models either do not learn the true underlying trajectory distribution reliably, or do not allow predictions to be associated with likelihoods. In our work, we model motion prediction directly as a density estimation problem with a normalizing flow between a noise distribution and the future motion distribution. Our model, named FloMo, allows likelihoods to be computed in a single network pass and can be trained directly with maximum likelihood estimation. Furthermore, we propose a method to stabilize training flows on trajectory datasets and a new data augmentation transformation that improves the performance and generalization of our model. Our method achieves state-of-the-art performance on three popular prediction datasets, with a significant gap to most competing models.

Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot to traverse off-road is still an open question. This research proposes a contrastive learning based method. With a set of human-annotated anchor patches, a feature representation is learned to discriminate regions with different traversability, a method of fine-grained semantic segmentation and mapping is subsequently developed for off-road scene understanding. Experiments are conducted on a dataset of three driving segments that represent very diverse off-road scenes. An anchor accuracy of 89.8% is achieved by evaluating the matching with human-annotated image patches in cross-scene validation. Examined by associated 3D LiDAR data, the fine-grained segments of visual images are demonstrated to have different levels of toughness and terrain elevation, which represents their semantical meaningfulness. The resultant maps contain both fine-grained labels and confidence values, providing rich information to support a robot traversing complex off-road scenes.

Flying Modular Structures offer a versatile mechanism that can change the arrangement of constituent actuators according to task requirements. In this work, we extend a modular aerial platform that can expand its actuation capabilities depending on the configuration. Each module is composed of a quadrotor in a cage that can rigidly connect with other modules. The quadrotor is connected with the cage by a revolute joint that allows it to rotate with respect to the cage. Modules located in the structure are either parallel or perpendicular to one another. The task specification defines forces and moments needed during the execution. We propose two search methods to find a configuration that can satisfy the specification. The first approach consists of an exhaustive search that yields optimal structure configurations by exploring the whole search space. The second approach proposes a heuristic based on subgroup search, reducing the problem complexity from exponential to linear. We validate our proposed algorithms with several simulations. Our results show that the proposed heuristic is computationally efficient and finds a near-optimal configuration even for flying modular structures composed of a large number of modules.

2D-to-3D correspondence estimation is the key step of 3D model-based image localization, and most of the existing research in this field focuses on improving the feature matching performance. Even with the best feature matching method, there are still some outliers, and thus, almost all the methods simply apply the RANSAC algorithm to select the inliers and estimate the camera pose afterwards. However, the reliability of RANSAC depends considerably on the inlier ratio. Once the inlier ratio decreases, for example a challenging scenario occurs, it will be unable to select the inliers well and lead to a worse camera pose. In this study, we attempted to build a neural network to learn the geometric relationship between 2D images and the 3D model to select the correct correspondence from the initial 2D-to-3D matching results to improve the performance of camera localization. Because the number of inputs, i.e., the number of 2D-to-3D correspondences, is unknown and different for each image, we propose a PointNet-based Geometric Consistency Network (GCC-Net) for the correct correspondence estimation and an LSTM-based Hypothesis Rating Network (HR-Net) to enhance GCC-Net with the camera localization loss. Experimental results showed that the proposed method outperforms RANSAC considerably on the camera pose estimation, particularly when the inlier ratio of the initial correspondence was low.

Validating the safety of autonomous systems generally requires the use of high-fidelity simulators that adequately capture the variability of real-world scenarios. However, it is generally not feasible to exhaustively search the space of simulation scenarios for failures. Adaptive stress testing (AST) is a method that uses reinforcement learning to find the most likely failure of a system. AST with a deep reinforcement learning solver has been shown to be effective in finding failures across a range of different systems. This approach generally involves running many simulations, which can be very expensive when using a high-fidelity simulator. To improve efficiency, we present a method that first finds failures in a low-fidelity simulator. It then uses the backward algorithm, which trains a deep neural network policy using a single expert demonstration, to adapt the low-fidelity failures to high-fidelity. We have created a series of autonomous vehicle validation case studies that represent some of the ways low-fidelity and high-fidelity simulators can differ, such as time discretization. We demonstrate in a variety of case studies that this new AST approach is able to find failures with significantly fewer high-fidelity simulation steps than are needed when just running AST directly in high-fidelity. As a proof of concept, we also demonstrate AST on NVIDIA’s DriveSim simulator, an industry state-of-the-art high-fidelity simulator for finding failures in autonomous vehicles.

Autonomous robotic weeding in grasslands requires robust weed segmentation. Deep learning models can provide solutions to this problem, but they need to be trained on large amounts of images, which in the case of grasslands are notoriously difficult to obtain and manually annotate. In this work we introduce Few-leaf Learning, a concept that facilitates the training of accurate weed segmentation models and can lead to easier generation of weed segmentation datasets with minimal human annotation effort. Our approach builds upon the fact that each plant species within the same field has relatively uniform visual characteristics due to similar environmental influences. Thus, we can train a field-and-day-specific weed segmentation model on synthetic training data stemming from just a handful of annotated weed leaves. We demonstrate the efficacy of our approach for different fields and for two common grassland weeds: Rumex obtusifolius (broad-leaved dock) and Cirsium vulgare (spear thistle). Our code is publicly available at https://github.com/RGring/WeedAnnotator.

Bundle adjustment refines scene geometry and relative camera poses simultaneously via reprojection error, computed by a set of images from different viewpoints, which is the gold standard for visual odometry. However, deep learning methods have not been well exploited within this area of study. This paper introduces a self-supervised learning framework for monocular visual odometry, inside which depth maps, relative camera poses, and dense feature maps (with the same resolution as images) are estimated and used for photometric, geometric, and feature-metric losses in bundle adjustment. In this manner, we consider that the learning of neural networks can be geometrically constrained by multi-view geometry. Furthermore, bundle adjustment is only required during the training time, allowing the networks to benefit from bundle adjustment without any additional computation burden during the inference time. To stabilize the training process, we apply a two-stage strategy that yields promising results. Finally, we carefully select the neural network architectures to ensure efficiency, and experimental results demonstrate the success of our proposed approach in terms of visual odometry accuracy and high speed.

Cricothyrotomy is a life-saving emergency intervention that secures an alternate airway route after a neck injury or obstruction. The procedure starts with identifying the correct location (the cricothyroid membrane) for creating an incision to insert an endotracheal tube. This location is determined using a combination of visual and palpation cues. Enabling robot-assisted remote cricothyrotomy may extend this life-saving procedure to injured soldiers or patients who may not be readily accessible for on-site intervention during search-and-rescue scenarios. As a first step towards achieving this goal, this paper explores the feasibility of palpation-assisted landmark identification for cricothyrotomy. Using a cricothyrotomy training simulator, we explore several alternatives for in-situ remote localization of the cricothyroid membrane. These alternatives include a) unaided telemanipulation, b) telemanipulation with direct force feedback, c) telemanipulation with superimposed motion excitation for on-line stiffness estimation and display, and d) fully autonomous palpation scan initialized based on the user’s understanding of key anatomical landmarks. Using the manually digitized cricothyroid membrane location as ground truth, we compare these four methods for accuracy and repeatability of identifying the landmark for cricothyrotomy, time of completion, and ease of use. These preliminary results suggest that the accuracy of remote cricothyrotomy landmark identification is improved when the user is aided with visual and force cues. They also show that, with proper user initialization, landmark identification using remote palpation is feasible - therefore satisfying a key prerequisite for future robotic solutions for remote cricothyrotomy.

Grasping objects in cluttered scenarios is a challenging task in robotics. Performing pre-grasp actions such as pushing and shifting to scatter objects is a way to reduce clutter. Based on deep reinforcement learning, we propose a Fast-Learning Grasping (FLG) framework, that can integrate pre-grasping actions along with grasping to pick up objects from cluttered scenarios with reduced real-world training time. We associate rewards for performing moving actions with the change of environmental clutter and utilize a hybrid triggering method, leading to data-efficient learning and synergy. Then we use the output of an extended fully convolutional network as the value function of each pixel point of the workspace and establish an accurate estimation of the grasp probability for each action. We also introduce a mask function as prior knowledge to enable the agents to focus on the accurate pose adjustment to improve the effectiveness of collecting training data and, hence, to learn efficiently. We carry out pre-training of the FLG over simulated environment, and then the learnt model is transferred to the real world with minimal fine-tuning for further learning during actions. Experimental results demonstrate a 94% grasp success rate and the ability to generalize to novel objects. Compared to state-of-the-art approaches in the literature, the proposed FLG framework can achieve similar or higher grasp success rate with lesser amount of training in the real world. Supplementary video is available at https://youtu.be/KTGj1fGU6ho.

To improve the efficiency of medical visualization for computer aided surgery, we propose a fast and unsupervised 3D-CNN based non-local feature learning network. The proposed network consists of an encoder structure and a decoder structure. The encoder of the network projects the cube into a high-dimensional feature space, and the decoder of the network reconstructs the cube from the feature space. The decoder of the network serves as a dictionary shared by the cube to enforce the features for similar parts to be similar although they may distribute at disjointed locations. With such structures, the network is able to extract non-local features of the entire data. Moreover, a sparse constraint is incorporated into the network to increase the discriminative of the non-local features. Then the extracted non-local features of each voxel are fused with the corresponding position matrix and Hessian matrix for the voxel classification using Random Forest. Finally, a multidimensional transfer function is designed to enable the volume rendering. Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods with much less training time.