ICRA 2021 论文列表

Right invariant extended Kalman filter (RIEKF) based simultaneous localization and mapping (SLAM) proposed recently has shown to be able to produce more consistent SLAM estimates as compared with traditional EKF based SLAM methods, including some improved EKF SLAM methods such as observability constrained-EKF (OC-EKF) SLAM. Latest results have demonstrated that its performance is very close to optimization based SLAM algorithms such as iSAM. In this paper, we propose to use RIEKF SLAM algorithm in active SLAM where both the predicted SLAM results for choosing control actions and the actual estimated SLAM results applying the selected control actions are computed using RIEKF algorithms. The advantages over traditional EKF based active SLAM are the more accurate and consistent predicted uncertainty estimates which result in robustness of the active SLAM algorithm. The advantages over optimization based active SLAM is the reduced computational cost. Simulation results are presented to validate the advantages of the proposed algorithm3.

End-to-end visuomotor control is emerging as a compelling solution for robot manipulation tasks. However, imitation learning-based visuomotor control approaches tend to suffer from a common limitation, lacking the ability to recover from an out-of-distribution state caused by compounding errors. In this paper, instead of using tactile feedback or explicitly detecting the failure through vision, we investigate using the uncertainty of a policy neural network. We propose a novel uncertainty-based approach to detect and recover from failure cases. Our hypothesis is that policy uncertainties can implicitly indicate the potential failures in the visuomotor control task and that robot states with minimum uncertainty are more likely to lead to task success. To recover from high uncertainty cases, the robot monitors its uncertainty along a trajectory and explores possible actions in the state-action space to bring itself to a more certain state. Our experiments verify this hypothesis and show a significant improvement on task success rate: 12% in pushing, 15% in pick-and-reach and 22% in pick-and-place.

A key challenge towards autonomous multi-part object assembly is robust sensorimotor control under uncertainty. In contrast to previous works that rely on a priori knowledge on whether two parts match, we aim to learn this through physical interaction. We propose a hierarchical approach that enables a robot to autonomously assemble parts while being uncertain about part types and positions. In particular, our probabilistic approach learns a set of differentiable filters that leverage the tactile sensorimotor trace from failed assembly attempts to update its belief about part position and type. This enables a robot to overcome assembly failure. We demonstrate the effectiveness of our approach on a set of object fitting tasks. The experimental results show that the proposed approach achieves higher precision in object position and type estimation, and accomplishes object fitting tasks faster than baselines.

We propose an integrated prediction and planning system for autonomous driving which uses rational inverse planning to recognise the goals of other vehicles. Goal recognition informs a Monte Carlo Tree Search (MCTS) algorithm to plan optimal maneuvers for the ego vehicle. Inverse planning and MCTS utilise a shared set of defined maneuvers and macro actions to construct plans which are explainable by means of rationality principles. Evaluation in simulations of urban driving scenarios demonstrate the system’s ability to robustly recognise the goals of other vehicles, enabling our vehicle to exploit non-trivial opportunities to significantly reduce driving times. In each scenario, we extract intuitive explanations for the predictions which justify the system’s decisions.

Deep Neural Networks (NNs) have been widely utilized in contact-rich manipulation tasks to model the complicated contact dynamics. However, NN-based models are often difficult to decipher which can lead to seemingly inexplicable behaviors and unidentifiable failure cases. In this work, we address the interpretability of NN-based models by introducing the kinodynamic images. We propose a methodology that creates images from kinematic and dynamic data of contact-rich manipulation tasks. By using images as the state representation, we enable the application of interpretability modules that were previously limited to vision-based tasks. We use this representation to train a Convolutional Neural Network (CNN) and we extract interpretations with Grad-CAM to produce visual explanations. Our method is versatile and can be applied to any classification problem in manipulation tasks to visually interpret which parts of the input drive the model’s decisions and distinguish its failure modes, regardless of the features used. Our experiments demonstrate that our method enables detailed visual inspections of sequences in a task, and high-level evaluations of a model’s behavior. Code for this work is available at [1].

Assisting surgeons with automation of surgical subtasks is challenging due to backlash, hysteresis, and variable tensioning in cable-driven robots. These issues are exacerbated as surgical instruments are changed during an operation. In this work, we propose a framework for automation of high- precision surgical subtasks by learning local, sample-efficient, accurate, closed-loop policies that use visual feedback instead of robot encoder estimates. This framework, which we call deep Intermittent Visual Servoing (IVS), switches to a learned visual servo policy for high-precision segments of repetitive surgical tasks while relying on a coarse open-loop policy for the segments where precision is not necessary. We train the policy using only 180 human demonstrations that are roughly 2 seconds each. Results on a da Vinci Research Kit suggest that combining the coarse policy with half a second of corrections from the learned policy during each high-precision segment improves the success rate on the Fundamentals of Laparoscopic Surgery peg transfer task from 72.9% to 99.2%, 31.3% to 99.2%, and 47.2% to 100.0% for 3 instruments with differing cable properties. In the contexts we studied, IVS attains the highest published success rates for automated surgical peg transfer and is significantly more reliable than previous techniques when instruments are changed. Supplementary material is available at https://tinyurl.com/ivs-icra.

The "Thinking, Fast and Slow" paradigm of Kahneman proposes that we use two different styles of thinking—a fast and intuitive System 1 for certain tasks, along with a slower but more analytical System 2 for others. While the idea of using this two-system style of thinking is gaining popularity in AI and robotics, our work considers how to interleave the two styles of decision-making, i.e., how System 1 and System 2 should be used together. For this, we propose a novel and general framework which includes a new System 0 to oversee Systems 1 and 2. At every point when a decision needs to be made, System 0 evaluates the situation and quickly hands over the decision-making process to either System 1 or System 2. We evaluate such a framework on a modified version of the classic Pac-Man game, with an already-trained RL algorithm for System 1, a Monte-Carlo tree search for System 2, and several different possible strategies for System 0. As expected, arbitrary switches between Systems 1 and 2 do not work, but certain strategies do well. With System 0, an agent is able to perform better than one that uses only System 1 or System 2.

Autonomous urban driving among human-driven cars requires a holistic understanding of road rules, driver intents and driving styles. This is challenging as a short-term, single instance, driver intent of lane change may not correspond to their driving styles for a longer duration. This paper presents an interactive behavior planner which accounts for road context, short-term driver intent, and long-term driving style to infer beliefs over the latent states of surrounding vehicles. We use a specialized Partially Observable Markov Decision Process to provide risk-averse decisions. Specifically, we consider adversarial driving scenarios caused by irrational drivers to validate the robustness of our proposed interactive behavior planner in simulation as well as on a full-size self-driving car. Our experimental results show that our algorithm enables safer and more travel time-efficient autonomous driving compared to baselines even in adversarial scenarios.

Many human-robot collaboration scenarios can be seen as a redundant leader-follower setup where the human (i.e., the leader) can potentially perform the task without the assistance of the robot (i.e., the follower). Thus, the goal of the collaboration, beside stable execution of the task, is to reduce the human cost; e.g., ergonomic, or cognitive cost. Such system redundancies (where the same task be achieved in different manner) can also be exploited as a communication channel for the human to convey his/her intention to the robot; since it is essential for the overall performance (both execution and assistance) that the follower recognizes the intended task in an online fashion. Having an estimation for the intended task, the robot can assist the human by reducing the human cost over the task null-space; i.e., the null-space which arises from the overall system redundancies with respect to the intended task. With the prospective of supernumerary and prosthetic robots, in this work, we primarily focus on serial manipulation in which the proximal/distal part of the kinematic chain is controlled by the leader/follower respectively. By exploiting kinematic redundancies for intention-recognition and cost-minimization, our proposed control strategy (for the follower) ensures assistance under stable execution of the task. Our results (simulations and preliminary experimentation) show the efficacy of our method in providing a seamless robotic assistance (i.e., improving human posture) toward human intended tasks (i.e., reaching motions) for wearable robotics.

Safety integration components for robotic applications are a mandatory feature for any autonomous mobile application, including human avoidance behaviors. This paper proposes a novel parametrizable scene risk evaluator for open-field applications that use humans motion predictions and pre-defined hazard zones to estimate a braking factor. Parameters optimization uses simulated data. The evaluation is carried out by simulated and real-time scenarios, showing the impact of human predictions in favor of risk reductions on agricultural applications.

In this work, we aim to achieve efficient end-to-end learning of driving policies in dynamic multi-agent environments. Predicting and anticipating future events at the object level are critical for making informed driving decisions. We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view, conditioned on the selected action sequence of the ego-vehicle. To decide the action at each step, we seek the action sequence that can lead to safe future states based on the prediction module outputs by repeatedly sampling likely action sequences. We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level. Our method establishes a new state of the art in the challenging CARLA multi-agent driving simulation environments without expert demonstration, giving better explainability and sample efficiency.

This paper presents a transformable RHex-inspired robot, RHex-T3, with high energy efficiency, excellent flexibility and versatility. By using the innovative 2-DoF transformable structure, RHex-T3 inherits most of RHex’s mobility, and can also switch to other 4 modes for handling various missions. The wheel-mode improves the efficiency of RHex-T3, and the leg-mode helps to generate a smooth locomotion when RHex-T3 is overcoming obstacles. In addition, RHex-T3 can switch to the claw-mode for transportation missions, and even climb ladders by using the hook-mode. The simulation model is conducted based on the mechanical structure, and thus the properties in different modes are verified and analyzed through numerical simulations.

General Visual Inspection is a manual inspection process regularly used to detect and localise obvious damage on the exterior of commercial aircraft. There has been increasing demand to perform this process at the boarding gate to minimize the downtime of the aircraft and automating this process is desired to reduce the reliance on human labour. This automation typically requires the first step of estimating a camera’s pose with respect to the aircraft for initialisation. However, localisation methods often require infrastructure, which can be very challenging when performed in uncontrolled outdoor environments and within the limited turnover time (approximately 2 hours) on an airport tarmac. In addition, access to commercial aircraft can be very restricted, causing development and testing of solutions to be a challenge. Hence, this paper proposes an on-site infrastructure-less initialisation method, by using the same pan-tilt-zoom camera used for the inspection task to estimate its own pose. This is achieved using a Deep Convolutional Neural Network trained with only synthetic images to regress the camera’s pose. We apply domain randomisation when generating our dataset for training our network and improve prediction accuracy by introducing a new component to an existing loss function that leverages on known aircraft geometry to relate position and orientation. Experiments are conducted and we have successfully regressed camera poses with a median error of 0.22 m and 0.73°.

This paper presents a comprehensive inertial aided 3D LiDAR SLAM system with hybrid geometric primitives in large-scale environments, including a tightly-coupled LiDAR-Inertial-Odometry (LIO), a global mapping module supported by learning-based loop closure detection and a sub-maps matching algorithm. An efficient method is developed to simultaneously extract explicit plane features and point features from each raw point cloud. To make full use of the structural information of the surroundings, plane features and point features (ground and edge) are tracked across a fix-sized group of LiDAR keyframes in the local map. For effective loop closure detection in large-scale environments, we integrate the learning-based point cloud network and a keyframe sequence matching method to detect loops. Finally, a novel, deterministic and near real-time plane-driven sub-maps matching algorithm is proposed to close the loops. The proposed SLAM system is validated with experiments on different types of environments.

Due to manufacturing tolerances, the geometry parameters of workpieces are not constant in industrial welding applications. Today, this problem is addressed by either accepting fluctuating part quality or by measuring the geometry and adjusting the configuration of the robot and process controller for each individual part. However, measuring the geometry requires additional manufacturing time or reduces the accessibility of the welding tool. This paper presents an approach for estimating the workpiece geometry based on process data during the welding process, which eliminates the need of measuring the geometry in advance, yet provides the same accessibility of the welding tool. For this approach, a data set is generated by conducting welding experiments with varying part geometries. Welding current, welding voltage and acoustic emissions are recorded. Three supervised learning techniques, which are Support Vector Machines, Random Forest Decision Trees and k-Nearest-Neighbors, are used to train models for estimating the geometry. The results show, that the trained models successfully estimate the part geometry with over 90% accuracy.

Hand-eye calibration of laser profile sensors is the process of extracting the homogeneous transformation between the laser profile sensor frame and the end-effector frame of a robot in order to express the data extracted by the sensor in the robot’s global coordinate system. For laser profile scanners this is a challenging procedure, as they provide data only in two dimensions and state-of-the-art calibration procedures require the use of specialised calibration targets. This paper presents a novel method to extract the translation-part of the hand-eye calibration matrix with rotation-part known a priori in a target-agnostic way. Our methodology is applicable to any 2D image or 3D object as a calibration target and can also be performed in situ in the final application. The method is experimentally validated on a real robot-sensor setup with 2D and 3D targets.

We present a system which improves the safety and accuracy of impedance controlled robotic manipulators with proximity perception. Proximity servoed manipulators, which use proximity sensors attached to the robot’s outer shell, have recently demonstrated robust collision avoidance abilities. Nevertheless, unwanted collisions cannot be avoided entirely. As a fallback safety mechanism, robots with joint force/torque sensing rely on impedance controllers for impact attenuation and compliant behavior. However, impedance controllers induce undesired deflections of the robot from its trajectory when it is not in contact. These deviations are more pronounced at soft configurations and when the robot grasps objects of unknown weight distribution, thus a compromise must be made between high positional accuracy and softness (safety). The proximity information allows the robot to react to anticipated impacts proactively for attenuation and damage reduction of unavoidable collisions, while still maintaining high accuracy during regular operation. This is achieved through variations of impedance parameters according to proximity measurements and motions towards safe joint configurations during the preimpact phase.

Ranging systems can provide inexpensive, accurate, energy- and computationally-efficient navigation solutions for mobile robots. This work focuses on location and pose estimation in ranging networks composed of anchors with known positions as well as mobile robots modeled as rigid bodies, each carrying multiple tags to localize. Noisy distance measurements can be obtained between a subset of the nodes (anchors and tags), and the robots can move in order to improve the accuracy of the localization process, which depends on the geometry of the network. We propose a method to find trajectories for the robots leading to configurations that locally optimize this localization accuracy. These trajectories minimize a cost function based on the constrained Cramér-Rao Lower Bound (CRLB), where the constraints capture the information about the known distances between tags carried by the same robot. A primal-dual optimization scheme aims to enforce these distance constraints between tags in the motion planner as well. An important feature of the approach is that the gradient terms necessary to plan the motion can be computed essentially in closed form, thereby simplifying the implementation. We compare the proposed method to a naive two-stage algorithm that optimizes the positions and orientations of the robots independently. Simulation results illustrate the benefits of using the constrained optimization approach.

Autonomous ground vehicles have improved greatly over the past decades, but they still have their limitations when it comes to off-road environments. There is still a need for planning techniques that effectively handle physical interactions between a vehicle and its surroundings. We present a method of modifying a standard path planning algorithm to address these problems by incorporating a learned model to account for complexities that would be too hard to address manually. The model predicts how well a vehicle will be able to follow a potential plan in a given environment. These predictions are then used to assign costs to their associated paths, where the path predicted to be the most feasible will be output as the final path. This results in a planner that doesn't rely solely on engineered features to evaluate traversability of obstacles, and can also choose a better path based on an understanding of its own capability that it has learned from previous interactions. This modification was integrated into the Hybrid A* algorithm and experimental results demonstrated an improvement of 14.29% over the original version on a physical platform.