ICRA 2020 论文列表

Incorporating touch as a sensing modality for robots can enable finer and more robust manipulation skills. Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. In this paper, we introduce OmniTact, a multi-directional high-resolution tactile sensor. OmniTact is designed to be used as a fingertip for robotic manipulation with robotic hands, and uses multiple micro-cameras to detect multi-directional deformations of a gel-based skin. This provides a rich signal from which a variety of different contact state variables can be inferred using modern image processing and computer vision methods. We evaluate the capabilities of OmniTact on a challenging robotic control task that requires inserting an electrical connector into an outlet, as well as a state estimation problem that is representative of those typically encountered in dexterous robotic manipulation, where the goal is to infer the angle of contact of a curved finger pressing against an object. Both tasks are performed using only touch sensing and deep convolutional neural networks to process images from the sensor's cameras. We compare with a state-of-the-art tactile sensor that is only sensitive on one side, as well as a state-of-the-art multi-directional tactile sensor, and find that OmniTact's combination of high-resolution and multi-directional sensing is crucial for reliably inserting the electrical connector and allows for higher accuracy in the state estimation task. Videos and supplementary material can be found here4.

In this paper, we present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras, which has a 360° coverage of stereo observations of the environment. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation, which are faster and more accurate than the existing networks. Second, we integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency. Using the estimated depth map, we reproject keypoints onto each other view, which leads to a better and more efficient feature matching process. Finally, we fuse the omnidirectional depth maps and the estimated rig poses into the truncated signed distance function (TSDF) volume to acquire a 3D map. We evaluate our method on synthetic datasets with ground-truth and real-world sequences of challenging environments, and the extensive experiments show that the proposed system generates excellent reconstruction results in both synthetic and real-world environments.

A mobility mechanism for robots to be used in tight spaces shared with people requires it to have a small footprint, to move omnidirectionally, as well as to be highly maneuverable. However, currently there exist few such mobility mechanisms that satisfy all these conditions well. Here we introduce Omnidirectional Balancing Unicycle Robot (OmBURo), a novel unicycle robot with active omnidirectional wheel. The effect is that the unicycle robot can drive in both longitudinal and lateral directions simultaneously. Thus, it can dynamically balance itself based on the principle of dual-axis wheeled inverted pendulum. This paper discloses the early development of this novel unicycle robot involving the overall design, modeling, and control, as well as presents some preliminary results including station keeping and path following. With its very compact structure and agile mobility, it might be the ideal locomotion mechanism for robots to be used in human environments in the future.

This paper proposes an offline and runtime combined framework for the autonomous motion of snake robots. With the dynamic feedback of its state during runtime, the robot utilizes the linear regression to update its control parameters for better performance and thus adaptively reacts to the environment. To reduce interference from infeasible samples and improve efficiency, the data set for runtime training is chosen from one in several clusters categorized from samples collected in offline practice. Moreover, only the most sensitive control parameter is updated at one iteration for better robustness and efficiency. The effectiveness and efficiency of our approach are evaluated by a set of case studies of pole climbing. Experimental results demonstrate that with the proposed framework, the snake robot can adapt its locomotion gait to poles with different unknown diameters.

UAVs require reliable, cost-efficient onboard flight state estimation that achieves high accuracy and robustness to perturbation. We analyze a multi-sensor extended Kalman filter (EKF) based on the work by Leutenegger. The EKF uses measurements from a MEMS-based inertial system, static and dynamic pressure sensors as well as GPS. As opposed to other implementations we do not use a magnetic sensor because the weak magnetic field of the earth is subject to disturbances. Observability of the state is a necessary condition for the EKF to work. In this paper, we demonstrate that the system state is observable - which is in contrast to statements in the literature - if the random nature of the air mass is taken into account. Therefore, we carry out an in-depth observability analysis based on a singular value decomposition (SVD). The numerical SVD delivers a wealth of information regarding the observable (sub)spaces. We validated the theoretical findings based on sensor data recorded in test flights on a glider. Most importantly, we demonstrate that the EKF works. It is capable of absorbing large perturbations in the wind state variable converging to the undisturbed estimates.

Generating a natural question can enable autonomous robots to propose problems according to their surroundings. However, recent studies on question generation rarely consider semantic graph mapping, which is widely used to understand environments. In this paper, we introduce a method to generate natural questions using object-oriented semantic graphs. First, a graph convolutional network extracts features from the graph. Then, a recurrent neural network generates the natural question from the extracted features. Using graphs, we can generate natural questions for both single and sequential scenes. The proposed method outperforms conventional methods on a publicly available dataset for single scenes and can generate questions for sequential scenes.

Safe autonomous driving requires reliable 3D object detection-determining the 6 DoF pose and dimensions of objects of interest. Using stereo cameras to solve this task is a cost-effective alternative to the widely used LiDAR sensor. The current state-of-the-art for stereo 3D object detection takes the existing PSMNet stereo matching network, with no modifications, and converts the estimated disparities into a 3D point cloud, and feeds this point cloud into a LiDAR-based 3D object detector. The issue with existing stereo matching networks is that they are designed for disparity estimation, not 3D object detection; the shape and accuracy of object point clouds are not the focus. Stereo matching networks commonly suffer from inaccurate depth estimates at object boundaries, which we define as streaking, because background and foreground points are jointly estimated. Existing networks also penalize disparity instead of the estimated position of object point clouds in their loss functions. We propose a novel 2D box association and object-centric stereo matching method that only estimates the disparities of the objects of interest to address these two issues. Our method achieves state-of-the-art results on the KITTI 3D and BEV benchmarks.

Object finding in clutter is a skill that requires perception of the environment and in many cases physical interaction. In robotics, interactive perception defines a set of algorithms that leverage actions to improve the perception of the environment, and vice versa use perception to guide the next action. Scene interactions are difficult to model, therefore, most of the current systems use predefined heuristics. This limits their ability to efficiently search for the target object in a complex environment. In order to remove heuristics and the need for explicit models of the interactions, in this work we propose a reinforcement learning based active and interactive perception system for scene exploration and object search. We evaluate our work both in simulated and in real-world experiments using a robotic manipulator equipped with an RGB and a depth camera, and compare our system to two baselines. The results indicate that our approach, trained in simulation only, transfers smoothly to reality and can solve the object finding task efficiently and with more than 88% success rate.

We propose a new capacitive proximity sensor that detects deformations of a suction cup as a tactile sense. We confirmed that one sensor module provides three applications for reliable picking and a simplified setup. The first application is the picking height decision. The second one is the placing height decision for detecting whether the grasped object is placed on the placement surface. These two applications are achieved by detecting the push-in stroke of the suction cup. The final application is detection of whether the suction cup is in partial contact or full contact with the object. This function can correct the picking posture as well as detect whether picking is possible before the pull-up motion. We also demonstrate that the partial contact position can be estimated in real time.

Agile fixed-wing aircraft integrate the efficient, high-speed capabilities of conventional fixed-wing platforms with the extreme maneuverability of rotorcraft. This work presents a nonlinear control strategy that harnesses these capabilities to enable autonomous flight through aggressive, time-constrained, three-dimensional trajectories. The cascaded control structure consists of two parts; an inner attitude control loop developed on the Special Orthornormal group that avoids singularities commonly associated with other parametrizations, and an outer position control loop that jointly determines the thrust command and attitude references by implementing a novel vector-projection algorithm. The objective of the algorithm is to decouple roll from the reference attitude to ensure that thrust and lift forces can always be pointed such that position errors converge to zero. The proposed control system represents a single, unified solution that remains effective throughout the aircraft's flight envelope, including aerobatic operation. Controller performance is verified through simulations and experimental flight tests; results show the unified control scheme is capable of performing a wide range of operations that would normally require multiple, single-purpose controllers, and their associated switching logic.

This paper presents a synchronization controller for mobile sensors that are minimally actuated and can only communicate with each other over a very short range. This work is motivated by ocean monitoring applications where large-scale sensor networks consisting of drifters with minimal actuation capabilities, i.e., active drifters, are employed. We assume drifters are tasked to monitor regions consisting of gyre flows where their trajectories are periodic. As drifters in neighboring regions move into each other's proximity, it presents an opportunity for data exchange and synchronization to ensure future rendezvous. We present a nonlinear synchronization control strategy to ensure that drifters will periodically rendezvous and maximize the time they are in their rendezvous regions. Numerical simulations and small-scale experiments validate the efficacy of the control strategy and hint at extensions to large-scale mobile sensor networks.

Safe and precise reference tracking is a crucial characteristic of Micro Aerial Vehicles (MAVs) that have to operate under the influence of external disturbances in cluttered environments. In this paper, we present a Nonlinear Model Predictive Control (NMPC) that exploits the fully physics based non-linear dynamics of the system. We furthermore show how the moment and thrust control inputs can be transformed into feasible actuator commands. In order to guarantee safe operation despite potential loss of a motor under which we show our system keeps operating safely, we developed an Extended Kalman Filter (EKF) based motor failure identification algorithm. We verify the effectiveness of the developed pipeline in flight experiments with and without motor failures.

We propose a human-operator guided planning approach to pushing-based manipulation in clutter. Most recent approaches to manipulation in clutter employs randomized planning. The problem, however, remains a challenging one where the planning times are still in the order of tens of seconds or minutes, and the success rates are low for difficult instances of the problem. We build on these control-based randomized planning approaches, but we investigate using them in conjunction with human-operator input. In our framework, the human operator supplies a high-level plan, in the form of an ordered sequence of objects and their approximate goal positions. We present experiments in simulation and on a real robotic setup, where we compare the success rate and planning times of our human-in-the-loop approach with fully autonomous sampling-based planners. We show that with a minimal amount of human input, the low-level planner can solve the problem faster and with higher success rates.

Developing artificial tactile sensing capabilities that rival human touch is a long-term goal in robotics and prosthetics. Gradually more elaborate biomimetic tactile sensors are being developed and applied to grasping and manipulation tasks to help achieve this goal. Here we present the neuroTac, a novel neuromorphic optical tactile sensor. The neuroTac combines the biomimetic hardware design from the TacTip sensor which mimicks the layered papillae structure of human glabrous skin, with an event-based camera (DAVIS240, iniVation) and algorithms which transduce contact information in the form of spike trains. The performance of the sensor is evaluated on a texture classification task, with four spike coding methods being implemented and compared: Intensive, Spatial, Temporal and Spatiotemporal. We found timing-based coding methods performed with the highest accuracy over both artificial and natural textures. The spike-based output of the neuroTac could enable the development of biomimetic tactile perception algorithms in robotics as well as non-invasive and invasive haptic feedback methods in prosthetics.

In this paper, we present Neural-Swarm, a nonlinear decentralized stable controller for close-proximity flight of multirotor swarms. Close-proximity control is challenging due to the complex aerodynamic interaction effects between multirotors, such as downwash from higher vehicles to lower ones. Conventional methods often fail to properly capture these interaction effects, resulting in controllers that must maintain large safety distances between vehicles, and thus are not capable of close-proximity flight. Our approach combines a nominal dynamics model with a regularized permutation-invariant Deep Neural Network (DNN) that accurately learns the high-order multi-vehicle interactions. We design a stable nonlinear tracking controller using the learned model. Experimental results demonstrate that the proposed controller significantly outperforms a baseline nonlinear tracking controller with up to four times smaller worst-case height tracking errors. We also empirically demonstrate the ability of our learned model to generalize to larger swarm sizes.

Most current surgical robotic systems lack the ability to sense tool/tissue interaction forces, which motivates research in methods to estimate these forces from other available measurements, primarily joint torques. These methods require the internal joint torques, due to the robot inverse dynamics, to be subtracted from the measured joint torques. This paper presents the use of neural networks to estimate the inverse dynamics of the da Vinci surgical robot, which enables estimation of the external environment forces. Experiments with motions in free space demonstrate that the neural networks can estimate the internal joint torques within 10% normalized rootmean-square error (NRMSE), which outperforms model-based approaches in the literature. Comparison with an external force sensor shows that the method is able to estimate environment forces within about 10% NRMSE.

We consider the problem of optimal reactive synthesis - compute a strategy that satisfies a mission specification in a dynamic environment, and optimizes a given performance metric. We incorporate task-critical information, that is only available at runtime, into the strategy synthesis in order to improve performance. Existing approaches to utilising such time-varying information require online re-synthesis, which is not computationally feasible in real-time applications. In this paper, we presynthesize a set of strategies corresponding to candidate instantiations (pre-specified representative information scenarios). We then propose a novel switching mechanism to dynamically switch between the strategies at runtime while guaranteeing all safety and liveness goals are met. We also characterize bounds on the performance suboptimality. We demonstrate our approach on two examples - robotic motion planning where the likelihood of the position of the robot's goal is updated in real-time, and an air traffic management problem for urban air mobility.

Navigation underwater traditionally is done by keeping a safe distance from obstacles, resulting in "fly-overs" of the area of interest. Movement of an autonomous underwater vehicle (AUV) through a cluttered space, such as a shipwreck or a decorated cave, is an extremely challenging problem that has not been addressed in the past. This paper proposes a novel navigation framework utilizing an enhanced version of Trajopt for fast 3D path-optimization planning for AUVs. A sampling-based correction procedure ensures that the planning is not constrained by local minima, enabling navigation through narrow spaces. Two different modalities are proposed: planning with a known map results in efficient trajectories through cluttered spaces; operating in an unknown environment utilizes the point cloud from the visual features detected to navigate efficiently while avoiding the detected obstacles. The proposed approach is rigorously tested, both on simulation and in-pool experiments, proven to be fast enough to enable safe real-time 3D autonomous navigation for an AUV.

Learning an optimal policy for autonomous driving task to confront with complex environment is a long- studied challenge. Imitative reinforcement learning is accepted as a promising approach to learn a robust driving policy through expert demonstrations and interactions with environments. However, this model utilizes non-smooth rewards, which have a negative impact on matching between navigation commands and trajectory (state-action pairs), and degrade the generalizability of an agent. Smooth rewards are crucial to discriminate actions generated from sub-optimal policy. In this paper, we propose a navigation command matching (NCM) model to address this issue. There are two key components in NCM, 1) a matching measurer produces smooth navigation rewards that measure matching between navigation commands and trajectory; 2) attention-guided agent performs actions given states where salient regions in RGB images (i.e. roadsides, lane markings and dynamic obstacles) are highlighted to amplify their influence on the final model. We obtain navigation rewards and store transitions to replay buffer after an episode, so NCM is able to discriminate actions generated from suboptimal policy. Experiments on CARLA driving benchmark show our proposed NCM outperforms previous state-of-the- art models on various tasks in terms of the percentage of successfully completed episodes. Moreover, our model improves generalizability of the agent and obtains good performance even in unseen scenarios.