ICRA 2021 Papers

Reactive stepping and push recovery for biped robots is often restricted to flat terrains because of the difficulty in computing capture regions for nonlinear dynamic models. In this paper, we address this limitation by proposing a novel 3D reactive stepper, the DeepQ stepper, that can approximately learn the 3D capture regions of both simplified and full robot dynamic models using reinforcement learning, which can then be used to find optimal steps. The stepper can take into account the entire dynamics of the robot, ignored in most reactive steppers, leading to a significant improvement in performance. The DeepQ stepper can handle nonconvex terrain with obstacles, walk on restricted surfaces like stepping stones while tracking different velocities, and recover from external disturbances for a constant low computational cost.

Research on generating or processing point clouds has become an increasingly popular domain in robotic research due to its extensive applications, such as robotic grasping, augmented reality and autonomous vehicle navigation. In this paper, we explore a new research area on point clouds - Learning to rank 3D models captured from a single depth image. In the Learning To Rank (LTR) task, we aim at optimizing the order of a list of 3D models according to the given query. Inspired by the recent advances in Natural Language Processing (NLP), we propose a novel framework, namely Deep3DRanker, for ranking 3D models by leveraging graph-based encoding and self-attention mechanisms. Comprehensive experiments are conducted to validate our methods on publicly available YCB synthetic and YCB video datasets. The promising results have shown that our proposed framework is generic enough to be applicable with any combinations of randomly positioned, oriented, and unseen object items with accuracy ranging from 59.2% to 94.9%, which shows great potentials of the proposed framework for robotic applications, in particular, for making decisions under different circumstances.

In this paper, we explore a multi-agent reinforcement learning approach to address the design problem of communication and control strategies for multi-agent cooperative transport. Typical end-to-end deep neural network policies may be insufficient for covering communication and control; these methods cannot decide the timing of communication and can only work with fixed-rate communications. Therefore, our framework exploits event-triggered architecture, namely, a feedback controller that computes the communication input and a triggering mechanism that determines when the input has to be updated again. Such event-triggered control policies are efficiently optimized using a multi-agent deep deterministic policy gradient. We confirmed that our approach could balance the transport performance and communication savings through numerical simulations.

An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined – a vehicle will yield to us if we decide to proceed first at the intersection but will proceed first if we decide to yield. However, this is not captured in most self-driving pipelines, where planning follows prediction. In this paper we propose a novel data-driven, reactive planning objective which allows a self-driving vehicle to jointly reason about its own plans as well as how other actors will react to them. We formulate the problem as an energy-based deep structured model that is learned from observational data and encodes both the planning and prediction problems. Through simulations based on both real-world driving and synthetically generated dense traffic, we demonstrate that our reactive model outperforms a non-reactive variant in successfully completing highly complex maneuvers (lane merges/turns in traffic) faster, without trading off collision rate. Please see our supplementary document https://tinyurl.com/3nukpn5b for all additional details.

Since the application of Deep Q-Learning to the continuous action domain in Atari-like games, Deep Reinforcement Learning (Deep-RL) techniques for motion control have been qualitatively enhanced. Nowadays, modern Deep-RL can be successfully applied to solve a wide range of complex decision-making tasks for many types of vehicles. Based on this context, in this paper, we propose the use of Deep-RL to perform autonomous mapless navigation for Hybrid Unmanned Aerial Underwater Vehicles (HUAUVs), robots that can operate in both, air or water media. We developed two approaches, one deterministic and the other stochastic. Our system uses the relative localization of the vehicle and simple sparse range data to train the network. We compared our approaches with an adapted version of the BUG2 algorithm for mapless navigation of aerial vehicles. Based on experimental results, we can conclude that Deep-RL-based approaches can be successfully used to perform mapless navigation and obstacle avoidance for HUAUVs. Our vehicle accomplished the navigation in two scenarios, being capable to achieve the desired target through both environments, and even outperforming the behavior-based algorithm on the obstacle-avoidance capability.

Concentric Tube Robots (CTRs), a type of continuum robot, are a collection of concentric, pre-curved tubes composed of super elastic nickel titanium alloy. CTRs can bend and twist from the interactions between neighboring tubes causing the kinematics and therefore control of the end-effector to be very challenging to model. In this paper, we develop a control scheme for a CTR end-effector in Cartesian space with no prior kinematic model using a deep reinforcement learning (DRL) approach with a goal-based curriculum reward strategy. We explore the use of curricula by changing the goal tolerance through training with constant, linear and exponential decay functions. Also, relative and absolute joint representations as a way of improving training convergence are explored. Quantitative comparisons for combinations of curricula and joint representations are performed and the exponential decay relative approach is used for training a robust policy in a noise-induced simulation environment. Compared to a previous DRL approach, our new method reduces training time and employs a more complex simulation environment. We report mean Cartesian errors of 1.29 mm and a success rate of 0.93 with a relative decay curriculum. In path following, we report mean errors of 1.37 mm in a noise-induced path following task. Albeit in simulation, these results indicate the promise of using DRL in model free control of continuum robots and CTRs in particular.

We solve active target tracking, one of the essential tasks in autonomous systems, using a deep reinforcement learning (RL) approach. In this problem, an autonomous agent is tasked with acquiring information about targets of interests using its on-board sensors. The classical challenges in this problem are system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. RL provides solutions for these challenges as the length of its effective planning horizon does not affect the computational complexity, and it drops the strong dependency of an algorithm on system models. In particular, we introduce Active Tracking Target Network (ATTN), a unified deep RL policy that is capable of solving major sub-tasks of active target tracking – in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model. Additionally, the same policy is able to navigate in obstacle environments to reach distant targets as well as explore the environment when targets are positioned in unexpected locations.

Soft robotics is an emerging technology with excellent application prospects. However, due to the inherent compliance of the materials used to build soft robots, it is extremely complicated to control soft robots accurately. In this paper, we introduce a data-based control framework for solving the soft robot underwater locomotion problem using deep reinforcement learning (DRL). We first built a soft robot that can swim based on the dielectric elastomer actuator (DEA). We then modeled it in a simulation for the purpose of training the neural network and tested the performance of the control framework through real experiments on the robot. The framework includes the following: a simulation method for the soft robot that can be used to collect data for training the neural network, the neural network controller of the swimming robot trained in the simulation environment, and the computer vision method to collect the observation space from the real robot using a camera. We confirmed the effectiveness of the learning method for the soft swimming robot in the simulation environment by allowing the robot to learn how to move from a random initial state to a specific direction. After obtaining the trained neural network through the simulation, we deployed it on the real robot and tested the performance of the control framework. The soft robot successfully achieved the goal of moving in a straight line in disturbed water. The experimental results suggest the potential of using deep reinforcement learning to improve the locomotion ability of mobile soft robots.

In this work, we propose a novel deep online correction (DOC) framework for monocular visual odometry. The whole pipeline has two stages: First, depth maps and initial poses are obtained from convolutional neural networks (CNNs) trained in self-supervised manners. Second, the poses predicted by CNNs are further improved by minimizing photometric errors via gradient updates of poses during inference phases. The benefits of our proposed method are twofold: 1) Different from online-learning methods, DOC does not need to calculate gradient propagation for parameters of CNNs. Thus, it saves more computation resources during inference phases. 2) Unlike hybrid methods that combine CNNs with traditional methods, DOC fully relies on deep learning (DL) frameworks. Though without complex back-end optimization modules, our method achieves outstanding performance with relative transform error (RTE) = 2.0% on KITTI Odometry benchmark for Seq. 09, which outperforms traditional monocular VO frameworks and is comparable to hybrid methods.

Current aerial robots are increasingly adaptive; they can morph to enable operation in changing conditions to complete diverse missions. Each mission may require the robot to conduct a different task. A conventional learning approach can handle these variations when the system is trained for similar tasks in a representative environment. However, it may result in overfitting to the new data stream or the failure to adapt, leading to degradation or a potential crash. These problems can be mitigated with an excessive amount of data and embedded model, but the computational power and the memory of the aerial robots are limited. In order to address the variations in the model, environment as well as the tasks within onboard computation limitations, we propose a deep neuromorphic controller approach with variable topologies to handle each different condition and the data stream with a feasible computation and memory allocation. The proposed approach is based on a deep neuromorphic (multi and variable layered neural network) controller with dynamic depth and progressive layer adaptation for each new data stream. This adaptive structure is combined with a switching function to form a sliding mode controller. The network parameter update rule guarantees the stability of the closed loop system by the convergence of the error dynamics to the sliding surface. Being the first implementation on an aerial robot in this context, the results illustrate the adaptation capability, stability, computational efficiency as well as the real-time validation.

In this paper, we address the problem of estimating dense depth from a sequence of images using deep neural networks. Specifically, we employ a dense-optical-flow network to compute correspondences and then triangulate the point cloud to obtain an initial depth map. Parts of the point cloud, however, may be less accurate than others due to lack of common observations or small parallax. To further increase the triangulation accuracy, we introduce a depth-refinement network (DRN) that optimizes the initial depth map based on the image’s contextual cues. In particular, the DRN contains an iterative refinement module (IRM) that improves the depth accuracy over iterations by refining the deep features. Lastly, the DRN also predicts the uncertainty in the refined depths, which is desirable in applications such as measurement selection for scene reconstruction. We show experimentally that our algorithm outperforms state-of-the-art approaches in terms of depth accuracy, and verify that our predicted uncertainty is highly correlated to the actual depth error.

Tool tip visualization is an essential component of multiple robotic surgical and interventional procedures. In this paper, we introduce a real-time photoacoustic visual servoing system that processes information directly from raw acoustic sensor data, without requiring image formation or segmentation in order to make robot path planning decisions to track and maintain visualization of tool tips. The performance of this novel deep learning-based visual servoing system is compared to that of a visual servoing system which relies on image formation followed by segmentation to make and execute robot path planning decisions. Experiments were conducted with a plastisol phantom, ex vivo tissue, and a needle as the interventional tool. Needle tip tracking performance with the deep learning-based approach outperformed that of the image-based segmentation approach by 67.7% and 55.3% in phantom and ex vivo tissue, respectively. In addition, the deep learning-based system operated within the frame-rate-limiting 10 Hz laser pulse repetition frequency rate, with mean execution times of 75.2 ms and 73.9 ms per acquisition frame with phantom and ex vivo tissue, respectively. These results highlight the benefits of our new approach to integrate deep learning with robotic systems for improved automation and visual servoing of tool tips.

Increasing research attention has been attracted to automatic plug-in charging in an unmanned and dangerous environment. In this work, we develop an object detection solution based on deep learning on 3D point clouds using a mobile robot manipulator to provide mobility and manipulation. In this solution, the 3D point cloud technology is adopted to measure the shapes and depth information for plugin charging. Then the deep learning is employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions, irregular distribution, and structural ambiguity of point clouds. We utilize a mobile robot manipulator carrying a 3D camera and a gripper to detect the targeted objects and automate plug-in charging operations. The proposed 3D object detection principle and procedure for the automatic plug-in charging are presented in detail. The automatic plug-in charging testing is conducted to validate the developed 3D object detection algorithm using a mobile robot manipulator.

Teleoperation of robots can be challenging, especially for novice users with little to no experience at such tasks. The difficulty is largely due to the numerous degrees of freedom users must control and their limited perception bandwidth. To help mitigate these challenges, we propose in this paper a solution which relies on artificial intelligence to understand user intended motion and then on mixed reality to communicate the estimated trajectories to the users in an intuitive manner. User intended motion is estimated using a deep learning network trained on a dataset of motion primitives. During teleoperation, the estimated motions are augmented onto a first-person live video feed from the robot. Finally, if a suggested motion is accepted by the user, the robot is driven along that trajectory in an autonomous manner. We validate our proposed mixed reality teleoperation scheme with simulation experiments on a drone and demonstrate, through subjective and objective evaluation, its advantages over other teleoperation methods.

Navigation through dynamic pedestrian environments in a socially compliant manner is still a challenging task for autonomous vehicles. Classical methods usually lead to unnatural vehicle behaviours for pedestrian navigation due to the difficulty in modeling social conventions mathematically. This paper presents an end-to-end path planning system that achieves autonomous navigation in dynamic environments through imitation learning. The proposed system is based on a fully convolutional neural network that maps the raw sensory data into a confidence map for path extraction. Additionally, a classification network is introduced to reduce the unnecessary re-plannings and ensures that the vehicle goes back to the global path when re-planning is not needed. The imitation learning based path planner is implemented on an autonomous wheelchair and tested in a new real-world dynamic pedestrian environment. Experimental results show that the proposed system is able to generate paths for different driving tasks, such as pedestrian following, static and dynamic obstacles avoidance, etc. In comparison to the state-of-the-art method, our system is superior in terms of generating human-like trajectories.

Rotation invariance is a crucial property for 3D object classification, which is still a challenging task. State-of-the-art deep learning-based works require a massive amount of data augmentation to tackle this problem. This is however inefficient and classification accuracy suffers a sharp drop in experiments with arbitrary rotations. We introduce a new descriptor that can globally and locally capture the surface geometry properties and is based on a combination of spherical harmonics energy and point feature representation. The proposed descriptor is proven to fulfill the rotation-invariant property. A limited bandwidth spherical harmonics energy descriptor globally describes a 3D shape and its rotation-invariant property is proven by utilizing the properties of a Wigner D-matrix, while the point feature representation captures the local features with a KNN to build the connection to its neighborhood. We propose a new network structure by extending PointNet++ with several adaptations that can hierarchically and efficiently exploit local rotation-invariant features. Extensive experimental results show that our proposed method dramatically outperforms most state-of-the-art approaches on standard rotation-augmented 3D object classification benchmarks as well as in robustness experiments on point perturbation, point density, and partial point clouds.

The analysis of facial expression is a very complex and challenging problem. Most researches for automated Facial Expression Recognition (FER) are mainly based on deep learning networks, rarely considering data imbalance. This paper commits to addressing the long-tail distribution problems among large-scale datasets in wild. Inspired by the continual learning method, we reconstruct multi-subsets first by randomly selecting from head classes and up-sampling tail classes. A pre-trained backbone is then introduced to learn general weights in a repeatedly train-prune fashion. Hereafter, our approach creatively trains a new classifier based on union parameters previously preserved and achieves an outperformance without extra parameters added in, using the gradual-prune technique. The results show that the independent training of classifiers has been a contributing factor. We successfully conduct this experiment with several classic networks, prove its effectiveness in training a deep network on imbalanced dataset. In the face of the poor performance in current FER, we find that domain knowledge is somehow affecting the accuracy of recognition by further exploring the obstacles from the image itself.Code available at https://github.com/Epicghx/FER

Planning in realistic environments requires searching in large planning spaces. Affordances are a powerful concept to simplify this search, because they model what actions can be successful in a given situation. However, the classical notion of affordance is not suitable for long horizon planning because it only informs the robot about the immediate outcome of actions instead of what actions are best for achieving a long-term goal. In this paper, we introduce a new affordance representation that enables the robot to reason about the longterm effects of actions through modeling what actions are afforded in the future. Based on the new representation, we develop a learning-to-plan method, Deep Affordance Foresight (DAF), that learns partial environment models of affordances of parameterized motor skills through trial-and-error. We evaluate DAF on two challenging manipulation domains and show that it can effectively learn to carry out multi-step tasks, share learned affordance representations among different tasks, and learn to plan with high-dimensional image inputs.

Robotic manipulation of unknown objects is an important field of research. Practical applications occur in many real-world settings where robots need to interact with an unknown environment. We tackle the problem of reactive grasping by proposing a method for unknown object tracking, grasp point sampling and dynamic trajectory planning. Our object tracking method combines Siamese Networks with an Iterative Closest Point approach for pointcloud registration into a method for 6-DoF unknown object tracking. The method does not require further training and is robust to noise and occlusion. We propose a robotic manipulation system, which is able to grasp a wide variety of formerly unseen objects and is robust against object perturbations and inferior grasping points.