ICRA 2021 Papers

The sampling-based partial motion planning algorithm has come into widespread application in dynamic mobile robot navigation due to its low calculation costs and excellent performance in avoiding obstacles. However, when confronted with complicated scenarios, the motion planning algorithms are easily caught in traps. In order to solve this problem, this paper proposes a knowledge-based fast motion planning algorithm based on Risk-RRT, which guides motion planning by constructing a topological feature tree and generating a heuristic path from the tree. Firstly, an online topological feature learning method is proposed to simultaneously extract the features during the motion of the robot by means of the dual-channel scale filter and the secondary distance fusion. The learning process is completed until the feature points can represent arbitrary obstacle-free grid points of the whole map. Secondly, the topological feature tree is constructed with environmental feature points and the heuristic motion planning can be carried out on the feature tree. For one map, once the construction of the feature tree finishes, it can be reused as a prior knowledge in the following heuristic motion planning process, which will further improve the efficiency of searching feasible paths. The experimental results demonstrate that our proposed method can remarkably reduce the time taken to find a heuristic path and enhance the success rate of navigation in trapped environments.

We consider the task of grasping a target object based on a natural language command query. Previous work primarily focused on localizing the object given the query, which requires a separate grasp detection module to grasp it. The cascaded application of two pipelines incurs errors in overlapping multi-object cases due to ambiguity in the individal outputs. This work proposes a model named Command Grasping Network (CGNet) to directly output command satisficing grasps from RGB image and textual command inputs. A dataset with ground truth (image, command, grasps) tuple is generated based on the VMRD dataset to train the proposed network. Experimental results on the generated test set show that CGNet outperforms a cascaded object-retrieval and grasp detection baseline by a large margin. Three physical experiments demonstrate the functionality and performance of CGNet.

In this paper, we present a hybrid position/force controller for operating joint robots. The hybrid controller has two goals—motion tracking and force regulating. As long as these two goals are not mutually exclusive, they can be decoupled in some way. In this work, we make use of the smooth and invertible mapping from the joint space to the task space to decouple the two control goals and design controllers separately. The traditional motion controller in the task space is used for motion control, while the force controller is designed through manipulating the desired trajectory to regulate the force indirectly. Two case studies—contour tracking/polishing surfaces and grabbing boxes with two robotic arms—are presented to show the efficacy of the hybrid controller, and simulations with physics engines are carried out to validate the efficacy of the proposed method.

Self-collision detection and avoidance are essential for reactive control, in particular for dynamics robots equipped with legs or arms. Yet, only few control methods are able to handle such constraints, and it is often necessary to rely on path planning to define a collision-free trajectory that the controller would then track. In this paper, we introduce a combination of two lightweight, conservative and smooth models to generically handle self-collisions in robot control. For pairs of bodies that are far from one another on average (e.g. segments of distinct legs), we rely on a standard forward kinematics approach, using simplified geometries for which we provide analytical derivatives. For bodies that are moving close to one another, we propose to use a data-driven approach, with datasets generated thanks to a standard collision library. We then build a simple torque-based controller that can be implemented on top of any control law to prevent unexpected self-collision. This controller is meant to be implemented as a low-level protection, directly on the robot hardware. We also provide an open-source library to generate ANSI-C code for any robot model, experimented on the real quadruped Solo.

A patient would contract surface muscles as a reaction called muscle guarding when experiencing discomfort and pain during physical palpation. This reaction carries important information about an affected location. Training physicians to regulate palpation forces to elicit just enough muscle tension is a challenge using real patients. Tunable stiffness mechanisms enabled by soft robotics can be effectively integrated into medical simulator designs for effective clinical education. In this paper, we propose a controllable stiffness muscle layer to simulate guarding for abdominal palpation training. Designs with soft, fine, and rigid granular jamming, stretchable and non-stretchable layer jamming mechanisms were tested and evaluated as methods to create controllable stiffness muscle. User studies have been carried out on 10 naive participants to differentiate the tense and relaxed abdomen with the proposed jamming mechanisms. Muscle samples made of ground coffee (fine granular jamming) and latex layers (stretchable layer jamming) show good usability in simulating abdomen with different stiffness with at least 75% of the user data exhibits more than 70% of decision accuracy for both tested palpation gestures (single finger and multiple fingers) after short pre-training.

We address the problem of class agnostic, joint instance segmentation of scene data. While learning-based semantic instance segmentation methods have achieved impressive progress, their use is limited in robotics applications due to reliance on expensive training data annotations and assumptions of single sensor modality or known object classes. We propose a novel graph-based instance segmentation approach that combines information from a 2D image sequence and a 3D point cloud capturing the scene. Our approach propagates information with a general graph representation to produce a segmentation taking into account both geometric and photometric information. This allows us to leverage information from complementary sensor modalities without requiring training data. Our method shows improved object recall and boundary identification over state-of-the-art RGB-D segmentation methods. We demonstrate generality by evaluating on both RGB-D data and a LiDAR+image sensor data.

Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D human pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatiotemporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatiotemporal sequences and effectively achieves real-time 3D human pose estimation. In this work, we improve the learning of kinematic constraints in the human skeleton: posture, local kinematic connections, and symmetry by modeling local and global spatial information via attention mechanisms. To adapt to single- and multi-frame estimation, the dilated temporal model is employed to process varying skeleton sequences. Also, importantly, we carefully design the interleaving of spatial semantics with temporal dependencies to achieve a synergistic effect. To this end, we propose a simple yet effective graph attention spatio-temporal convolutional network (GAST-Net) that comprises of interleaved temporal convolutional and graph attention blocks. Experiments on two challenging benchmark datasets (Human3.6M and HumanEva-I) and YouTube videos demonstrate that our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation. Code, video, and supplementary information is available at: http://www.juanrojas.net/gast/

Conventional coverage path planning algorithms are mainly based on the zigzag and spiral patterns or their combinations. The traversal order is limited by the linear or inside-outside manner. We propose a new set of coverage patterns induced from geometric folding operations, called the geometric folding pattern, to make coverage paths with more flexible traversal order. We study the modeling and parameterization of the geometric folding patterns. Then, a sampling operator is introduced. Based on the computational tools, we demonstrate the application of the proposed patterns in designing coverage paths. We show that the simple geometric folding patterns are flexible and controllable, which enables more choices for the coverage path planning problem.

We propose a new generative model-based predictive display for robotic teleoperation over high-latency communication links. Our method is capable of rendering photo-realistic images of the scene to the human operator in real time from RGB-D images acquired by the remote robot. A preliminary exploration stage is used to build a coarse 3D map of the remote environment and to train a generative model, both of which are then used to generate photo-realistic images for the human operator based on the commanded pose of the robot. Data captured by the remote robot is used to dynamically update the 3D map, enabling teleoperation in the presence of new and relocated objects. Various experiments validate our proposed method’s performance and benefits over alternative methods.

Both geometric and semantic information of the search space are imperative for a good plan. We encode those properties in a weighted colored graph (geometric information in terms of edge weight and semantic information in terms of edge and vertex color) and propose a generalized A∗ to find the shortest path among the set of paths with minimal inclusion of low-ranked color edges. We prove the completeness and optimality of this Class-Ordered A∗ (COA∗ ) algorithm with respect to the hereto defined notion of optimality. The utility of COA∗ is numerically validated in a ternary graph with feasible, infeasible, and unknown vertices and edges for the cases of a 2D mobile robot, a 3D robotic arm, and a 5D robotic arm with limited sensing capabilities. We compare the results of COA∗ to that of the regular A∗ algorithm, the latter of which finds a shortest path regardless of the semantic information, and we show that the COA∗ dominates the A∗ solution in terms of finding less uncertain paths.

Data synthesizing is crucial for data-driven anomaly prognostics on physical machines. We propose the first general-purpose anomalous scenario synthesizer, GPASS, for rotary equipment. More specifically, we present a design of implementing modular rotational damping, large lateral force, with high-frequency range capability as fundamental modes of physical inputs. The GPASS is a general-purpose platform that can impose inputs independently or jointly, and generate an extensive range of anomalous scenarios on the same subject. Finally, it has the capability of capturing multi-variate sensor readings on the same anomalous event. Experimental results demonstrate that the synthesizer can dynamically and accurately introduce lateral force at specified magnitudes and frequencies, proving the effectiveness of the proposed device.

The cerebellum plays a distinctive role within our motor control system to achieve fine and coordinated motions. While cerebellar lesions do not lead to a complete loss of motor functions, both action and perception are severally impacted. Hence, it is assumed that the cerebellum uses an internal forward model to provide anticipatory signals by learning from the error in sensory states. In some studies, it was demonstrated that the learning process relies on the jointspace error. However, this may not exist. This work proposes a novel fully spiking neural system that relies on a forward predictive learning by means of a cellular cerebellar model. The forward model is learnt thanks to the sensory feedback in task-space and it acts as a Smith predictor. The latter predicts sensory corrections in input to a differential mapping spiking neural network during a visual servoing task of a robot arm manipulator. In this paper, we promote the developed control system to achieve more accurate target reaching actions and reduce the motion execution time for the robotic reaching tasks thanks to the cerebellar predictive capabilities.

Recent achievements in depth prediction from a single RGB image have powered the new research area of combining convolutional neural networks (CNNs) with classical simultaneous localization and mapping (SLAM) algorithms. The depth prediction from a CNN provides a reasonable initial point in the optimization process in the traditional SLAM algorithms, while the SLAM algorithms further improve the CNN prediction online. However, most of the current CNN-SLAM approaches have only taken advantage of the depth prediction but not yet other products from a CNN. In this work, we explore the use of the outlier mask, a by-product from unsupervised learning of depth from video, as a prior in a classical probability model for depth estimate fusion to step up the outlier-resistant tracking performance of a SLAM front-end. On the other hand, some of the previous CNN-SLAM work builds on feature-based sparse SLAM methods, wasting the per-pixel dense prediction from a CNN. In contrast to these sparse methods, we devise a dense CNN-assisted SLAM frontend that is implementable with TensorFlow and evaluate it on both indoor and outdoor datasets.

Predicting future sensory states is crucial for learning agents such as robots, drones, and autonomous vehicles. In this paper, we couple multiple sensory modalities with exploratory actions and propose a predictive neural network architecture to address this problem. Most existing approaches rely on large, manually annotated datasets, or only use visual data as a single modality. In contrast, the unsupervised method presented here uses multi-modal perceptions for predicting future visual frames. As a result, the proposed model is more comprehensive and can better capture the spatio-temporal dynamics of the environment, leading to more accurate visual frame prediction. The other novelty of our framework is the use of sub-networks dedicated to anticipating future haptic, audio, and tactile signals. The framework was tested and validated with a dataset containing 4 sensory modalities (vision, haptic, audio, and tactile) on a humanoid robot performing 9 behaviors multiple times on a large set of objects. While the visual information is the dominant modality, utilizing the additional non-visual modalities improves the accuracy of predictions.

Loop closure detection (LCD) is an essential module for simultaneous localization and mapping (SLAM), which can correct accumulated errors after long-term explorations. The widely used bag-of-words (BoW) model can not satisfy well the requirements of both low time consumption and high accuracy for a mobile platform. In this paper, we propose a novel LCD algorithm based on motion knowledge. We give a flexible and efficient detection strategy and also give flexible and efficient combinations of a global binary feature extracted by convolutional neural network (CNN) and a hand-crafted local binary feature. We take a continuous motion model, grid-based motion statistics (GMS) and motion states as motion knowledge. Furthermore, we fuse the proposed LCD with a visual-inertial odometry (VIO) system to correct localization errors by a pose graph optimization. Comparative experiments with state-of-the-art LCD algorithms on typical datasets have been carried out, and the results demonstrate that our proposed method achieves quite high recall rates and quite high speed at 100% precision. Moreover, experimental results from VIO further validate the effectiveness of the proposed method.

Magnetic manipulation provides a versatile, remote, noninvasive, and cost-effective strategy in a variety of applications. Till now, many different configurations of magnetic manipulation systems have been developed to address different needs on force, torque, accuracy, and accessibilities. Magnetic field mapping can help to explore the exact map of the magnetic field in the working space and guarantee the homogeneity of the magnetic field. In this paper, a flexible mapping method is employed to solve the scalar potential of the magnetic source by using the separation of variables in Cartesian coordinates. Levenberg-Marquardt Algorithm (LMA) and Whale Optimization Algorithm (WOA) are set to the solver of the model. The work is evaluated in the mapping of an eight-pole magnetic manipulation system. The result of numerical simulation shows that the coefficient of determination R2 of the model reaches99.81%, and the actual system mapping obtains R2 value of 99.57%. This technique can directly be used to calculate the magnetic flux density and gradient field in a short period (≈1ms). Finally, the manipulation of a permanent magnet under the control magnetic field mapping and PID controller demonstrates the effectiveness of the proposed method.

Multi-agent control of a robot using multiple controllers is vital in domains like shared control and reliable control. The strategy of assigning a varying priority (authority) to each agent controller, and commanding the robot using an authority-weighted sum of the forces produced by all the agents has been exploited in prior works. In this paper, firstly, we show that this strategy results in a loss of passivity, and we identify the passivity-disrupting scaling-related terms. Secondly, we propose a model independent method to ensure finite-gain L2 stability of such a generic multi-agent robot control system with time-varying force scaling factors. Thirdly, the analysis is validated with simulations and hardware experiments.

Target-driven visual navigation is a challenging problem that requires a robot to find the goal using only visual inputs. Many researchers have demonstrated promising results using deep reinforcement learning (deep RL) on various robotic platforms, but typical end-to-end learning is known for its poor extrapolation capability to new scenarios. Therefore, learning a navigation policy for a new robot with a new sensor configuration or a new target still remains a challenging problem. In this paper, we introduce a learning algorithm that enables rapid adaptation to new sensor configurations or target objects with a few shots. We design a policy architecture with latent features between perception and inference networks and quickly adapt the perception network via meta-learning while freezing the inference network. Our experiments show that our algorithm adapts the learned navigation policy with only three shots for unseen situations with different sensor configurations or different target colors. We also analyze the proposed algorithm by investigating various hyperparameters.

Robotic systems evolving in hazardous and harsh environment are prone to mission failure or system loss in presence of faults. This paper presents a fault tolerant methodology, implemented into a control architecture of an underwater robot that executes biological monitoring missions. High level constraint violations (mission, safety, energy, time and localization) and low level faults (software and hardware faults) are considered using a method based on fault trees. These undesirable events are detected and treated by a fault tolerant module that decides to recover at low level or to give a feedback to the mission manager which selects the high level reaction. This fault tolerant architecture has been tested on real field conditions, and we illustrate our methodology on a set of selected events. We conclude about reliability improvement of low cost underwater robots for complex and long missions.