IROS 2022 Papers

Particle filters are suited to solve nonlinear and non-Gaussian estimation problems which find numerous applications in autonomous systems navigation. Previous works on Laplace Particle Filter on Lie groups (LG-LPF) demonstrated its robustness and accuracy on challenging navigation scenarios compared to classic particle filters. Nevertheless, LG-LPF is applicable when the prior probability density and the likelihood have a predominant mode, which narrows the scope of applications of this method. Thus, this paper proposes a generalized strategy to use LG-LPF while keeping its benefits. The core idea is to compute an accurate multimodal importance function based on local optimizations and resample the particles accordingly. This approach is compared to a Laplace Particle Filter (LPF) designed in the Euclidean space, on a UAV navigation scenario with ambiguous Doppler measurements. The Lie group approach shows improved accuracy and robustness in every case, even with a reduced number of particles.

Trajectory prediction is one of the essential tasks for autonomous vehicles. Recent progress in machine learning gave birth to a series of advanced trajectory prediction algorithms. Lately, the effectiveness of using graph neural networks (GNNs) with vectorized representations for trajec-tory prediction has been demonstrated by many researchers. Nonetheless, these algorithms either pay little attention to models' generalizability across various scenarios or simply assume training and test data follow similar statistics. In fact, when test scenarios are unseen or Out-of-Distribution (OOD), the resulting train-test domain shift usually leads to significant degradation in prediction performance, which will impact downstream modules and eventually lead to severe accidents. Therefore, it is of great importance to thoroughly investigation of the prediction models in terms of their generalizability, which can not only help identify their weaknesses but also provide insights on how to improve these models. This paper proposes a generalizability analysis framework using feature attribution methods to help interpret black-box models. For the case study, we provide an in-depth generalizability analysis of one of the state-of-the-art graph-based trajectory predictors that utilize vectorized representation. Results show significant performance degradation due to domain shift, and feature attribution provides insights to identify potential causes of these problems. Finally, we conclude the common prediction challenges and how weighting biases induced by the training process can deteriorate the accuracy.

Physical simulation is an indispensable component of robotics simulation platforms that serves as the basis for a plethora of research directions. Looking strictly at robotics, the common characteristic of the most popular physics engines, such as ODE, DART, MuJoCo, bullet, SimBody, PhysX or RaiSim, is that they focus on the solution of articulated rigid bodies with collisions and contacts problems, while paying less attention to other physical phenomena. This restriction limits the range of addressable simulation problems, rendering applications such as soft robotics, cloth simulation, simulation of viscoelastic materials, and fluid dynamics, especially surface swimming, infeasible. In this work, we present Gazebo Fluids, an open-source extension of the popular Gazebo robotics simulator that enables the interaction of articulated rigid body dynamics with particle-based fluid and deformable solid simulation. We implement fluid dynamics and highly viscous and elastic material simulation capabilities based on the Smoothed Particle Hydrodynamics method. We demonstrate the practical impact of this extension for previously infeasible application scenarios in a series of experiments, showcasing one of the first self-propelled robot swimming simulations with SPH in a robotics simulator.

Accurate and reliable state estimation is becoming increasingly important as robots venture into the real world. Gaussian variational inference (GVI) is a promising alternative for nonlinear state estimation, which estimates a full probability density for the posterior instead of a point estimate as in maximum a posteriori (MAP)-based approaches. GVI works by optimizing for the parameters of a multivariate Gaussian (MVG) that best agree with the observed data. However, such an optimization procedure must ensure the parameter constraints of a MVG are satisfied; in particular, the inverse covariance matrix must be positive definite. In this work, we propose a tractable algorithm for performing state estimation using GVI that guarantees that the inverse covariance matrix remains positive definite and is well-conditioned throughout the optimization procedure. We evaluate our method extensively in both simulation and real-world experiments for range-only localization. Our results show GVI is consistent on this problem, while MAP is over-confident.

For biomedical applications in targeted therapy delivery and interventions, a large swarm of micro-scale particles (“agents”) has to be moved through a maze-like environment (“vascular system”) to a target region (“tumor”). Due to limited on-board capabilities, these agents cannot move autonomously; instead, they are controlled by an external global force that acts uniformly on all particles. In this work, we demonstrate how to use a time-varying magnetic field to gather particles to a desired location. We use reinforcement learning to train networks to efficiently gather particles. Methods to overcome the simulation-to-reality gap are explained, and the trained networks are deployed on a set of mazes and goal locations. The hardware experiments demonstrate fast convergence, and robustness to both sensor and actuation noise. To encourage extensions and to serve as a benchmark for the reinforcement learning community, the code is available at Github.

Legged locomotion in humans is governed by natural dynamics of the human body and neural control. One mechanism that is assumed to contribute to the high efficiency of human walking is the impulsive ankle push-off, which potentially powers the swing leg catapult. However, the mechanics of the human lower leg with its complex muscle-tendon units spanning over single and multiple joints is not yet understood. Legged robots allow testing the interaction between complex leg mechanics, control, and environment in real-world walking gait. We developed a 0.49 m tall, 2.2 kg anthropomorphic bipedal robot with Soleus and Gastrocnemius muscle-tendon units represented by linear springs, acting as mono- and biarticular elastic structures around the robot's ankle and knee joints. We tested the influence of three Soleus and Gastrocnemius spring-tendon configurations on the ankle power curves, the coordination of the ankle and knee joint movements, the total cost of transport, and walking speed. We controlled the robot with a feed-forward central pattern generator, leading to walking speeds between 0.35 m/s and 0.57 m/s at 1.0 Hz locomotion frequency, at 0.35 m leg length. We found differences between all three configurations; the Soleus spring-tendon modulates the robot's speed and energy efficiency likely by ankle power amplification, while the Gastrocnemius spring-tendon changes the movement coordination between ankle and knee joints during push-off.

Chemical gas dispersion poses considerable threat to humans, animals and the environment. The research areas of gas source localization and gas distribution mapping aim to localize the source of gas leaks and map the gas plume respectively, in order to help the coordination of swift rescue missions. Although very similar, these two areas are often treated separately in literature. In some cases, inferences on the gas distribution are made a posteriori from the source location, or vice-versa. In this paper, we introduce GaSLAM, a methodology that couples the estimation of the gas map and the source location using two state of the art algorithms with a novel navigation strategy based on informative quantities. The synergistic approach allows our algorithm to achieve a good estimation of both objectives and push the navigation strategies towards informative areas of the experimental volume. We validate the algorithm in simulation and with physical experiments in varying environmental conditions. We show that the algorithm improves on the source location estimate compared to a similar approach found in literature, and is able to deliver good quality maps of the gas distribution.

Collision avoidance in emergency situations is a crucial and challenging task in motion planning for autonomous vehicles. Especially in the field of optimization-based planning using nonlinear model predictive control, many efforts to achieve real-time performance are still ongoing. Among various approaches, the iterative linear quadratic regulator (iLQR) is known as an efficient means of nonlinear optimization. Additionally, parallel computing architectures, such as GPUs, are more widely applied in autonomous vehicles. In this paper, we propose 1) a parallel computing framework for iLQR with input constraints considering the characteristics of the problem and 2) a proper environmental formulation that can be covered with single-precision floating-point computation of the GPU. The GPU-accelerated framework was tested on a real-time simulation-in-the-loop system using CarMaker and ROS at a 20 Hz sampling rate on a low-performance mobile computer and was compared against the same framework realized with a CPU.

Robot-assisted gastrointestinal endoscopic surgery (GES) as a kind of natural orifice transluminal endoscopic surgery (NOTES) is the next-generation minimally invasive surgery (MIS). Besides, rendering certain autonomy to a Gas-trointestinal Endoscopic Surgical Robot (GESR) is promising but highly challenging. Therefore, to accelerate the development and augment the autonomy of GESR, we use CoppeliaSim to develop the first robotic simulator for the GESR system (GESRsim) based on our previous design. The GESRsim provides several 3D models and kinematics of our designed manipulators and endoscopic snake bone. Additionally, we build several scenes for robotic GES training and then utilize different programming interfaces to perform teleoperation. Furthermore, several advanced control algorithms, including visual servoing (VS) and deep reinforcement learning (DRL), are implemented to verify the performance of the GESRsim.

Grasping in dense clutter is a fundamental skill for autonomous robots. However, the crowdedness and oc-clusions in the cluttered scenario cause significant difficul-ties to generate valid grasp poses without collisions, which results in low efficiency and high failure rates. To address these, we present a generic framework called GE-Grasp for robotic motion planning in dense clutter, where we leverage diverse action primitives for occluded object removal and present the generator-evaluator architecture to avoid spatial collisions. Therefore, our GE-Grasp is capable of grasping objects in dense clutter efficiently with promising success rates. Specifically, we define three action primitives: target-oriented grasping for target capturing, pushing, and nontarget-oriented grasping to reduce the crowdedness and occlusions. The gen-erators effectively provide various action candidates referring to the spatial information. Meanwhile, the evaluators assess the selected action primitive candidates, where the optimal action is implemented by the robot. Extensive experiments in simulated and real-world environments show that our approach outperforms the state-of-the-art methods of grasping in clutter with respect to motion efficiency and success rates. Moreover, we achieve comparable performance in the real world as that in the simulation environment, which indicates the strong gen-eralization ability of our GE-Grasp. Supplementary material is available at: https://github.com/CaptainWuDaoKou/GE-Grasp.

Combining multiple sensors enables a robot to maximize its perceptual awareness of environments and enhance its robustness to external disturbance, crucial to robotic navigation. This paper proposes the FusionPortable benchmark, a complete multi-sensor dataset with a diverse set of sequences for mobile robots. This paper presents three contributions. We first advance a portable and versatile multi-sensor suite that offers rich sensory measurements: 10Hz LiDAR point clouds, 20Hz stereo frame images, high-rate and asynchronous events from stereo event cameras, 200Hz inertial readings from an IMU, and 10Hz GPS signal. Sensors are already temporally synchronized in hardware. This device is lightweight, self-contained, and has plug-and-play support for mobile robots. Second, we construct a dataset by collecting 17 sequences that cover a variety of environments on the campus by exploiting multiple robot platforms for data collection. Some sequences are challenging to existing SLAM algorithms. Third, we provide ground truth for the decouple localization and mapping performance evaluation. We additionally evaluate state-of-the-art SLAM approaches and identify their limitations. The dataset, consisting of raw sensor measurements, ground truth, calibration data, and evaluated algorithms, will be released.

In an attempt to imitate the success of transformers in the field of natural language processing into computer vision tasks, vision transformers (ViTs) have recently gained attention. Performance breakthroughs have been achieved in coarse-grained tasks like classification. However, dense prediction tasks, such as detection, segmentation, and depth estimation, require additional modifications and have been tackled only in an ad-hoc manner, by replacing the convolutional neural network encoder backbone of an existing architecture with a ViT. This study proposes a fully convolutional transformer that can perform both coarse and dense prediction tasks. The proposed architecture is, to the best of our knowledge, the first architecture composed of attention layers, even in the decoder part of the network. This is because our newly proposed local-global attention (LGA) can flexibly perform both downsampling and upsampling of spatial features, which are key operations required for dense prediction. Against existing ViTs on classification tasks, our architecture shows a reasonable trade-off between performance and efficiency. In the depth estimation task, our architecture achieves performance comparable to that of state-of-the-art transformer-based methods.

Timing is an important property for robotic systems that continuously interact with our physical world. Variation in program execution time caused by limited computational resources or system resource contention can lead to significant impact on algorithmic result accuracy. Even though recent work has found Simultaneous Localization And Mapping (SLAM) to be timing-sensitive, little exists in understanding the interactions between the timing variations in SLAM systems and the corresponding degradation. In this paper we conduct a systematic analysis of nine state-of-the-art SLAM systems and dissect the root causes of their degradation. We discovered that timing-induced errors are generated either from delayed execution in certain critical tasks, or from desynchronization in sensor fusion. Based on the insights from our analysis, we propose a solution that combines selective fusion on data in the front end and temporal budget optimization on bundle adjust-ment in the backend to mitigate the impacts of unexpected timing variation adaptively. Experimental results show that our proposed method makes it possible to migrate expensive algorithms to low-cost platforms without laborious tuning, while making the SLAM system robust against the effects of abnormal timing.

Single image 3D face reconstruction with accurate geometric details is a critical and challenging task due to the similar appearance on the face surface and fine details in organs. In this work, we introduce a self-supervised 3D face reconstruction approach from a single image that can recover detailed textures under different camera settings. The proposed network learns high-quality disparity maps from stereo face images during the training stage, while just a single face image is required to generate the 3D model in real applications. To recover fine details of each organ and facial surface, the framework introduces facial landmark spatial consistency to constrain the face recovering learning process in local point level and segmentation scheme on facial organs to constrain the correspondences at the organ level. The face shape and textures will further be refined by establishing holistic constraints based on the varying light illumination and shading information. The proposed learning framework can recover more accurate 3D facial details both quantitatively and qualitatively compared with state-of-the-art 3DMM and geometry-based reconstruction algorithms based on a single image.

Humans are able to negotiate downstep behaviors-both planned and unplanned-with remarkable agility and ease. The goal of this paper is to systematically study the translation of this human behavior to bipedal walking robots, even if the morphology is inherently different. Concretely, we begin with human data wherein planned and unplanned downsteps are taken. We analyze this data from the perspective of reduced-order modelling of the human, encoding the center of mass (CoM) kinematics and contact forces, which allows for the translation of these behaviors into the corresponding reduced-order model of a bipedal robot. We embed the resulting behaviors into the full-order dynamics of a bipedal robot via nonlinear optimization-based controllers. The end result is the demonstration of planned and unplanned downsteps in simulation on an underactuated walking robot.

Autonomous navigation in dense vegetation remains an open challenge and is an area of major interest for the research community. In this paper we propose a novel traversability estimation method, the Forest Traversability Map, that gives autonomous ground vehicles the ability to navigate in harsh forests or densely vegetated environments. The method estimates travers ability in unstructured environments dominated by vegetation, void of any dominant human structures, gravel or dirt roads, with higher accuracy than the state of the art: we demonstrate an improvement of over 20% F1 score (from 0.71 to 0.91) on challenging real-world data. Our method is based on 3D voxel representation and introduces a robust colour fusion method to overcome occlusion and frequent changes of lighting conditions in these environments. We also introduce and fuse multi-return lidar measurements into our probabilistic map representation in a recursive manner. Finally, we include information of neighboring voxels to increase our ability to assess the terrain travers ability correctly. These measures improve the state-of-the-art results and allow for effective traversability estimation in very challenging, densely vegetated environments.

Due to the precise manipulations required in dental surgery, robotic technologies have been applied to dentistry. So far, most dental robots are designed for implant surgery, helping dentists accurately place the implant to the desired position and depth. This paper presents the DentiBot, the first robot designed for dental endodontic treatment. Without visual feedback, the DentiBot is integrated with a force and torque sensor to monitor the contact between the root canal and endodontic file. Additionally, DentiBot is implemented with force-guided alignment and file feedrate control to autonomously adjust surgical path and compensate for patient movement in real-time while protecting against endodontic file fracture. The feasibility of robot-assisted endodontic treatment is verified by the pre-clinical evaluation performed on acrylic root canal models.

Tele-impedance increases interaction performance between a robotic tool and unstructured/unpredictable en-vironments during teleoperation. However, the existing tele-impedance interfaces have several ongoing issues, such as long calibration times and various obstructions for the human operator. In addition, they are all designed to be controlled by the operator's arms, which can cause difficulties when both arms are used, as in bi-manual teleoperation. To resolve these issues, we designed a novel foot-based tele-impedance control method inspired by the human limb stiffness ellipse modulation. The proposed mechanical interface design includes a disc and a foot pressure sensor that controls the orientation and size/shape of the stiffness ellipse, respectively. We evaluated the disc interface control method in an experimental study with 12 participants, who performed a complex drilling task in a virtual environment. The results show the ability of the operator to use the proposed interface in order to dynamically adapt to different phases of the task and changes in the environment. In addition, a comparison with low and high uniform impedance modes demonstrates a superior interaction performance of the proposed method.

From ancient times, humans have been using cables and ropes to tie, carry, and manipulate objects by folding knots. However, automating knot folding is challenging because it requires dexterity to move a cable over and under itself. In this paper, we propose a method to fold knots in midair using a team of aerial vehicles. We take advantage of the fact that vehicles are able to fly in between cable segments without any re-grasping. So the team grasps the cable from the floor, and releases it once the knot is folded. Based on a composition of catenary curves, we simplify the complexity of dealing with an infinite-dimensional configuration space of the cable, and formally propose a new knot representation. Such representation allows us to design a trajectory that can be used to fold knots using a leader-follower approach. We show that our method works for different types of knots in simulations. Additionally, we show that our solution is also computationally efficient and can be executed in real-time.