IROS 2022 Papers

Recent advances in deep reinforcement learning and scalable photorealistic simulation have led to increasingly mature embodied AI for various visual tasks, including navigation. However, while impressive progress has been made for teaching embodied agents to navigate static environments, much less progress has been made on more dynamic environments that may include moving pedestrians or movable obstacles. In this study, we aim to benchmark different augmentation techniques for improving the agent's performance in these challenging environments. We show that adding several dynamic obstacles into the scene during training confers significant improvements in test-time generalization, achieving much higher success rates than baseline agents. We find that this approach can also be combined with image augmentation methods to achieve even higher success rates. Additionally, we show that this approach is also more robust to sim-to-sim transfer than image augmentation methods. Finally, we demonstrate the effectiveness of this dynamic obstacle augmentation approach by using it to train an agent for the 2021 iGibson Challenge at CVPR, where it achieved 1st place for Interactive Navigation.

We propose an approach that enables simultaneous interpretable learning of a high-level discrete behaviour and its low-level rhythmic sub-behaviour. We do this though a unified reward function, where a reward function that only describes low-level behaviour, with less impact on learning of other behaviours is recovered from few-shot motion demonstrations. To this end, we first extract local behaviour motifs from state-only human demonstrations and random driving samples using an adaptive motif discovery approach derived from the Matrix Profile algorithm. We then optimize parameters for motif discovery by maximizing the sum and entropy over motif sizes. Interacting Multiple Model (IMM) estimators are constructed on top of linear-Gaussian dynamics of discovered motifs, the cumulative distributions over motifs estimated by IMMs serve as the basis of the reward function. By combining the recovered reward with the terrain type signal gathered from the environment, we are able to train a dual-objective off-road vehicle controller that demonstrates both terrain selection and human-like driving behaviours. Compared with related approaches across 10 people, our rhythmic behaviour reward recovery approach enables the controller to produce higher preference over human driving demonstrations. In addition to performing more stable across different people with 87% less variance than the best baseline in rhythmic behaviour indicator, our method reduces the negative effects on higher-level behaviour learning while maintaining high interpretability at all stages of the algorithm.

Recently, the behavior tree is gaining popularity as a robotic task-level knowledge representation. Manual design of behavior trees from scratch is tedious and cumbersome. Motivated by the need for an efficient way to reuse or transfer robot task-level knowledge, we propose a vector-space embedding approach that encodes a symbolic task into a numerical form. This approach, called behavior-tree embedding, takes a behavior tree that produces a single task as input and generates a corresponding vector. By exploiting the pretrained language-embedding model and the node-aggregation mechanism, the produced embedding is capable of preserving both semantic information of task description and structural information of the hierarchical task organization. We evaluated the effectiveness and versatility of our proposed vector-space embedding approach in three different tasks.

While learning from synthetic training data has recently gained an increased attention, in real-world robotic applications, there are still performance deficiencies due to the so-called Sim-to-Real gap. In practice, this gap is hard to resolve with only synthetic data. Therefore, we focus on an efficient acquisition of real data within a Sim-to-Real learning pipeline. Concretely, we employ deep Bayesian active learning to minimize manual annotation efforts and devise an autonomous learning paradigm to select the data that is considered useful for the human expert to annotate. To achieve this, a Bayesian Neural Network (BNN) object detector providing reliable un-certainty estimates is adapted to infer the informativeness of the unlabeled data. Furthermore, to cope with misalignments of the label distribution in uncertainty-based sampling, we develop an effective randomized sampling strategy that performs favorably compared to other complex alternatives. In our experiments on object classification and detection, we show benefits of our approach and provide evidence that labeling efforts can be reduced significantly. Finally, we demonstrate the practical effectiveness of this idea in a grasping task on an assistive robot.

This paper proposes a novel approach to controlling wheel-legged quadrupedal robots using pose optimization and force-based control via quadratic programming (QP). Our method allows the robot to leverage the whole-body motion and the wheel actuation to roll over high obstacles while keeping wheel traction with the terrain. In detail, we first present linear rigid body dynamics with wheels that can be used for real-time balancing control of wheel-legged robots. We then introduce an effective pose optimization method for wheel-legged robot's locomotion over steep ramp and stair terrains. The pose optimization solves for optimal poses to enhance stability and enforce collision-free constraints at critical pose locations for rolling over high obstacles. Experimental validation of the real robot demonstrated the capability of rolling up on a 0.36 m obstacle. The robot can also successfully roll up and down multiple stairs without lifting its legs or colliding with the terrain.

Traditional model-based reinforcement learning (RL) methods generate forward rollout traces using the learnt dynamics model to reduce interactions with the real environment. The recent model-based RL method considers the way to learn a backward model that specifies the conditional probability of the previous state given the previous action and the current state to additionally generate backward rollout trajectories. However, in this type of model-based method, the samples derived from backward rollouts and those from forward rollouts are simply aggregated together to optimize the policy via the model-free RL algorithm, which may decrease both the sample efficiency and the convergence rate. This is because such an approach ignores the fact that backward rollout traces are often generated starting from some high-value states and are certainly more instructive for the agent to improve the behavior. In this paper, we propose the backward imitation and forward reinforcement learning (BIFRL) framework where the agent treats backward rollout traces as expert demonstrations for the imitation of excellent behaviors, and then collects forward rollout transitions for policy reinforcement. Consequently, BIFRL empowers the agent to both reach to and explore from high-value states in a more efficient manner, and further reduces the real interactions, making it potentially more suitable for real-robot learning. Moreover, a value-regularized generative adversarial network is introduced to augment the valuable states which are infrequently received by the agent. Theoretically, we provide the condition where BIFRL is superior to the baseline methods. Experimentally, we demonstrate that BIFRL acquires the better sample efficiency and produces the competitive asymptotic performance on various MuJoCo locomotion tasks compared against state-of-the-art model-based methods.

Learning from previously collected datasets of expert data offers the promise of acquiring robotic policies without unsafe and costly online explorations. However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time. While prior works mainly studied the distribution shift caused by the policy during the offline training, the problem of recovering from out-of-distribution states at the deployment time is not very well studied yet. We alleviate the distributional shift at the deployment time by introducing a recovery policy that brings the agent back to the training manifold whenever it steps out of the in-distribution states, e.g., due to an external perturbation. The recovery policy relies on an approximation of the training data density and a learned equivariant mapping that maps visual observations into a latent space in which translations correspond to the robot actions. We demonstrate the effectiveness of the proposed method through several manipulation experiments on a real robotic platform. Our results show that the recovery policy enables the agent to complete tasks while the behavioral cloning alone fails because of the distributional shift problem.

Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.

In this paper we present BOEM-SLAM, a backend for visual-inertial SLAM systems capable of creating a globally consistent trajectory and map without retaining the entire history of data. By leveraging the hidden Markov model structure, BOEM-SLAM can summarize historical data into sufficient statistics and then discard it. As a data-efficient algorithm, BOEM-SLAM addresses the growing computational costs and storage requirements of the SLAM backend. To demonstrate the performance of our algorithm we compare BOEM-SLAM to other fundamental approaches on both synthetic data and the EuRoC dataset. For evaluation on the EuRoC dataset, we use the open source okvis frontend and apply the Lie group state space representation and visual outlier removal. Overall, BOEM-SLAM shows a considerably lower computation time with comparable estimation performance. For example, the processing time of BOEM-SLAM is 50 times smaller than the optimization-based method using simulated data and 5 times smaller than the optimization-based method in the EuRoC dataset experiments.

We present our framework Behaviors, Objectives and Binary states for Coordinated Autonomous Tasks (BOBCAT), a multi-agent decision making and task management system for autonomous robots. BOBCAT builds on behavior-based systems and the Belief-Desire-Intention model for decision-making, by using mission objectives as the basis for selecting tasks, or behaviors. We describe how BOBCAT is formulated and present the results of a real-world implementation in the context of exploring austere underground environments in the DARPA Subterranean Challenge.

Optimal sampling based motion planning and trajectory optimization are two competing frameworks to generate optimal motion plans. Both frameworks have complementary properties: Sampling based planners are typically slow to converge, but provide optimality guarantees. Trajectory optimizers, however, are typically fast to converge, but do not provide global optimality guarantees in nonconvex problems, e.g. scenarios with obstacles. To achieve the best of both worlds, we introduce a new planner, BITKOMO, which integrates the asymptotically optimal Batch Informed Trees (BIT*) planner with the K-Order Markov Optimization (KOMO) trajectory optimization framework. Our planner is anytime and maintains the same asymptotic optimality guarantees provided by BIT*, while also exploiting the fast convergence of the KOMO trajectory optimizer. We experimentally evaluate our planner on manipulation scenarios that involve high dimensional configuration spaces, with up to two 7-DoF manipulators, obstacles and narrow passages. BITKOMO performs better than KOMO by succeeding even when KOMO fails, and it outperforms BIT* in terms of convergence to the optimal solution.

Sample efficiency has been a key issue in reinforcement learning (RL). An efficient agent must be able to leverage its prior experiences to quickly adapt to similar, but new tasks and situations. Meta-RL is one attempt at formalizing and ad-dressing this issue. Inspired by recent progress in meta-RL, we introduce BIMRL, a novel multi-layer architecture along with a novel brain-inspired memory module that will help agents quickly adapt to new tasks within a few episodes. We also utilize this memory module to design a novel intrinsic reward that will guide the agent's exploration. Our architecture is inspired by findings in cognitive neuroscience and is compatible with the knowledge on connectivity and functionality of different regions in the brain. We empirically validate the effectiveness of our proposed method by competing with or surpassing the performance of some strong baselines on multiple MiniGrid environments.

The ability to produce large-scale maps for nav-igation, path planning and other tasks is a crucial step for autonomous agents, but has always been challenging. In this work, we introduce BEV-SLAM, a novel type of graph-based SLAM that aligns semantically-segmented Bird's Eye View (BEV) predictions from monocular cameras. We introduce a novel form of occlusion reasoning into BEV estimation and demonstrate its importance to aid spatial aggregation of BEV predictions. The result is a versatile SLAM system that can operate across arbitrary multi-camera configurations and can be seamlessly integrated with other sensors. We show that the use of multiple cameras significantly increases performance, and achieves lower relative error than high-performance GPS. The resulting system is able to create large, dense, globally-consistent world maps from monocular cameras mounted around an ego vehicle. The maps are metric and correctly-scaled, making them suitable for downstream navigation tasks.

We present a real-time motion planner that avoids multiple moving obstacles without knowing their dynamics or intentions. This method uses convex optimization to generate trajectories for linear plant models over a planning horizon (i.e. model-predictive control). While convex optimizations allow for fast planning, obstacle avoidance can be challenging to incorporate because Euclidean distance calculations tend to break convexity. By using a half-space convex relaxation, our planner reasons about an approximated distance-to-obstacle measure that is linear in its decision variables and preserves convexity. Further, by iteratively updating the relaxation over the planning horizon, the half-space approximation is improved, enabling nimble avoidance maneuvers. We further augment avoidance performance with a soft penalty slack-variable for-mulation that introduces a piecewise quadratic cost. As a proof of concept, we demonstrate the planner on double-integrator models in both single-agent and multi-agent tasks-avoiding multiple obstacles and other agents in 2D and 3D environments. We show extensions to legged locomotion by bipedally walking around obstacles in simulation using the Linear Inverted Pendulum Model (LIPM). We then present two sets of hardware experiments showing real-time obstacle avoid-ance with quadcopter drones: (1) avoiding a 10m/s swinging pendulum and (2) dodging a chasing drone.

This paper presents an autonomous approach for landing a quadrotor on inclined surfaces up to 40 degrees using radar perception in a high particle environment, such as dust, rain, or fog. This system uses five radar sensors to determine the direction, angle, and smoothness of a slope through eigenvalue decomposition of a point cloud covariance matrix. The point cloud itself is generated using a FIFO queue with the radar sensors after their points are transformed to a common frame. Then, two asymmetric landing skids of different lengths actively conform to a slope in order to maintain level body attitude upon landing. For perception error tolerance, a study to understand the distance between the propeller and slope surface with respect to slope angles was developed. We evaluate the accuracy and consistency of radar sensors in accomplishing these tasks, to include a comparison of the results with a depth camera while in a high particle environment. Finally, the experimental result shows that the detected slope angle and direction were within 2.2 and 2.4 degrees of ground, and the proposed system is viable and robust for use in real-world applications.

Inspection of subsea pipelines is crucial for avoiding any hazards and minimizing the risks to infrastructure and the environment. These inspections are achieved using Autonomous Underwater Vehicles (AUVs) in favour of reduced operational costs. This work presents a vehicle agnostic approach for tracking subsea pipelines at close-range for autonomous guidance along the pipeline using an AUV. A multibeam echosounder is used as the primary tracking sensor augmented by fluxgate magnetometers that can track buried pipelines over short ranges until they are exposed again. A Bernoulli filter is proposed to efficiently track pipelines in presence of environmental clutter. Field experiments were carried out in a dock and at sea to evaluate the proposed system and the benefits of using a Bernoulli filter over a Kalman filter-based solution.

Mobile 3D Printing (M3DP), using printing-in-motion, is a powerful paradigm for automated construction. A mobile robot, equipped with its own power, materials and an arm-mounted extruder, simultaneously navigates and creates its environment. Such systems can be highly scalable, parallelizable and flexible. However, planning and controlling the motion of the arm and base at the same time is challenging and most deployments either avoid robot-base motion entirely or use human prescribed robot-base paths. In a previous paper, we developed a high-level planning algorithm to automate M3DP given a print task. The generated robot-base paths avoid collisions and maintain task reachability. In this paper, we extend this work to robot control. We develop and compare three different ways to integrate the long-duration planned path with a short horizon Model Predictive Controller. Experiments are carried out via a new M3DP system - Armstone. We evaluate and demonstrate our algorithm in a 250 m long multi-layer print which is about 5 times longer than any previous physical printing-in-motion system.

Navigation inside luminal organs is an arduous task that requires non-intuitive coordination between the movement of the operator's hand and the information obtained from the endoscopic video. The development of tools to automate certain tasks could alleviate the physical and mental load of doctors during interventions allowing them to focus on diagnosis and decision-making tasks. In this paper we present a synergic solution for intraluminal navigation consisting of a 3D printed endoscopic soft robot that can move safely inside luminal structures. Visual servoing based on Convolutional Neural Networks (CNNs) is used to achieve the autonomous navigation task. The CNN is trained with phantoms and in-vivo data to segment the lumen and a model-less approach is presented to control the movement in constrained environments. The proposed robot is validated in anatomical phantoms in different path configurations. We analyze the movement of the robot using different metrics such as task completion time smoothness error in the steady-state mean and maximum error. We show that our method is suitable to navigate safely in hollow environments and conditions which are different than the ones the network was originally trained on.