IROS 2022 Papers

Micro aerial vehicles usually have strap-down sensors on the vehicle body, leading to the severe coupling effect between perception and trajectory planning. As a result, visual-inertial simultaneous localization and mapping (VI-SLAM) technologies implemented on MAVs suffer from tracking failure problems, especially in featureless environments. To overcome these challenges, based on MAVs with movable camera mechanisms (e.g., gimbal stabilizer, pan-tilt, or bionic neck-eye system), we proposed two sampling-based algorithms for known and unknown environments respectively. The first active perception planning algorithm based on a scene richness model is developed with a built feature map for the environment. Differ from the first algorithm, the second one is modified for active localization in unknown 3D space. It is basically a time-based sampling-based approach that uses the same scene richness model. In addition, it also achieved a balance between exploitation and exploration. With the above solutions, the robustness of visual perception is improved while avoiding over-exploitation of known information. Simulation and real-world experiments are performed to verify the feasibility of our algorithms.

Robots with constrained hardware resources usually rely on Multi-access Edge Computing infrastructures to offload computationally expensive tasks to meet real-time and safety requirements. Offloading every task might not be the best option due to dynamic changes in the network conditions and can result in network congestion or failures. This work proposes a task offloading strategy for mobile robots in a Human-Robot Collaboration scenario that optimizes the edge resource usage and reduces network delays, leading to safety enhancement. The solution utilizes a Deep Reinforcement Learning (DRL) agent that observes safety and network metrics to dynamically decide at runtime if (i) a less accurate model should run on the robot; (ii) a more complex model should run on the edge; or (iii) the previous output should be reused through temporal coherence verification. Experiments are performed in a simulated warehouse where humans and robots have close interactions and safety needs are high. Our results show that the proposed DRL solution outperforms the baselines in several aspects. The edge is used only when the network performance is reliable, reducing the number of failures (up to 47 %). The latency is not only decreased (up to 68 %) but also adapted to the safety requirements (risk × latency reduced up to 48 %), avoiding unnecessary network congestion in safe situations and letting other devices use the network. Overall, the safety metrics get improved, such as the increased time in the safe zone by up to 3.1%.

In reinforcement learning (RL), exploration is essential to achieve a globally optimal policy but unconstrained exploration can cause damages to robots and nearby people. To handle this safety issue in exploration, safe RL has been proposed to keep the agent under the specified safety constraints while maximizing cumulative rewards. This paper introduces a new safe RL method which can be applied to robots to operate under the safety constraints while learning. The key component of the proposed method is the safeguard module. The safeguard predicts the constraints in the near future and corrects actions such that the predicted constraints are not violated. Since actions are safely modified by the safeguard during exploration and policies are trained to imitate the corrected actions, the agent can safely explore. Additionally, the safeguard is sample efficient as it does not require long horizontal trajectories for training, so constraints can be satisfied within short time steps. The proposed method is extensively evaluated in simulation and experiments using a real robot. The results show that the proposed method achieves the best performance while satisfying safety constraints with minimal interaction with environments in all experiments.

Learning a risk-aware policy is essential but rather challenging in unstructured robotic tasks. Safe reinforcement learning methods open up new possibilities to tackle this problem. However, the conservative policy updates make it intractable to achieve sufficient exploration and desirable performance in complex, sample-expensive environments. In this paper, we propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. Concretely, the baseline agent is responsible for maximizing rewards under standard RL settings. Thus, it is compatible with off-the-shelf training techniques of unconstrained optimization, exploration and exploitation. On the other hand, the safe agent mimics the baseline agent for policy improvement and learns to fulfill safety constraints via off-policy RL tuning. In contrast to training from scratch, safe policy correction requires significantly fewer interactions to obtain a near-optimal policy. The dual policies can be optimized synchronously via a shared replay buffer, or leveraging the pre-trained model or the non-learning-based controller as a fixed baseline agent. Experimental results show that our approach can learn feasible skills without prior knowledge as well as deriving risk-averse counterparts from pre-trained unsafe policies. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks with respect to both safety constraint satisfaction and sample efficiency.

Satisfying safety constraints is the top priority in safe reinforcement learning (RL). However, without proper exploration, an overly conservative policy such as freezing at the same position can be generated. To this end, we utilize maximum entropy RL methods for exploration. In particular, an RL method with Tsallis entropy maximization, called Tsallis actor-critic (TAC), is used to synthesize policies which can explore with more promising actions. In this paper, we propose a Tsallis entropy-regularized safe RL method for safer exploration, called SafeTAC. For more expressiveness, we extend the TAC to use a Gaussian mixture model policy, which improves the safety performance. To stabilize the training process, the retrace estimators for safety critics are formulated, and a safe policy update rule using a trust region method is proposed.

This paper presents an application of human-drone interaction (HDI) for inventory management in a ware-house 4.0 that aims at improving the operators' safety and well-being together with increasing efficiency and reducing production costs. In our work, the speed and separation monitoring (SSM) methodology is applied for the first time to HDI, in analogy to the human-robot interaction (HRI) ISO safety requirements as well as the rapid upper limb assessment (RULA), for evaluating the operator's ergonomic posture during the interaction with the drone. With the aim of validating the proposed approach in a realistic scenario, a quadrotor is controlled to perform a pick and place task along a desired trajectory, from the picking bay to the palletizing area where the operator is located, avoiding collisions with the warehouse shelves by implementing the artificial potential field technique (APF) for planning and the linear quadratic regulator (LQR) and iterative LQR (iLQR) algorithms for tracking. The obtained results of the HDI architecture simulations are presented and discussed in detail proving the effectiveness of the proposed method for a safe and ergonomic HDI.

Many collaborative human-robot tasks require the robot to stay safe and work efficiently around humans. Since the robot can only stay safe with respect to its own model of the human, we want the robot to learn a good model of the human in order to act both safely and efficiently. This paper studies methods that enable a robot to safely explore the space of a human-robot system to improve the robot's model of the human, which will consequently allow the robot to access a larger state space and better work with the human. In particular, we introduce active exploration under the framework of energy-function based safe control, investigate the effect of different active exploration strategies, and finally analyze the effect of safe active exploration on both analytical and neural network human models.

Achieving the capability of adapting to ever-changing environments is a critical step towards building fully autonomous robots that operate safely in complicated scenarios. In multiagent competitive scenarios, agents may have to adapt to new opponents with previously unseen behaviors by learning from the interaction experiences between the ego-agent and the opponent. However, this adaptation is susceptible to opponent exploitation. As the ego-agent updates its own behavior to exploit the opponent, its own behavior could become more exploitable as a result of overfitting to this specific opponent's behavior. To overcome this difficulty, we developed a safe adaptation approach in which the ego-agent is trained against a regularized opponent model, which effectively avoids overfitting and consequently improves the robustness of the ego-agent's policy. We evaluated our approach in the Mujoco domain with two competing agents. The experiment results suggest that our approach effectively achieves both adaptation to the specific opponent that the ego-agent is interacting with and maintaining low exploitability to other possible opponent exploitation.

Designing control policies for legged locomotion11In this work, we specifically consider quadruped locomotion. is complex due to the under-actuated and non-continuous robot dynamics. Model-free reinforcement learning provides promising tools to tackle this challenge. However, a major bottleneck of applying model-free reinforcement learning in real world is safety. In this paper, we propose a safe reinforcement learning framework that switches between a safe recovery policy that prevents the robot from entering unsafe states, and a learner policy that is optimized to complete the task. The safe recovery policy takes over the control when the learner policy violates safety constraints, and hands over the control back when there are no future safety violations. We design the safe recovery policy so that it ensures safety of quadruped locomotion while minimally intervening in the learning process. Furthermore, we theoretically analyze the proposed framework and provide an upper bound on the task performance. We verify the proposed framework in four tasks on a simulated and real quadrupedal robot: efficient gait, catwalk, two-leg balance, and pacing. On average, our method achieves 48.6% fewer falls and comparable or better rewards than the baseline methods in simulation. When deployed it on real-world quadruped robot, our training pipeline enables 34% improvement in energy efficiency for the efficient gait, 40.9% narrower of the feet placement in the catwalk, and two times more jumping duration in the two-leg balance. Our method achieves less than five falls over the duration of 115 minutes of hardware time.22Video is included in the submission and the project website: https://sites.google.com/view/saferlleggedlocomotion/

The weight, space, and power limitations of small aerial vehicles often prevent the application of modern control techniques without significant model simplifications. Moreover, high-speed agile behavior, such as that exhibited in drone racing, make these simplified models too unreliable for safety-critical control. In this work, we introduce the concept of time-varying backup controllers (TBCs): user-specified maneuvers combined with backup controllers that generate reference trajectories which guarantee the safety of nonlinear systems. TBCs reduce conservatism when compared to traditional backup controllers and can be directly applied to multi-agent coordination to guarantee safety. Theoretically, we provide conditions under which TBCs strictly reduce conservatism, describe how to switch between several TBC's and show how to embed TBCs in a multi-agent setting. Experimentally, we verify that TBCs safely increase operational freedom when filtering a pilot's actions and demonstrate robustness and computational efficiency when applied to decentralized safety filtering of two quadrotors.

This paper introduces a stereo thermal camera dataset (STheReO) with multiple navigation sensors to encourage thermal SLAM researches. A thermal camera measures infrared rays beyond the visible spectrum therefore it could provide a simple yet robust solution to visually degraded environments where existing visual sensor-based SLAM would fail. Existing thermal camera datasets mostly focused on monocular configuration using the thermal camera with RGB cameras in a visually challenging environment. A few stereo thermal rig were examined but in computer vision perspective without supporting sequential images for state estimation algorithms. To encourage the academia for the evolving stereo thermal SLAM, we obtain nine sequences in total across three spatial locations and three different times per location (e.g., morning, day, and night) to capture the variety of thermal characteristics. By using the STheReO dataset, we hope diverse types of researches will be made, including but not limited to odometry, mapping, and SLAM (e.g., thermal-LiDAR mapping or long-term thermal localization). Our datasets are available at https://sites.google.com/view/rpmsthereo/.

Place recognition is key to Simultaneous Localization and Mapping (SLAM) and spatial perception. However, a place recognition in the wild often suffers from erroneous predictions due to image variations, e.g., changing viewpoints and street appearance. Integrating uncertainty estimation into the life cycle of place recognition is a promising method to mitigate the impact of variations on place recognition performance. However, existing uncertainty estimation approaches in this vein are either computationally inefficient (e.g., Monte Carlo dropout) or at the cost of dropped accuracy. This paper proposes STUN, a self-teaching framework that learns to simultaneously predict the place and estimate the prediction uncertainty given an input image. To this end, we first train a teacher net using a standard metric learning pipeline to produce embedding priors. Then, supervised by the pretrained teacher net, a student net with an additional variance branch is trained to finetune the embedding priors and estimate the uncertainty sample by sample. During the online inference phase, we only use the student net to generate a place prediction in conjunction with the uncertainty. When compared with place recognition systems that are ignorant of the uncertainty, our framework features the uncertainty estimation for free without sacrificing any prediction accuracy. Our experimental results on the large-scale Pittsburgh30k dataset demonstrate that STUN outperforms the state-of-the-art methods in both recognition accuracy and the quality of uncertainty estimation.

Accurate kinodynamic models play a crucial role in many robotics applications such as off-road navigation and high-speed driving. Many state-of-the-art approaches for learning stochastic kinodynamic models, however, require precise measurements of robot states as labeled input/output examples, which can be hard to obtain in outdoor settings due to limited sensor capabilities and the absence of ground truth. In this work, we propose a new technique for learning neural stochastic kinodynamic models from noisy and indirect observations by performing simultaneous state estimation and dynamics learning. The proposed technique iteratively improves the kinodynamic model in an expectation-maximization loop, where the E Step samples posterior state trajectories using particle filtering, and the M Step updates the dynamics to be more consistent with the sampled trajectories via stochastic gradient ascent. We evaluate our approach on both simulation and real-world benchmarks and compare it with several baseline techniques. Our approach not only achieves significantly higher accuracy but is also more robust to observation noise, thereby showing promise for boosting the performance of many other robotics applications.

Category-level pose estimation is a challenging problem due to intra-class shape variations. Recent methods deform pre-computed shape priors to map the observed point cloud into the normalized object coordinate space and then retrieve the pose via post-processing, i.e., Umeyama's Algorithm. The shortcomings of this two-stage strategy lie in two aspects: 1) The surrogate supervision on the intermediate results can not directly guide the learning of pose, resulting in large pose error after post-processing. 2) The inference speed is limited by the post-processing step. In this paper, to handle these shortcomings, we propose an end-to-end trainable network SSP-Pose for category-level pose estimation, which integrates shape priors into a direct pose regression network. SSP-Pose stacks four individual branches on a shared feature extractor, where two branches are designed to deform and match the prior model with the observed instance, and the other two branches are applied for directly regressing the totally 9 degrees-of-freedom pose and performing symmetry reconstruction and point-wise inlier mask prediction respectively. Consistency loss terms are then naturally exploited to align the outputs of different branches and promote the performance. During inference, only the direct pose regression branch is needed. In this manner, SSP-Pose not only learns category-level pose-sensitive characteristics to boost performance but also keeps a real-time inference speed. Moreover, we utilize the symmetry information of each category to guide the shape prior deformation, and propose a novel symmetry-aware loss to mitigate the matching ambiguity. Extensive experiments on public datasets demon-strate that SSP-Pose produces superior performance compared with competitors with a real-time inference speed at about 25Hz. The codes will be released soon.

ROS 2 is rapidly becoming a standard in the robotics industry. Built upon DDS as its default communication middleware and used in safety-critical scenarios, adding secu-rity to robots and ROS computational graphs is increasingly becoming a concern. The present work introduces SROS2, a series of developer tools and libraries that facilitate adding security to ROS 2 graphs. Focusing on a usability-centric approach in SROS2, we present a methodology for securing graphs systematically while following the DevSecOps model. We also demonstrate the use of our security tools by presenting an application case study that considers securing a graph using the popular Navigation2 and SLAM Toolbox stacks applied in a TurtieBot3 robot. We analyse the current capabilities of SROS2 and discuss the shortcomings, which provides insights for future contributions and extensions. Ultimately, we present SROS2 as usable security tools for ROS 2 and argue that without usability, security in robotics will be greatly impaired.

Existing work in physical robot caregiving is limited in its ability to provide long-term assistance. This is majorly due to (i) lack of well-defined problems, (ii) diversity of tasks, and (iii) limited access to stakeholders from the caregiving community. We propose Structuring Physically Assistive Robotics for Caregiving with Stakeholders-in-the-loop (SPARCS) to address these challenges. SPARCS is a framework for physical robot caregiving comprising (i) Building Blocks, models that define physical robot caregiving scenarios, (ii) Structured Workflows, hierarchical workflows that enable us to answer the Whats and Hows of physical robot caregiving, and (iii) SPARCS-box, a web-based platform to facilitate dialogue between all stakeholders. We collect clinical data for six care recipients with varying disabilities and demonstrate the use of SPARCS in designing well-defined caregiving scenarios and identifying their care requirements. All the data and workflows are available on SPARCS-box. We demonstrate the utility of SPARCS in building a robot-assisted feeding system for one of the care recipients. We also perform experiments to show the adaptability of this system to different caregiving scenarios. Finally, we identify open challenges in physical robot caregiving by consulting care recipients and caregivers. Supplementary material can be found at emprise.cs.cornell.edu/sparcs.

Global localization is essential for autonomous mobile systems, especially indoor applications where the GPS signal is denied. Although the appearance-based methods have been successfully applied in various localization tasks, they face various challenges such as light variation, viewpoint changing, and dynamic interference. Additionally, the appearance-based methods usually require a visual feature point map, which increases the storage burden. This paper proposes a novel global localization solution that leverages sparse and repetitive semantic object information. The proposal can fulfill global localization based on object-level maps that are self-built or externally provided. In this solution, the semantic objects are firstly modeled with a point cloud. Then, the object's semantic information is embedded into the geometry of the corresponding point, and the Semantic Object-based Point Feature Histogram (SO-PFH) descriptors of the modeled point clouds are estimated. Finally, the global localization is executed by applying a Geometric Consistency Filter-based RANdom SAmple Consensus (GCF-RANSAC) method to match point clouds. Experiments and simulations are conducted in indoor parking lots. The results demonstrate the effectiveness of the proposed method.

Model predictive control is a promising method in robot control tasks. How to design an effective model structure and efficient prediction framework for model predictive control is still an open challenge. To reduce the time consumption and avoid compounding-error of the multi-step prediction process in model predictive control, we propose a single-model simultaneous framework, which uses single dynamics model to predict the entire prediction horizon simultaneously by taking all control actions with the current state as inputs. Based on this framework, we further propose an adversarial dynamics model that contains two parts. The generator provides a dynamics model for the prediction process, while the discriminator provides constraints that are hard to describe by manually defined loss. This adversarial dynamics model can accelerate training and improve model accuracy in unstructured environments. Experiments conducted in Gazebo simulator and on a real mobile robot demonstrate the efficiency and accuracy of the single-model simultaneous framework with an adversarial dynamics model.

In target tracking with mobile multi-sensor sys-tems, sensor deployment impacts the observation capabilities and the resulting state estimation quality. Based on a partially observable Markov decision process (POMDP) formulation comprised of the observable sensor dynamics, unobservable target states, and accompanying observation laws, we present a distributed information-driven solution approach to the multi-agent target tracking problem, namely, sequential multi-agent nominal belief-state optimization (SMA-NBO). SMA-NBO seeks to minimize the expected tracking error via receding horizon control including a heuristic expected cost-to-go (HECTG). SMA-NBO incorporates a computationally efficient approximation of the target belief-state over the horizon. The agent-by-agent decision-making is capable of leveraging on-board (edge) compute for selecting (sub-optimal) target-tracking maneuvers exhibiting non-myopic cooperative fleet behavior. The opti-mization problem explicitly incorporates semantic information defining target occlusions from a world model. To illustrate the efficacy of our approach, a random occlusion forest environment is simulated. SMA-NBO is compared to other baseline approaches. The simulation results show SMA-NBO 1) maintains tracking performance and reduces the computational cost by replacing the calculation of the expected target trajectory with a single sample trajectory based on maximum a posteriori estimation; 2) generates cooperative fleet decision by sequentially optimizing single-agent policy with efficient usage of other agents' policy of intent; 3) aptly incorporates the multiple weighted trace penalty (MWTP) HECTG, which improves tracking performance with a computationally efficient heuristic.