ICRA 2022 Papers

This paper presents a data-driven multiple model framework for estimating the intention of a target from observations. Multiple model (MM) state estimation methods have been extensively used for intention estimation by mapping one intention to one dynamic model assuming one-to-one relations. However, intentions are subjective to humans and it is difficult to establish the one-to-one relations explicitly. The proposed framework infers the multiple-to-multiple relations between intentions and models directly from observations that are labeled with intentions. For intention estimation, both the relations and model probabilities of an Interacting Multiple Model (IMM) state estimation approach are integrated into a recursive Bayesian framework. Taking advantage of the inferred multiple-to-multiple relations, the framework incorpo-rates more accurate relations and avoids following the strict one-to-one relations. Numerical and real experiments were performed to investigate the framework through the intention estimation of a maneuvered quadrotor. Results show higher estimation accuracy and superior flexibility in designing mod-els over the conventional approach that assumes one-to-one relations.

Robots are typically designed as occupants of rooms, adapting to, and navigating within them. “Robot surfaces,” an emerging robot typology, are not occupants of but integral with rooms, physically shaping rooms to support human activity. We report on an advancement of robot surfaces formed by weaving McKibben Pneumatic Air Muscles that, when actuated, morph a 2D planar surface to generate 3D geometries including a “spherical cap.” Following our foundational study at different scales with different materials, we developed a full-scale prototype that offers an intimate and private space for people meeting in open plan environments. We report on our research, focusing on a design case, and validate the full-scale prototype as compared to our Non-Uniform Rational B-Splines (NURBS) model for three useful configurations. Our quantitative and qualitative results suggest that our robot surface can support human activity as envisioned. This research contributes foundational understanding of an emerging category of robotics from which our team and peers can build.

Human Motion Prediction (HMP) plays a crucial role in safe Human-Robot-Interaction (HRI). Currently, the majority of HMP algorithms are trained by massive pre-collected data. As the training data only contains a few pre-defined motion patterns, these methods cannot handle the unfamiliar motion patterns. Moreover, the pre-collected data are usually non-interactive, which does not consider the real-time responses of collaborators. As a result, these methods usually perform unsatisfactorily in real HRI scenarios. To solve this problem, in this paper, we propose a novel Continual Learning (CL) approach for probabilistic HMP which makes the robot continually learns during its interaction with collaborators. The proposed approach consists of two steps. First, we leverage a Bayesian Neural Network to model diverse uncertainties of observed human motions for collecting online interactive data safely. Then we take Experience Replay and Knowledge Distillation to elevate the model with new experiences while maintaining the knowledge learned before. We first evaluate our approach on a large-scale benchmark dataset Human3.6m. The experimental results show that our approach achieves a lower prediction error compared with the baselines methods. Moreover, our approach could continually learn new motion patterns without forgetting the learned knowledge. We further conduct real-scene experiments using Kinect DK. The results show that our approach can learn the human kinematic model from scratch, which effectively secures the interaction.

In this paper, we consider the European Rail Traffic Management System (ERTMS) as a System-of-Systems (SoS) and propose modeling it using colored Petri nets. We formally control the European rail transport, while guaranteeing a set of cross-border security properties. This becomes an essential and challenging task since each of them have mainly developed safety and trackside rules regardless of its neighbors. The feature of this work lies in the approach that considers ERTMS Level 2 as an SoS and addresses the cross-border railway as a mode management problem. In addition, the aspects of mode activation/deactivation, starting state and handling of resource states common to multiple operating modes are taken into account in the proposed model.

In this paper, we present a real-time whole-body planner for collision-free legged mobile manipulation. We enforce both self-collision and environment-collision avoidance as soft constraints within a Model Predictive Control (MPC) scheme that solves a multi-contact optimal control problem. By penalizing the signed distances among a set of representative primitive collision bodies, the robot is able to safely execute a variety of dynamic maneuvers while preventing any self-collisions. Moreover, collision-free navigation and manipulation in both static and dynamic environments are made viable through efficient queries of distances and their gradients via a euclidean signed distance field. We demonstrate through a comparative study that our approach only slightly increases the computational complexity of the MPC planning. Finally, we validate the effectiveness of our framework through a set of hardware experiments involving dynamic mobile manipulation tasks with potential collisions, such as locomotion balancing with the swinging arm, weight throwing, and autonomous door opening.

Quadrotors are highly nonlinear dynamical systems that require carefully tuned controllers to be pushed to their physical limits. Recently, learning-based control policies have been proposed for quadrotors, as they would potentially allow learning direct mappings from high-dimensional raw sensory observations to actions. Due to sample inefficiency, training such learned controllers on the real platform is impractical or even impossible. Training in simulation is attractive but requires to transfer policies between domains, which demands trained policies to be robust to such domain gap. In this work, we make two contributions: (i) we perform the first benchmark comparison of existing learned control policies for agile quadrotor flight and show that training a control policy that commands body-rates and thrust results in more robust sim-to-real transfer compared to a policy that directly specifies individual rotor thrusts, (ii) we demonstrate for the first time that such a control policy trained via deep reinforcement learning can control a quadrotor in real-world experiments at speeds over 45 km/h.

We present a novel application of continuum robots acting as concrete hoses to support 3D printing of cementitious materials. An industrial concrete hose was fitted with a cable harness and remotely actuated via tendons. The resulting continuum hose robot exhibited non constant curvature. In order to account for this, a new geometric approach to modeling variable curvature inverse kinematics using Euler curves is introduced herein. The new closed form model does not impose any additional computational cost compared to the constant curvature model and results in a marked improvement in the observed performance. Experiments involving 3D printing with cementitious mortar using a continuum hose robot were also conducted.

Automatic laparoscope motion control is fundamentally important for surgeons to efficiently perform operations. However, its traditional control methods based on tool tracking without considering information hidden in surgical scenes are not intelligent enough, while the latest supervised imitation learning (IL)-based methods require expensive sensor data and suffer from distribution mismatch issues caused by limited demonstrations. In this paper, we propose a novel Imitation Learning framework for Laparoscope Control (ILLC) with reinforcement learning (RL), which can efficiently learn the control policy from limited surgical video clips. Specially, we first extract surgical laparoscope trajectories from unlabeled videos as the demonstrations and reconstruct the corresponding surgical scenes. To fully learn from limited motion trajectory demonstrations, we propose Shape Preserving Trajectory Augmentation (SPTA) to augment these data, and build a simulation environment that supports parallel RGB-D rendering to reinforce the RL policy for interacting with the environment efficiently. With adversarial training for IL, we obtain the laparoscope control policy based on the generated rollouts and surgical demonstrations. Extensive experiments are conducted in unseen reconstructed surgical scenes, and our method outperforms the previous IL methods, which proves the feasibility of our unified learning-based framework for laparoscope control.

In this paper, we propose a novel direct visual odometry algorithm to take the advantage of a 360-degree camera for robust localization and mapping. Our system extends direct sparse odometry by using a spherical camera model to process equirectangular images without rectification to attain omnidirectional perception. After adapting mapping and optimization algorithms to the new model, camera parameters, including intrinsic and extrinsic parameters, and 3D mapping can be jointly optimized within the local sliding window. In addition, we evaluate the proposed algorithm using both real world and large-scale simulated scenes for qualitative and quantitative validations. The extensive experiments indicate that our system achieves start of the art results.

This paper addresses the problem of visual-inertial odometry (VIO) with a downward facing monocular camera when a micro aerial vehicle (MAV) flying at high altitude (over 100 meters). It is important to note that large scene depth causes visual motion constraints significantly less informative than that in near-sighted scenarios as considered in most existing VIO methods. To cope with this challenge, we develop an efficient MSCKF-based VIO algorithm aided by a single 1D laser range finder (LRF), termed LRF-VIO, which runs in real time on an embedded system. The key idea of the proposed LRF-VIO is to fully exploit the limited metric distance information provided by the 1D LRF to disambiguate the scale during visual feature tracking, thus improving the VIO performance at high altitude. Specifically, during the MSCKF visual measurement update, we deliberately constrain the depth of those SLAM features co-planar with the single LRF measuring point. Additionally, delayed initialization of features utilizes the LRF measurements whenever possible, and online extrinsic calibration between the LRF and monocular camera is performed to further improve estimation accuracy and robustness. The proposed LRF-VIO is extensively validated in both indoor and outdoor real-world experiments, outperforming the state-of-the-art methods.