ICRA 2020 Papers

Anytime algorithms offer a trade-off between solution quality and computation time that has proven to be useful in autonomous systems for a wide range of real-time planning problems. In order to optimize this trade-off, an autonomous system has to solve a challenging meta-level control problem: it must decide when to interrupt the anytime algorithm and act on the current solution. Prevailing meta-level control techniques, however, make a number of unrealistic assumptions that reduce their effectiveness and usefulness in the real world. Eliminating these assumptions, we first introduce a model-free approach to meta-level control based on reinforcement learning and prove its optimality. We then offer a general meta-level control technique that can use different reinforcement learning methods. Finally, we show that our approach is effective across several common benchmark domains and a mobile robot domain.

Robotic Wire Arc Additive Manufacturing (WAAM) utilizes a robot arm as a motion system to build 3D metallic objects by depositing weld beads one above the other in a layer by layer fashion. A key part of this approach is the process study and control of Multi-Layer Multi-Bead (MLMB) deposition, which is very sensitive to process parameters and prone to error stacking. Despite its importance, it has been receiving less attention than its single bead counterpart in literature, probably due to the higher experimental overhead and complexity of modeling. To address these challenges, this paper proposes an integrated learning-correction framework, adapted from Model-Based Reinforcement Learning, to iteratively learn the direct effect of process parameters on MLMB print while simultaneously correct for any inter-layer geometric digression such that the final output is still satisfactory. The advantage is that this learning architecture can be used in conjunction with actual parts printing (hence, in-situ study), thus minimizing the required training time and material wastage. The proposed learning framework is implemented on an actual robotic WAAM system and experimentally evaluated.

Recently, swarm control of micro-/nanorobots has drawn much attention in the field of microrobotics. This paper reports a mobile paramagnetic nanoparticle swarm with the capability of active shape deformation that can improve its environment adaptability. We show that, by applying elliptical rotating magnetic fields, a swarm pattern called the elliptical paramagnetic nanoparticle swarm (EPNS) would be formed. When changing the field ratio-α (i.e. the strength ratio between the minor axis and major axis of the elliptical field), the shape ratio-β of the EPNS (i.e. the length ratio between the major axis and minor axis) will change accordingly. However, automatically control this shape deformation process has difficulties because the deformation dynamics has strong nonlinearity, model variation and long time requirement. To solve this problem, we propose a fuzzy logic-based control scheme that utilizes the knowledge and control experience from skilled human operators. Experiments show that the proposed control scheme can stably maneuver the shape deformation of the EPNS with small overshoot, which cannot be achieved by conventional PI control. Moreover, experimental results show that, with the automatic shape deformation control, shape of the EPNS is controlled with high reversibility and also can be well maintained during the planar rotational and translational locomotion of the EPNS.

We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust learned dense visual embeddings representation of the scene, and a task graph of the taught behaviors. We demonstrate the robustness of the approach by presenting results for performing a variety of tasks, under different environmental conditions, in multiple real homes. Our approach achieves 85% overall success rate on three tasks that consist of an average of 45 behaviors each. The video is available at: https://youtu.be/HSyAGMGikLk.

To fuse information from inertial measurement units (IMU) with other sensors one needs an accurate model for IMU error propagation in terms of position, velocity and orientation, a triplet we call extended pose. In this paper we leverage a nontrivial result, namely log-linearity of inertial navigation equations based on the recently introduced Lie group SE2(3), to transpose the recent methodology of Barfoot and Furgale for associating uncertainty with poses (position, orientation) of SE(3) when using noisy wheel speeds, to the case of extended poses (position, velocity, orientation) of SE2(3) when using noisy IMUs. Besides, our approach to extended poses combined with log-linearity property allows revisiting the theory of preintegration on manifolds and reaching a further theoretic level in this field. We show exact preintegration formulas that account for rotating Earth, that is, centrifugal force and Coriolis effect, may be derived as a byproduct.

Consider the case of multiple visual sensors perceiving the same scene from different viewpoints. In order to achieve consistent visual perception, the problem of data association, in this case establishing correspondences between observed features, must be first solved. In this work, we consider multiway matching which is a specific instance of multi-sensory data association. Multiway matching refers to the problem of establishing correspondences among a set of images from noisy pairwise correspondences, typically by exploiting cycle- consistency. We propose a novel optimization-based formulation of multiway matching problem as a nonconvex low-rank matrix approximation problem. We propose two novel algorithms for numerically solving the problem at hand. The first one is an algorithm based on the Alternating Direction Method of Multipliers (ADMM). The second one is a Riemannian trust- region method on the multinomial manifold, the manifold of strictly positive stochastic matrices, equipped with the Fisher information metric. Experimental results demonstrate that the proposed methods have the state of the art performance in multiway matching while reducing the computational complexity compared to the state of the art.

In recent years, a myriad of advanced results have been reported in the community of imitation learning, ranging from parametric to non-parametric, probabilistic to non-probabilistic and Bayesian to frequentist approaches. Meanwhile, ample applications (e.g., grasping tasks and humanrobot collaborations) further show the applicability of imitation learning in a wide range of domains. While numerous literature is dedicated to the learning of human skills in unconstrained environments, the problem of learning constrained motor skills, however, has not received equal attention. In fact, constrained skills exist widely in robotic systems. For instance, when a robot is demanded to write letters on a board, its end-effector trajectory must comply with the plane constraint from the board. In this paper, we propose linearly constrained kernelized movement primitives (LC-KMP) to tackle the problem of imitation learning with linear constraints. Specifically, we propose to exploit the probabilistic properties of multiple demonstrations, and subsequently incorporate them into a linearly constrained optimization problem, which finally leads to a non-parametric solution. In addition, a connection between our framework and the classical model predictive control is provided. Several examples including simulated writing and locomotion tasks are presented to show the effectiveness of our framework.

This work studies the accuracy of a simple but effective analytical model for a flapping-wings UAV in longitudinal gliding flight configuration comparing it with experimental results of a real ornithopter. The aerodynamic forces are modeled following the linearized potential theory for a flat plate in gliding configuration, extended to flapping-wing episodes modeled also by the (now unsteady) linear potential theory, which are studied numerically. In the gliding configuration, the model reaches a steady-state descent at given terminal velocity and pitching and gliding angles, governed by the wings and tail position. In the flapping-wing configuration, it is noticed that the vehicle can increase its flight velocity and perform climbing episodes. A realistic simulation tool based on Unreal Engine 4 was developed to visualize the effect of the tail position and flapping frequencies and amplitudes on the ornithopter flight in real time. The paper also includes the experimental validation of the gliding flight and the data has been released for the community.

Over the last years, hand exoskeletons have become a popular and efficient technical solution for assisting people that suffer from neurological and musculoskeletal diseases and enhance the capabilities of healthy individuals. These devices can vary from rigid and complex structures to soft, lightweight, wearable gloves. Despite the significant progress in the field, most existing solutions do not provide the same dexterity as the healthy human hand. In this paper, we focus on the development of a hybrid (tendon-driven and pneumatic), lightweight, affordable, wearable exoskeleton glove equipped with abduction/adduction capabilities and a pneumatic telescopic extra thumb that increases grasp stability. The efficiency of the proposed device is experimentally validated through three different types of experiments: i) abduction/adduction tests, ii) force exertion experiments that capture the maximum forces that can be applied by the proposed device, and iii) grasp quality assessment experiments that focus on the effect of the inflatable thumb on enhancing grasp stability. The hybrid assistive glove considerably improves the grasping capabilities of the user, being able to exert the forces required to assist people in the execution of activities of daily living.

Defining a robot's expressivity is a difficult task that requires thoughtful consideration of the potential of various robot modalities and a model of communication that humans understand. Humanoid and zoomorphic-designed robots can easily take cues from human and animals, respectively when designing their expressivity. However, a robot design that is neither human nor animal-like does not have a clear model to follow in terms of designing expressivity. Animation presents a potential model in these circumstances as animated characters in movies take various forms, sizes, shapes and styles, and are successful in defining expressivity that is widely accepted across different languages and cultures. In this paper, we discuss the development and design of the expressivity of Haru, a table top robot that is neither human nor animal-like and the application of animation expertise to the holistic treatment of the different modalities. The method maximizes animation techniques and expertise normally applied to movies to generate expressivity that is then transferred to the robot hardware. Experimental results show that the robot's expressivity generated using our method is easily understood and are preferred to the conventional approach of generating expressions.

Performing collaborative semantic mapping is a critical challenge for cooperative robots to maintain a comprehensive contextual understanding of the surroundings. Most of the existing work either focus on single robot semantic mapping or collaborative geometry mapping. In this paper, a novel hierarchical collaborative probabilistic semantic mapping framework is proposed, where the problem is formulated in a distributed setting. The key novelty of this work is the mathematical modeling of the overall collaborative semantic mapping problem and the derivation of its probability decomposition. In the single robot level, the semantic point cloud is obtained based on heterogeneous sensor fusion model and is used to generate local semantic maps. Since the voxel correspondence is unknown in collaborative robots level, an Expectation-Maximization approach is proposed to estimate the hidden data association, where Bayesian rule is applied to perform semantic and occupancy probability update. The experimental results show the high quality global semantic map, demonstrating the accuracy and utility of 3D semantic map fusion algorithm in real missions.

Real-world autonomous systems often employ probabilistic predictive models of human behavior during planning to reason about their future motion. Since accurately modeling human behavior a priori is challenging, such models are often parameterized, enabling the robot to adapt predictions based on observations by maintaining a distribution over the model parameters. Although this enables data and priors to improve the human model, observation models are difficult to specify and priors may be incorrect, leading to erroneous state predictions that can degrade the safety of the robot motion plan. In this work, we seek to design a predictor which is more robust to misspecified models and priors, but can still leverage human behavioral data online to reduce conservatism in a safe way. To do this, we cast human motion prediction as a Hamilton-Jacobi reachability problem in the joint state space of the human and the belief over the model parameters. We construct a new continuous-time dynamical system, where the inputs are the observations of human behavior, and the dynamics include how the belief over the model parameters change. The results of this reachability computation enable us to both analyze the effect of incorrect priors on future predictions in continuous state and time, as well as to make predictions of the human state in the future. We compare our approach to the worst-case forward reachable set and a stochastic predictor which uses Bayesian inference and produces full future state distributions. Our comparisons in simulation and in hardware demonstrate how our framework can enable robust planning while not being overly conservative, even when the human model is inaccurate. Videos of our experiments can be found at the project website1.

Robots navigating in human crowds need to optimize their paths not only for their task performance but also for their compliance to social norms. One of the key challenges in this context is the lack of standard metrics for evaluating and optimizing a socially compliant behavior. Existing works in social navigation can be grouped according to the differences in their optimization objectives. For instance, the reinforcement learning approaches tend to optimize on the comfort aspect of the socially compliant navigation, whereas the inverse reinforcement learning approaches are designed to achieve natural behavior. In this paper, we propose NaviGAN, a generative navigation algorithm that jointly optimizes both of the comfort and naturalness aspects. Our approach is designed as an adversarial training framework that can learn to generate a navigation path that is both optimized for achieving a goal and for complying with latent social rules. A set of experiments has been carried out on multiple datasets to demonstrate the strengths of the proposed approach quantitatively. We also perform extensive experiments using a physical robot in a realworld environment to qualitatively evaluate the trained social navigation behavior. The video recordings of the robot experiments can be found in the link: https://youtu.be/61blDymjCpw.

This paper presents an approach to address data scarcity problems in underwater image datasets for visual detection of marine debris. The proposed approach relies on a two-stage variational autoencoder (VAE) and a binary classifier to evaluate the generated imagery for quality and realism. From the images generated by the two-stage VAE, the binary classifier selects "good quality" images and augments the given dataset with them. Lastly, a multi-class classifier is used to evaluate the impact of the augmentation process by measuring the accuracy of an object detector trained on combinations of real and generated trash images. Our results show that the classifier trained with the augmented data outperforms the one trained only with the real data. This approach will not only be valid for the underwater trash classification problem presented in this paper, but it will also be useful for any data-dependent task for which collecting more images is challenging or infeasible.

This paper proposes a Guidance, Navigation, and Control (GNC) architecture for planetary rovers targeting the conditions of upcoming Mars exploration missions such as Mars 2020 and the Sample Fetching Rover (SFR). The navigation requirements of these missions demand a control architecture featuring autonomous capabilities to achieve a fast and long traverse. The proposed solution presents a two-level architecture where the efficient navigation (lower) level is always active and the full navigation (upper) level is enabled according to the difficulty of the terrain. The first level is an efficient implementation of the basic functionalities for autonomous navigation based on hazard detection, local path replanning, and trajectory control with visual odometry. The second level implements an adaptive SLAM algorithm that improves the relative localization, evaluates the traversability of the terrain ahead for a more optimal path planning, and performs global (absolute) localization that corrects the pose drift during longer traverses. The architecture provides a solution for long-range, low supervision, and fast planetary exploration. Both navigation levels have been validated on planetary analog field test campaigns.

This paper reports the development of a fully actuated body-mounted robotic assistant for MRI-guided low back pain injection. The robot is designed with a 4-DOF needle alignment module and a 2-DOF remotely actuated needle driver module. The 6-DOF fully actuated robot can operate inside the scanner bore during imaging; hence, minimizing the need of moving the patient in or out of the scanner during the procedure, and thus potentially reducing the procedure time and streamlining the workflow. The robot is built with a lightweight and compact structure that can be attached directly to the patient’s lower back using straps; therefore, attenuating the effect of patient motion by moving with the patient. The novel remote actuation design of the needle driver module with beaded chain transmission can reduce the weight and profile on the patient, as well as minimize the imaging degradation caused by the actuation electronics. The free space positioning accuracy of the system was evaluated with an optical tracking system, demonstrating the mean absolute errors (MAE) of the tip position to be 0.99±0.46 mm and orientation to be 0.99±0.65◦. Qualitative imaging quality evaluation was performed± on a human volunteer, revealing minimal visible image degradation that should not affect the procedure. The mounting stability of the system was assessed on a human volunteer, indicating the 3D position variation of target movement with respect to the robot frame to be less than 0.7 mm.

Virtual fixtures (VFs) improve human operator performance in teleoperation scenarios. However, the generation of VFs is challenging, especially in unstructured environments. In this work, we introduce a framework for the interactive generation of VF. The method is based on the observation that a human can easily understand just by looking at the remote environment which VF could help in task execution. We propose a user interface that detects features on camera images and permits interactive selection of the features. We demonstrate how the feature selection can be used for designing VF, providing 6-DOF haptic feedback. In order to make the proposed framework more generally applicable to a wider variety of applications, we formalize the process of virtual fixture generation (VFG) into the specification of features, geometric primitives, and constraints. The framework can be extended further by the introduction of additional components. Through the human subject study, we demonstrate the proposed framework is intuitive, easy to use while effective, especially for performing hard contact tasks.

In this work we present a novel, approximate, efficient algorithm for determining the Trim Flight Envelope of a fixed-wing UAV, based on a generic, nonlinear numerical model. The resulting Flight Envelope is expressed as a convex intersection of half-spaces. Subsequently, a Model Predictive Controller (MPC) is designed which takes into account the Flight Envelope constraints, to avoid Loss-of-Control. The overall system is shown to operate in real-time in a simulation environment.

An important research issue in mobile robotics is performance assessment of robot SLAM algorithms in terms of their localization accuracy. Typically, SLAM algorithms are evaluated with the help of benchmark datasets or expensive equipment such as motion capture. Benchmark datasets however, are environment-specific, and use of motion capture constrains spatial coverage and affordability. In this paper, we present a novel method for SLAM performance evaluation, which only uses distinctive markers (such as AR tags), randomly placed in the robot navigation environment at arbitrary locations, and observes these markers with a camera onboard of the robot. Formulated as a generative latent optimization (GLO) problem, our method uses the local robot-to-marker poses to evaluate the global robot pose estimates by a SLAM algorithm and therefore its performance. Through extensive experiments on two robots, three localization/SLAM algorithms and both LiDAR and RGB-D sensors, we demonstrate the feasibility and accuracy of our proposed method.