IROS 2019 Papers

A Simultaneous Localization and Mapping (SLAM) system is a complex program consisting of several interconnected components with different functionalities such as optimization, tracking or loop detection. Whereas the literature addresses in detail how enhancing the algorithmic aspects of the individual components improves SLAM performance, the modal aspects, such as when to localize, relocalize or close a loop, are usually left aside. In this paper, we address the modal aspects of a SLAM system and show that the design of the modal controller has a strong impact on SLAM performance in particular in terms of robustness against unforeseen events such as sensor failures, perceptual aliasing or kidnapping. We preset a novel taxonomy for the components of a modern SLAM system, investigate their interplay and propose a highly modular architecture of a generic SLAM system using the Unified Modeling LanguageTM (UML) state machine formalism. The result, called SLAM state machine, is compared to the modal controller of several state-of-the-art SLAM systems and evaluated in two experiments. We demonstrate that our state machine handles unforeseen events much more robustly than the state-of-the-art systems.

Rigid body dynamics calculations are needed for many tasks in robotics, including online control. While there currently exist several competing software implementations that are sufficient for use in traditional control approaches, emerging sophisticated motion control techniques such as nonlinear model predictive control demand orders of magnitude more frequent dynamics calculations. Current software solutions are not fast enough to meet that demand for complex robots. The goal of this work is to examine the performance of current dynamics software libraries in detail. In this paper, we (i) survey current state-of-the-art software implementations of the key rigid body dynamics algorithms (RBDL, Pinocchio, Rigid-BodyDynamics.jl, and RobCoGen), (ii) establish a methodology for benchmarking these algorithms, and (iii) characterize their performance through real measurements taken on a modern hardware platform. With this analysis, we aim to provide direction for future improvements that will need to be made to enable emerging techniques for real-time robot motion control. To this end, we are also releasing our suite of benchmarks to enable others to help contribute to this important task.

This paper proposes a belief-driven control policy of a drone with microphones for multiple sound source search. As the sound source localization by drones is uncertain because of the observation significantly distorted by noise such as rotor noise, the belief on the estimated targets may consist of multiple peaks that are spread over the bounded search area. The proposed control policy is formulated with a robust cost function so that the function encodes the search mission properly. A peak management mechanism is additionally introduced to keep tracking all targets by masking sufficiently observed and well estimated targets whose peaks normally become steep and high. The proposed control policy was evaluated by numerical simulations, and experiments, and those results have validated the efficacy of the proposed control policy.

Due to the complexity of the real world, autonomous systems use decision-making models that rely on simplifying assumptions to make them computationally tractable and feasible to design. However, since these limited representations cannot fully capture the domain of operation, an autonomous system may encounter unanticipated scenarios that cannot be resolved effectively. We first formally introduce an introspective autonomous system that uses belief space metareasoning to recover from exceptions by interleaving a main decision process with a set of exception handlers. We then apply introspective autonomy to autonomous driving. Finally, we demonstrate that an introspective autonomous vehicle is effective in simulation and on a fully operational prototype.

We present a method and an open-source implementation (BeBOT) for the generation of trajectories for autonomous system operations using Bernstein polynomials. Bernstein polynomials possess convenient geometric properties that enable the trajectory planner to efficiently evaluate and impose constraints along the vehicles' trajectories, such as maximum speed and angular rates, minimum distance between trajectories and between the vehicles and obstacles. Thus, the proposed method is particularly suitable for generating trajectories in real-time for safe operations in complex environments and multiple vehicle missions.

Bayesian Optimization (BO) is an effective method for optimizing expensive-to-evaluate black-box functions with a wide range of applications for example in robotics, system design and parameter optimization. However, scaling BO to problems with large input dimensions (>10) remains an open challenge. In this paper, we propose to leverage results from optimal control to scale BO to higher dimensional control tasks and to reduce the need for manually selecting the optimization domain. The contributions of this paper are twofold: 1) We show how we can make use of a learned dynamics model in combination with a model-based controller to simplify the BO problem by focusing onto the most relevant regions of the optimization domain. 2) Based on (1) we present a method to find an embedding in parameter space that reduces the effective dimensionality of the optimization problem. To evaluate the effectiveness of the proposed approach, we present an experimental evaluation on real hardware, as well as simulated tasks including a 48-dimensional policy for a quadcopter.

We introduced the disturbance degree of ground and proposed an evaluation method to measure the mobile performance of a crawler on soft ground during direction switching. First, we developed a planar omnidirectional crawler, which had a configuration with two left and right unit crawlers for performing turning motion, as the target for evaluation. Second, by utilizing the proposed disturbance degree of ground evaluation method, we investigated how the turning and translational motions of the crawler mechanism affected soft ground by measuring the flow of sand on a horizontal surface. It was quantitatively shown that translational motion switched the travel direction with lesser disturbance to the road surface compared to turning motion. We confirmed that ground disturbance could be evaluated during direction switching using the proposed method.

Locomotion mode recognition based on the off-line trained model brings difficulties in integration and application to wearable robots. In this paper, we put forward an on-board training based on back propagation (BP) neural network and developed the real-time locomotion mode recognition research in robotic transtibial prosthesis. Three transtibial amputees participated in the study to finish the designed six experimental tasks (standing, level ground walking, stair ascending and descending, ramp ascending and descending) with robotic transtibial prostheses. Data of six locomotion modes were collected under normal speed condition as training data set to train model on board. Based on the on-board trained models, real-time recognition experiments were developed under three different speeds conditions. The total recognition accuracies were 91.54%, 96.72% and 95.35% corresponding to slow, normal and fast speeds, respectively. The results showed some adaptation of recognition for the six locomotion modes at different speeds. The on-board training strategy was feasible and effective with satisfactory performance.

This paper introduces an obstacle-avoiding algorithm for bipedal robots, especially in push recovery situations. Typically, There are many algorithms that plan footstep to avoid obstacles based on vision recognition data. However, if the robot is pushed, the planned footprint will change, and thus, there is no guarantee that it will avoid obstacles. Although modified stepping positions can be limited, the robot's stability is not assured. Our proposed algorithm focuses on avoiding obstacles through vision recognition in push recovery situations and generating compensation actions for instability by restricting modified footsteps. We fuse vision feedback with our previous push recovery algorithm, which optimizes the ankle, hip, and stepping strategies. We build simple grid data using vision recognition and apply it to the inequality constraint of the stepping position. We validate the effectiveness of our algorithm using the bipedal platform GAZELLE with the Kinect V2 RGBD sensor.

The use of unmanned aerial systems for industrial applications has evolved considerably in recent years. This paper presents an aerial system capable of perching autonomously on pipes for inspection and maintenance in industrial environments. The target pipe to perch on is detected using a visual algorithm based on a semantic convolutional neuronal network. The information from a color camera is used to segment the image. Then, the segmentation information is fused with a depth image to estimate the pipe's pose, so that the pose of the robot can be controlled relative to it. The aerial robot is equipped with a soft landing system that robustly attaches it to the pipe. The article presents the complete development of the system. Experimental results performed in outdoor environments are shown.

Concentric tube robots (CTR) can traverse tightly curved paths and offer dexterity in constrained environments, making them advantageous for minimally invasive surgical scenarios that experience strict anatomical and surgical constraints. Their shape is controlled via rotation and translation of several concentrically arranged super-elastic precurved tubes that form the robot backbone. As the elastic energy accumulated in the backbone due to bending and twist of the tubes increases, robots can exhibit sudden snapping motions, which can damage the surrounding tissues. In this paper, we proposed an approach for closed-loop steering of a redundant CTR that allows for snap-free motion and enhances its force/velocity manipulability, increasing the capacity of the robot to move and/or exercise forces along any direction. First, a controller stabilizes the CTR end-effector on a desired time-variant trajectory. Next, an online optimizer uses the robot's redundant Degrees of Freedom (DoF) to reshape its manipulability in real-time and steer it away from potentially snapping configurations or increase its capacity in delivering force payloads. Simulations and experiments demonstrate the performance of the proposed control strategy. The controller can steer a generally unstable CTR along trajectories while avoiding instabilities with a mean error of 850 μm, corresponding to 0.6% of arclength, and improves robot ability to exercise forces by 55%.

Humanoid robots are expected to conduct tasks on behalf of humans in places such as a disaster scattered environment. Although humanoid robots have potentials to walk on uneven ground unlike wheeled robots, it is difficult to reach a given destination without falling down based on only visual information. In this paper, to reach the destination safely, we propose the autonomous safe locomotion system applying vision and sole reaction force to the footstep planning. Considering force information in addition to visual information, the robot can plan a path avoiding unstable footholds. The planned path is safer than a path which is planned based on only visual information. In our system, the robot checks if the foothold is safe or not by the foothold ascertainment motion. In addition to that, the robot saves the results of the motion to the database with the foothold label given by the visual classifier. To judge foothold safety, stiffness of the foothold is estimated from the reaction force and stepping amount. We propose the system considering these requirements for safe locomotion for bipedal robots and show experimental results using a real bipedal robot CHIDORI.

Hybrid ground and aerial vehicles can possess distinct advantages over ground-only or flight-only designs in terms of energy savings and increased mobility. In this work we outline our unified framework for controls, planning, and autonomy of hybrid ground/air vehicles. Our contribution is three-fold: 1) We develop a control scheme for the control of passive two-wheeled hybrid ground/aerial vehicles. 2) We present a unified planner for both rolling and flying by leveraging differential flatness mappings. 3) We conduct experiments leveraging mapping and global planning for hybrid mobility in unknown environments, showing that hybrid mobility uses up to five times less energy than flying only1.1Video at https://youtu.be/nlGfYehTLpg.

This paper proposes a realistic indoor modeling framework for large-scale indoor spaces. The proposed framework reduces the geometric complexity of an indoor model to efficiently represent large-scale environments for image-based rendering (IBR) approaches. For this purpose, the proposed framework removes geometrically excluded objects (GEOs) in point cloud and images, which represent the primary factors in high geometric complexity. In particular, GEOs are coherently removed from all images using a global geometry model. Then, the remaining holes are inpainted using globally consistent guidelines, to achieve accurate image blending in IBR approaches. The experimental results verify that the proposed GEO removal framework provides efficient point clouds and images for realistic indoor modeling in large-scale indoor spaces.

We have successfully achieved manipulation and assembly of microbeads having the size of 100μm diameter by hemispherical end-effectors with high stability and accuracy. The motivation of achieving assembly of actual cells lies in the great significance of it in tissue regeneration and cell analysis. Firstly, the most difficult problem we need to solve is the releasing problem caused by adhesion force. The viscosity on cell surface is much larger than the microbeads which makes cell releasing challenging. Secondly, the cell can generate its deformation, then contact area with end-effector will change during grasping process. This may influence the adhesion force and also bring problem to releasing. Thirdly, cell is much smaller, around 15μm in diameter, so we need to fabricate smaller end-effector to achieve successful manipulation and ensure the stability in the meantime. In this paper, we realize the manipulation by decreasing the adhesion forces and apply vibration to release a cell stably. We found the appropriate scale size for the end-effector is around 10μm diameter. It can not only grasp a 15μm cell but also bring little interference to the environment. As a demonstration of the proposed manipulation method, the repeated experiments were conducted to explore the dependence of adhesion force on the grasping distance, which can be helpful in the improvement of successful rate. Finally, we achieved automatic cell assembly using Hela cells.

Multiple LiDARs have progressively emerged on autonomous vehicles for rendering a rich view and dense measurements. However, the lack of precise calibration negatively affects their potential applications. In this paper, we propose a novel system that enables automatic multi-LiDAR calibration method without any calibration target, prior environment information, and manual initialization. Our approach starts with a hand-eye calibration by aligning the motion of each sensor. The initial results are then refined by an appearance-based method by minimizing a cost function constructed by point-plane distance. Experimental results on simulated and real-world data demonstrate the reliability and accuracy of our calibration approach. The proposed approach can calibrate a multi-LiDAR system with the rotation and translation errors less than 0. 04rad and 0. 1m respectively for a mobile platform.

Domestic robots could eventually transform our lives, but safely operating in home environments requires a rich understanding of indoor scenes. Learning-based techniques for scene segmentation require large-scale, pixel-level annotations, which are laborious and expensive to collect. We propose an automatic method for pixel-wise semantic annotation of video sequences, that gathers cues from object detectors and indoor 3D room-layout estimation and then annotates all the image pixels in an energy minimization framework. Extensive experiments on a publicly available video dataset (SUN3D) evaluate the approach and demonstrate its effectiveness.

The research on rare cells makes a significant contribution to biology research and medical treatment for the application of diagnostic operation as well as prognoses treatment. Therefore, sorting them from heterogeneous mixtures is crucial and valuable. Traditional cell sorting methods featured with poor purity and recovery rate as well as limited flexibility, which are not ideal approaches for rare type. In this paper, we proposed a cell screening method based on automated microrobotic aspiration-and-placement strategy under fluorescence microscope. An innovative autofocusing visual feedback (AVF) method is proposed for precise three-dimensional (3D) locating of target cells. For depth detection, multiple depth from defocus (MDFD) method is adopted to solve symmetry problem and attain an average accuracy of 97.07%. For planar locating, Markov random field (MRF) based locating method is utilized to separate and locate the overlapped cells. The end actuator locating and real-time tracking are performed relying on normalized cross-correlation (NCC) method. Experiential results show that our system collects rare cells (100 cells ml-1) at a speed of 5 cells min-1 with 90% purity and 75% recovery rate, which is valuable for biological and medical application.

Extensive research efforts have been made toward automating the microinjection of biological cells by leveraging micro-robotic technologies. However, to best knowledge of the authors, there is no report on the automation of the time-consuming process: moving the injection tools (a micropipette, a grippers, etc.) and cells into the field of view (FOV) of microscope from the macro FOV(outside the microscopic FOV). This paper presents a novel macro-micro conversion strategy, and a grid detection and positioning algorithm to automate the time-consuming step of moving the injection tools and cells to the microscopic FOV. The proposed solution can free the technician from the laborious hand-eye coordination operations for moving the injection tools and cells to the target position within the microscopic FOV. Furthermore, this paper proposes an auto-focusing algorithm to automate the operation step: moving down the gripper from the air outside the culture media and then precisely clamping a cell in the liquid environment for injection. In the proposed solution, the active window-based auto-focusing algorithm is developed to solve the challenging problem: the image information is lost due to the “viscous effect” taking place when the gripper jaw touches the water surface. The proposed solutions are tested and validated by the microinjection experiments of zebrafish embryos using the in-house develop micro-robotic system. The technologies and strategies proposed in this paper significantly improve the automation level of the cell microinjections, and can be easily extended to any other micromanipulation of biological cells.