IROS 2018 Papers

In a human-robot interaction, a robot needs to move to a position where the robot can obtain high reliability data of people, such as positions, postures, and voice. This is because the human recognition reliability depends on the positional relation between the people and the robot. In addition, the robot should choose the sensor data which is necessary to perform the interaction task. Therefore, it is necessary to navigate the robot to the position to obtain the data for initiation of the interaction task. Accordingly, we need to design a path-planning method considering sensor characteristics, human recognition reliability, and task contents. Although previous studies proposed path-planning methods using an interaction potential considering sensor characteristics, they did not consider the task contents and the human recognition reliability, which are important for practical application and did not applied to interaction with multiple people. Consequently, we present a path-planning method considering the task contents and the human recognition reliability using multimodal potential field integrating these information. We verified effectiveness of the path-planning method for interaction with multiple people.

Capsule endoscopes have gained popularity over the last decade as minimally invasive devices for diagnosing gastrointestinal abnormalities such as colorectal cancer. While this technology offers a less invasive and more convenient alternative to traditional scopes, these capsules are only able to provide observational capabilities due to their passive nature. With the addition of a reliable mobility system and a real-time navigation system, capsule endoscopes could transform from observational devices into active surgical tools, offering biopsy and therapeutic capabilities and even autonomous navigation in a single minimally invasive device. In this work, a vision system is developed to allow for autonomous lumen center tracking and haustral fold identification and tracking during colonoscopy. This system is tested for its ability to accurately identify and track multiple haustral folds across many frames in both simulated and in vivo video, and the lumen center tracking is tested onboard a robotic endoscope platform (REP) within an active simulator to demonstrate autonomous navigation. In addition, real-time localization is demonstrated using open source ORB-SLAM2. The vision system successfully identified 95.6% of Haustral folds in simulator frames and 70.6% in in vivo frames and false positives occurred in less than 1% of frames. The center tracking algorithm showed in vivo center estimates within a mean error of 6.6% of physician estimates and allowed for the REP to traverse 2 m of the active simulator in 6 minutes without intervention.

This paper presents an autonomous perching concept for multirotor aerial vehicles. The Autonomous Grasping Robotic Aerial System for Perching (AGRASP)represents a novel integration of robotics perception, vision-based path planning, and biomimetically-inspired manipulation on a small, lightweight aerial robot with highly-constrained sensor and processing capacity. Computationally lightweight perception algorithms pull candidate perch structures out of a complex environment with no a priori knowledge of the operational space. The innovative manipulator design combines both active grasp and passive grip enabling it to maintain hold on the perch even with all power off. We experimentally demonstrate, for the first time, a quadrotor autonomously detecting and landing on a perch relying solely on onboard sensing and processing.

Behavior trees (BT) are a popular control architecture in the computer game industry, and have been more recently applied in robotics. One open question is how can intelligent agents/robots autonomously acquire their behavior trees for task level control? In contrast with existing approaches that either refine an initially given BT, or directly build the BT based on human feedback/demonstration, we leverage reinforcement learning (RL) that allows robots to autonomously learn control policies by repeated task interaction, but often expressed in a language more difficult to interpret than BTs. The learned control policy is then converted to a behavior tree via our proposed decanonicalization algorithm. The feasibility of this idea is based on a proposed notion of canonical behavior trees (CBT). In particular, we show (1) CBTs are sufficiently expressive to capture RL control policies, and (2) that RL can be independent of an optimal behavior permutation, despite the BT convention of left-to-right priority, thus obviating the need for a combinatorial search. Two evaluation domains help illustrate our approach.

The use of sensor technologies for material characterization is rapidly growing and innovative advancement is observed. However, the use of sensor combinations for a raw material characterization in mining is very limited and automation of the material identification process using a combined sensor signal is not defined. Potential sensor technologies for raw material characterization were evaluated based on the applicability and technological maturity. To ensure a rapid implementation of the Real-time mining (RTM) project concept, mature technologies such as Red Green Blue (RGB) imaging, Visible Near Infrared (VNIR) hyperspectral imaging, Short Wave Infrared (SWIR) hyperspectral imaging, Fourier-Transform Infrared Spectroscopy (FTIR), Laser Induced Breakdown Spectroscopy (LIBS) and Raman were selected. Each selected technology was assessed for automation in sensing and applicability (for characterization of the test case materials). Based on the results the sensor data were further considered for data fusion. The proposed sensor combinations approach encompasses three levels of data fusion: low-level, mid-level and high-level. The data of the different sensors are fused together in order to acquire a wide range of mineral properties within each lithotype and an improved classification and predictive models. The preferred level of data fusion and preferred sensor data combinations will be used to develop a multi-variate statistical interpretation rule which relates combination of sensors signals with raw material properties. Thus a tool which integrates the combined sensor signal with materials properties will be developed and used to automate the material characterization process.

Motion planning algorithms attempt to find a good compromise between planning time and quality of solution. Due to their heuristic nature, they are typically configured with several parameters. In this paper we demonstrate that, in many scenarios, the widely used default parameter values are not ideal. However, finding the best parameters to optimise some metric(s) is not trivial because the size of the parameter space can be large. We evaluate and compare the efficiency of four different methods (i.e. random sampling, AUC-Bandit, random forest, and bayesian optimisation) to tune the parameters of two motion planning algorithms, BKPIECE and RRT-connect. We present a table-top-reaching scenario where the seven degrees-of-freedom KUKA LWR robotic arm has to move from an initial to a goal pose in the presence of several objects in the environment. We show that the best methods for BKPIECE (AUC-Bandit) and RRT-Connect (random forest) improve the performance by 4.5x and 1.26x on average respectively. Then, we generate a set of random scenarios of increasing complexity, and we observe that optimal parameters found in simple environments perform well in more complex scenarios. Finally, we find that the time required to evaluate parameter configurations can be reduced by more than 2/3 with low error. Overall, our results demonstrate that for a variety of motion planning problems it is possible to find solutions that significantly improve the performance over default configurations while requiring very reasonable computation times.

Fall risk assessments provide a useful tool to prevent morbidity and mortality provoked by falls. Nowadays, these assessments are usually performed manually by the medical staff. This approach has three main drawbacks: (i) it is time consuming, so it is only performed a few times per volunteer during their rehabilitation process; (ii) it requires supervision by medical staff, so assessment at home or preferred environments is not feasible; and (iii) fall risk is evaluated in a global way, so imminent fall risk is not available for decision making in assistive navigation. In this paper we propose an imminent fall risk estimator for rollator's users that can be automatically obtained on the fly. Its main advantages are: (i) it can be used in everyday conditions in any environment; (ii) it does not require assistance of medical staff; and (iii) it is suitable for a variety of users with minimal configuration changes. We have validated our estimator with a set of volunteers (n=10) presenting different physical and cognitive disabilities. Although the number of volunteer is limited, results show that our estimator is coherent to two traditional, well accepted assessments: the Tinetti Mobility Test and the walking speed.

In this paper, we address the problem of extrinsic calibration of a camera and a 3D Light Detection and Ranging (LiDAR) sensor using a checkerboard. Unlike previous works which require at least three checkerboard poses, our algorithm reduces the minimal number of poses to one by combining 3D line and plane correspondences. Besides, we prove that parallel planar targets with parallel boundaries provide the same constraints in our algorithm. This allows us to place the checkerboard close to the LiDAR so that the laser points better approximate the target boundary without loss of generality. Moreover, we present an algorithm to estimate the similarity transformation between the LiDAR and the camera for the applications where only the correspondences between laser points and pixels are concerned. Using a similarity transformation can simplify the calibration process since the physical size of the checkerboard is not needed. Meanwhile, estimating the scale can yield a more accurate result due to the inevitable measurement errors of the checkerboard size and the LiDAR intrinsic scale factor that transforms the LiDAR measurement to the metric measurement. Our algorithm is validated through simulations and experiments. Compared to the plane-only algorithms, our algorithm can obtain more accurate result by fewer number of poses. This is beneficial to the large-scale commercial application.

In this work we present a novel approach for multi-sensor calibration that significantly outperforms current state-of-the-art. We introduce a new spherical calibration target which has major benefits over existing targets. Those are subresolution detection accuracy in both camera and depth sensor, view invariance and applicability to a wider range of sensor setups than current approaches. With our method a single person achieves high quality calibration in less than a minute. No preparations for setting up the environment for calibration is needed. Our method is fast, easy to use and fully automatic. We evaluate our method in simulation and show high accuracy with an error of less than 3mm in translation and 0.1 0 in rotation on real data.

One of the major challenges of model predictive control (MPC) for robotic applications is the non-trivial weight tuning process while crafting the objective function. This process is often executed using the trial-and-error method by the user. Consequently, the optimality of the weights and the time required for the process become highly dependent on the skill set and experience of the user. In this study, we present a generic and user-independent framework which automates the tuning process by reinforcement learning. The proposed method shows competency in tuning a nonlinear MPC (NMPC) which is employed for trajectory tracking control of aerial robots. It explores the desirable weights within less than an hour in iterative Gazebo simulations running on a standard desktop computer. The real world experiments illustrate that the NMPC weights explored by the proposed method result in a satisfactory trajectory tracking performance.

Due to the widespread use of industrial robots in market, its application has extended to welding, painting, and freight handling. And tool coordinate calibration is regularly modified after tool replacement due to collision accident or routine maintenance. After tool replacement, operators often rebuild tool coordinates. This is the traditional mode of operation in the current industrial practices. However, smart factory will make artificial intelligence method replace manual method. This paper presents a system independent method for automatic calibration of the tool coordinate system which is faster, simpler, cheaper and more effective than the manual method. The proposed method required images to be captured using two “eye to hand” cameras and one “eye in hand” camera. Tool position data is then acquired through CamShift and MeanShift algorithm for image trajectory tracking along with coordinate system conversion, several methods like PCA, LDA can deal with the vision data. Optimal Deep Neural Network (DNN) method error compensation of a robot allows the tool to automatically run with the calibration system functions. We have developed a 6 degrees of freedom(DoF) industrial robot for this experiment. Nine different kinds of DNN models are built and finally with suitable tool coordinate error compensation for the current robot, tool calibration can be achieved adaptively and efficiently.

We describe a novel approach to image based localisation in urban environments which uses semantic matching between images and a 2-D cartographic map. This contrasts with the majority of existing approaches which use image to image database matching. We use highly compact binary descriptors to represent locations, indicating the presence or not of semantic features, which significantly increases scalability and has the potential for greater invariance to variable imaging conditions. The approach is also more akin to human map reading, making it better suited to human-system interaction. In this initial study we use semantic features relating to buildings and road junctions in discrete viewing directions. CNN classifiers are used to detect the features in images and we match descriptor estimates with location tagged descriptors derived from the 2-D map to give localisation. The descriptors are not sufficiently discriminative on their own, but when concatenated sequentially along a route, their combination becomes highly distinctive and allows localisation even when using non-perfect classifiers. Performance is further improved by taking into account left or right turns over a route. Experimental results obtained using Google StreetView and OpenStreetMap data show that the approach has considerable potential, achieving localisation accuracy of around 85% using routes corresponding to approximately 200 meters.

Microrobotic cargo delivery possesses promising perspective for precision medicine, and has attracted much attention recently. However, its automation remains challenging, especially with complex environmental conditions, such as obstacles and obstructed optical feedback. In this paper, we propose an automated control approach for a new microrobotic cargo carrier, i. e. the multifunctional magnetic spore (Mag-Spore). By surface functionalization of the spore with Fe3O4 nanoparticles and carbon quantum dots, it can be remotely actuated and tracked by an electromagnetic coil system and the fluorescence microscopy, respectively. Our strategy utilizes fluorescence imaging for vision feedback, which enhances the recognition and tracking of Mag-Spores and cells. Then, information of the cells and Mag-Spores for planning and control is identified via image processing, and an optimal path planner with obstacle avoidance capability is designed based on the Particle Swarm Optimization (PSO)algorithm. To make the Mag-Spore follow the planed path accurately, an observer-based trajectory tracking controller is synthesized. Simulations and experiments are conducted to demonstrate the effectiveness of the proposed control approach.

An efficient, generalizable physical simulator with universal uncertainty estimates has wide applications in robot state estimation, planning, and control. In this paper, we build such a simulator for two scenarios, planar pushing and ball bouncing, by augmenting an analytical rigid-body simulator with a neural network that learns to model uncertainty as residuals. Combining symbolic, deterministic simulators with learnable, stochastic neural nets provides us with expressiveness, efficiency, and generalizability simultaneously. Our model outperforms both purely analytical and purely learned simulators consistently on real, standard benchmarks. Compared with methods that model uncertainty using Gaussian processes, our model runs much faster, generalizes better to new object shapes, and is able to characterize the complex distribution of object trajectories.

In the robotic field, navigation and path planning applications benefit from a wide range of visual systems (e.g, perspective cameras, depth cameras, catadioptric cameras, etc.). In outdoor conditions, these systems capture information in which sky regions cover a major segment of the images acquired. However, sky regions are discarded and are not considered as visual cue in vision applications. In this paper, we propose to estimate attitude of Unmanned Aerial Vehicle (UAV) from sky information using a polarimetric camera. Theoretically, we provide a framework estimating the attitude from the skylight polarized patterns. We showcase this formulation on both simulated and real-word data sets which proved the benefit of using polarimetric sensors along with other visual sensors in robotic applications.

Convolutional neural networks have achieved wide success in RGB saliency detection. Recently, the advent of RGB-D sensors such as Kinect provide additional geometric saliency cues. However, the key challenge for RGB-D salient object detection that how to fuse RGB and depth information sufficiently is still under-studied. Traditional works mainly follow the two-stream architecture and combine RGB and depth features/decisions in an early or late point. The multi-modal fusion stage is performed by directly concatenating the features from two modalities without selection. In this work, we address this question by proposing a novel network with a distinguished insight: A selection module is significantly helpful for more informative and sufficient cross-modal cross-level combination. To this end, we introduce a top-down RGB-D fusion network which integrates an attention-aware cross-modal cross-level fusion block in each level to select discriminative features from each level and each modality. Extensive experiments on public datasets show that the proposed network is able to solve the key problems in RGB-D fusion and achieves state-of-the-art performance on RGB-D salient object detection.

Muscle-tissue of earthworms is an excellent actuator due to its membranous structure, strong force, short response time, and controllability. In this paper, we first investigated the output force, control property including stimulation voltage, stimulation frequency, and duty. Secondly, we designed, fabricated, and demonstrated an atmospheric-operable walking robot by using a muscle-tissue of earthworms. The maximum walking speed was about 0.56 mm/s, which is about 2 times faster than other types of bio actuated walker. The maximum atmospheric driven time was over 45 minutes. These demonstrated results indicated that the muscle-tissue of earthworm has a high potential for using as a biological micro bio-actuator for multiple purposes.

Associative Skill Memories (ASMs) were formulated to encode stereotypical movements along with their stereotypical sensory events to increase the robustness of underlying dynamic movement primitives (DMPs) against noisy perception and perturbations. In ASMs, the stored sensory trajectories, such as the haptic and tactile measurements, are used to compute how much a perturbed movement deviates from the desired one, and to correct the movement if possible. In our work, we extend ASMs: rather than using stored single sensory trajectory instances, our system generates sensory event models and exploits those models to correct the perturbed movements during executions with the aim of generalizing to novel configurations. In particular, measured force and the torque trajectories are modelled using Parametric Hidden Markov Models, and then reproduced by Gaussian Mixture Regression. With Baxter robot, we demonstrate that our proposed force feedback model can be used to correct a trajectory while pushing an object with a mass never experienced before, and which otherwise slips away from the gripper because of noise. In the end, we discuss how far this skill can be generalized using the force model and possible future improvements.

This article presents an approach for assisted teleoperation of a robot arm, formulated within a real-time stack-of-tasks (SoT)whole-body motion control framework. The approach leverages the hierarchical nature of the SoT framework to integrate operator commands with assistive tasks, such as joint limit and obstacle avoidance or automatic gripper alignment. Thereby some aspects of the teleoperation problem are delegated to the controller and carried out autonomously. The key contributions of this work are two-fold: the first is a method for unobtrusive integration of autonomy in a telemanip-ulation system; and the second is a user study evaluation of the proposed system in the context of teleoperated pick-and-place tasks. The proposed approach of assistive control was found to result in higher grasp success rates and shorter trajectories than achieved through manual control, without incurring additional cognitive load to the operator.