IROS 2018 Papers

We are faced with the problem of optimally placing a heterogeneous team of sensors and effector robots in an area while taking into account the environment, anticipated arrival traffic, and desired power consumption of the team. We stage the problems of anticipating arrival traffic and determining a proper power schedule as an adversarial game, incorporating our analysis of the game in the objective function which evaluates sensor positions. We obtain the set of sensor positions which performs best at the desired power consumption, evaluating the mixed strategy of sensor activity that best counters the anticipated potential arrival paths. To determine an approximate global optima for a large number of heterogeneous nodes, we employ Adaptive Simulated Annealing (ASA) to ensure our algorithm is flexible over a varied range of scenarios. We compare the proposed algorithm to a gradient-based greedy placement algorithm with a uniform power schedule within simulation.

We present an illustrative example of an interactive auditory perception approach performed by a humanoid robot called NICO, the Neuro Inspired COmpanion [1]. The video demonstrates a material classification task in the style of a classic TV game show. NICO and another candidate are supposed to determine the content of small plastic capsules that are visually indistinguishable. Shaking the capsules produces audio signals that range from rattling stones, over tinkling coins to swooshing sand. NICO can perceive and analyze these sounds to determine the material of the capsules content.

Robotic wheelchairs with built-in assistive features, such as shared control, are an emerging means of providing independent mobility to severely disabled individuals. However, patients often struggle to build a mental model of their wheelchair's behaviour under different environmental conditions. Motivated by the desire to help users bridge this gap in perception, we propose a novel augmented reality system using a Microsoft Hololens as a head-mounted aid for wheelchair navigation. The system displays visual feedback to the wearer as a way of explaining the underlying dynamics of the wheelchair's shared controller and its predicted future states. To investigate the influence of different interface design options, a pilot study was also conducted. We evaluated the acceptance rate and learning curve of an immersive wheelchair training regime, revealing preliminary insights into the potential beneficial and adverse nature of different augmented reality cues for assistive navigation. In particular, we demonstrate that care should be taken in the presentation of information, with effort-reducing cues for augmented information acquisition (for example, a rear-view display) being the most appreciated.

Robotic and computer assistance can bring significant benefits to endovascular procedures in terms of precision and stability, reduced radiation doses, improved comfort and access to difficult and tortuous anatomy. However, the design of current commercially available platforms tends to alter the natural bedside manipulation skills of the operator, so that the manually acquired experience and dexterity are not well utilized. Furthermore, most of these systems lack of haptic feedback, preventing their acceptance and limiting the clinical usability. In this paper a new robotic platform for endovascular catheterization, the CathBot, is presented. It is an ergonomic master-slave system with navigation system and integrated vision-based haptic feedback, designed to maintain the natural bedside skills of the vascular surgeon. Unlike previous work reported in literature, dynamic motion tracking of both the vessel walls the catheter tip is incorporated to create dynamic active constraints. The system was evaluated through a combined quantitative and qualitative user study simulating catheterization tasks on a phantom. Forces exerted on the phantom were measured. The results showed a 70% decrease in mean force and 61% decrease in maximum force when force feedback is provided. This research provides the first integration of vision-based dynamic active constraints within an ergonomic robotic catheter manipulator. The technological advances presented here, demonstrates that vision-based haptic feedback can improve the effectiveness, precision, and safety of robot-assisted endovascular procedures.

A human-in-the-loop system is proposed to enable hands-free collaborative manipulation for people with physical disabilities. Studies show that the cognitive burden of interfacing with a robotic assistant decreases with increased robot autonomy. Incorporating modern advances in perception with augmented reality, this paper describes a framework for obtaining high-level intents from the user to specify manipulation tasks for execution. Augmented reality glasses provide an egocentric perspective to the robot. The glasses also provide visual feedback to users on a virtual menu showing a summary of robot affordances. The system processes the vision input to interpret the users environment. A Tongue Drive System serves as the input modality for triggering task execution by the robotic arm. Several manipulation experiments are performed with comparison to Cartesian control. The outcomes are also compared to reported state-of-the-art approaches. The results demonstrate competitive performance with minimal user input requirements.

We present a robust real-time system for simultaneous detection of hands and faces in RGB and gray-scale images, and a novel dataset used for training. Our goal is to provide a robust sensor front-end suitable for real-time human-robot interaction using face-engagement and gestures. Using hand-labelled videos obtained from real human-UAV interaction experiments, we re-trained the YOLOv2 Deep Convolutional Neural Network to detect only hands and faces. This model was then used to automatically label several much larger third-party datasets. After manual correction of these results, we modified and re-trained the model on all this labelled data. We obtain qualitatively good detection results at 60Hz on a commodity GPU: our simultaneous hand-and-face detector gives state of the art accuracy and speed in a hand detection benchmark and competitive results in a face detection benchmark. To demonstrate its effectiveness for human-robot interaction we describe its use as the input to a simple but practical gestural human-UAV interface for entertainment or industrial applications. All software, training and test data are freely available.

This paper presents measurements of human hand-impedance during a laparoscopic training program with physically interactive robotic manipulators. The knowledge of how the hand-impedance changes due to training might be useful to inform better training programs and to introduce co-manipulated robotic assistants for effective trainings. Ten novice subjects participated in a three weeks training program for a suturing activity in laparoscopy. The subjects have been instructed to set the needle, enter the skin, and tie knots by using laparoscopic tools within a Minimally Invasive Surgery training box. Variable admittance controlled robots, attached to the tools with force sensors, applied step vice velocity disturbances while subjects were trying to set the needle. Based on the interaction force and end-effector position information, impedances of the left and right hands were computed in four different directions. The computed results were compared with respect to the participants skill progression.

Many mobile robots rely on 2D laser scanners for localization, mapping, and navigation. However, those sensors are unable to correctly provide distance to obstacles such as glass panels and tables whose actual occupancy is invisible at the height the sensor is measuring. In this work, instead of estimating the distance to obstacles from richer sensor readings such as 3D lasers or RGBD sensors, we present a method to estimate the distance directly from raw 2D laser data. To learn a mapping from raw 2D laser distances to obstacle distances we frame the problem as a learning task and train a neural network formed as an autoencoder. A novel configuration of network hyperparameters is proposed for the task at hand and is quantitatively validated on a test set. Finally, we qualitatively demonstrate in real time on a Care-O-bot 4 that the trained network can successfully infer obstacle distances from partial 2D laser readings.

HTC Vive has been gaining attention as a cost-effective, off-the-shelf tracking system for collecting ground truth pose data. We assess this system's pose estimation through a series of controlled experiments where we show its precision to be in the millimeter magnitude and accuracy to range from millimeter to meter. We also show that Vive gives greater weight to inertial measurements in order to produce a smooth trajectory for virtual reality applications. Hence, the Vive's off the shelf algorithm is poorly suited for robotics applications such as measuring ground truth poses, where accuracy and repeatability are key. Therefore we introduce a new open-source tracking algorithm and calibration procedure for Vive which address these problems. We also show that our approach improves the pose estimation repeatability and accuracy by up to two orders of magnitude.

This paper presents a hybrid 3D-like grid-based mapping approach, that we called HMAP, used as a reliable and efficient 3D representation of the environment surrounding a mobile robot. Considering 3D point-clouds as input data, the proposed mapping approach addresses the representation of height-voxel (HVoxel) elements inside the HMAP, where free and occupied space is modeled through HVoxels, resulting in a reliable method for 3D representation. The proposed method corrects some of the problems inherent to the representation of complex environments based on 2D and 2.5D representations, while keeping an updated grid representation. Additionally, we also propose a complete pipeline for SLAM based on HMAPs. Indoor and outdoor experiments were carried out to validate the proposed representation using data from a Microsoft Kinect One (indoor) and a Velodyne VLP-16 LiDAR (outdoor). The obtained results show that HMAPs can provide a more detailed view of complex elements in a scene when compared to a classic 2.5D representation. Moreover, validation of the proposed SLAM approach was carried out in an outdoor dataset with promising results, which lay a foundation for further research in the topic.

The Heterogeneous Extensible Robot Open (HERO) platform is designed for autonomous robotic research. While bringing in the flexible computational capacities by CPU and FPGA, it addresses the challenges of heterogeneous computing by embracing OpenCL programming. We propose heterogeneous computing approaches for three fundamental robotic tasks: simultaneous localization and mapping (SLAM), motion planning and convolutional neural network (CNN) inference. With FPGA acceleration, the SLAM and motion planning tasks are performed 2–4 times faster on the HERO platform against fine-tuned software implementation. For CNN inference, it can process 20–30 images per second with the network of VGG-16 or ResNet-50. We expect the open platform and the developing experiences shared in this paper can facilitate future robotic research, especially for those compute intensive tasks of perception, movement and manipulation.

This paper introduces the design of a novel under-actuated hand with highly integrated modular finger units, which can be easily reconfigured in terms of finger arrangement and number to account for the manipulation needs of different applications. Each finger module is powered by a single actuator through an under-actuated transmission and equipped with a sensory system for delicate and precise grasping, which includes absolute position measurements, contact pressure sensing at finger phalanxes and motor current readings. Finally, intrinsic elasticity integrated in the transmission system make the hand robust and adaptive to impacts when interacting with the objects and environment. This highly integrated hand (HERI II) was developed for the Centauro Robot to enable robust and resilient manipulation. A set of experiments demonstrating the hand's grasping performance were carried out and fully verified the design effectiveness of the proposed hand.

This paper addresses real-time bird song scene analysis. Observation of animal behavior such as communication of wild birds would be aided by a portable device implementing a real-time system that can localize sound sources, measure their timing, classify their sources, and visualize these factors of sources. The difficulty of such a system is an integration of these functions considering the real-time requirement. To realize such a system, we propose a cascaded approach, cascading sound source detection, localization, separation, feature extraction, classification, and visualization for bird song analysis. Our system is constructed by combining an open source software for robot audition called HARK and a deep learning library to implement a bird song classifier based on a convolutional neural network (CNN). Considering portability, we implemented this system on a single-board computer, Jetson TX2, with a microphone array and developed a prototype device for bird song scene analysis. A preliminary experiment confirms a computational time for the whole system to realize a real-time system. Also, an additional experiment with a bird song dataset revealed a trade-off relationship between classification accuracy and time consuming and the effectiveness of our classifier.

We consider pursuit-evasion games in which the pursuer is tasked with intercepting the evader using only partial measurements. Motivated by the utilization of visual sensing on board the pursuer, we focus on the case when only bearing measurements are available to the pursuer. The resulting partially-observable interception problem is computationally challenging, and the separation principle does not hold in general. In this paper, we identify a set of maneuvers that improve observability, and we propose an algorithm that utilizes these maneuvers to move the pursuer so that the expected payoff of the differential game is maximized. The algorithm uses in-the-loop uncertainty propagation based on linear covariance analysis to assess the effect of the maneuvers. We evaluate the resulting guidance law in experiments involving a quadcopter in flight representing the pursuer, and a simulated evader.

Interpreting the brain activity to identify user goals or to ground a robot's hypotheses about them is a promising direction for non-intrusive and intuitive communication. Such a capability can be of particular relevance in the context of human-robot cooperation scenarios. This paper proposes a novel approach to utilize the natural brain responses to highlighted objects in the scene for object selection. By this, it circumvents the need for additional interfaces or user training. Our approach uses methods from information geometry to classify the target/non-target response of these event-related potentials. Online experiments carried out with a real robot demonstrate an accurate detection of target objects solely based on the user's attention.

We consider the problem of coverage planning for a particular type of very simple mobile robot. The robot must be able to translate in a commanded direction (specified in a global reference frame), with bounded error on the motion direction, until reaching the environment boundary. The objective, for a given environment map, is to generate a sequence of motions that is guaranteed to cover as large a portion of that environment as possible, in spite of the severe limits on the robot's sensing and actuation abilities. We show how to model the knowledge available to this kind of robot about its own position within the environment, show how to compute the region whose coverage can be guaranteed for a given plan, and characterize regions whose coverage cannot be guaranteed by any plan. We also describe a heuristic algorithm that generates coverage plans for this robot, based on a search across a specially-constructed graph. Simulation results demonstrate the effectiveness of the approach.

In this paper, a system to determine the emotion of a group of people via facial expression analysis is proposed for the Waseda Entertainment Robots. General models and standard methods for emotion definition and recognition are briefly described, as well as strategies for computing the group global emotion, knowing the individual emotions of group members. This work is based on Ekman's extended “Big Six” emotional model, popular in Computer Science and Affective Computing. Emotion recognition via facial expression analysis is performed with a cloud-computing based solution, using Microsoft Azure Cognitive services. First, the performances of both the Face API to detect faces, and Emotion API, to compute emotion via face expression analysis, are tested. After that, a solution to compute the emotion of a group of people has been implemented and its performances compared to human perceptions. This work presents concepts and strategies which can be generalized for applications within the scope of assistive robotics and, more broadly, affective computing, wherever it will be necessary to determine the emotion of a group of people.

Reducing material spillage by robotic mining mobile manipulators, such as front-end loaders, is necessary to improve mining operations. To this end, the present work proposes an approach to reduce disturbances on the end-effector induced by the terrain and propagated through the wheels and arm links of the machine. The proposed approach is based on an H∞ control strategy that includes a feedforward action, computed using the pitch rate of the mobile base, and considers the hydraulic arm dynamics, as well as the reaction forces in the contact points of the mobile base, which is modeled as a floating body with non-permanent ground contacts. Alternative control schemes based on the classic proportional-derivative (PD) control, and the Active Disturbance Rejection Control (ADRC), with and without feedforward action, were also implemented and experimentally evaluated using a semiautonomous Cat® 262C compact skid-steer loader equipped with inclination and inertial sensors. The proposed method reduces disturbances by at least 70% when climbing ramps at 25% of the machine's maximum speed, and by at least 20% when driving over speed bumps which produce disturbances similar to that caused by stones. The proposed disturbance attenuation strategy should help reducing the spillage of material when driving over mounds, inclines or spilled rocks, especially considering that even if existing autonomous machines are able to drive with little operator supervision along mining galleries, they are often unable to avoid disturbing material on the ground or the characteristic unevenness of mining terrains.

This paper presents the motion planning framework for a hexapod, based on advanced motions, for accessing challenging spaces, namely narrow pathways and large holes, both of which are surrounded by walls. The advanced motions, wall and chimney walking, utilise environment surfaces that are perpendicular to the ground plane to support the robot motion. Such techniques have not yet been studied in the literature. The hierarchical planning framework proposed here is an extension to existing approaches which have only considered ground walking where foothold contacts are confined to the ground plane. During the pre-processing phase of the 2.5D grid map, the motion primitives employed are assessed for each cell and stacked to the graph if valid. The A* algorithm is then used to find a path to the goal position. Following that, the path is post-processed to smoothen the motions and generate a continuous path. Footholds are then selected along the path. The framework has been evaluated in simulation on the custom-designed Corin hexapod. The resulting path enables access to areas that are previously thought to be inaccessible and reduces the travelling distance compared to previous studies.