IROS 2018 Papers

In this paper, we introduce an innovative method to improve the convergence speed and accuracy of object detection neural networks. Our approach, Converge-fast-auxnet, is based on employing multiple, dependent loss metrics and weighting them optimally using an on-line trained auxiliary network. Experiments are performed in the well-known RoboCup@Work challenge environment. A fully convolutional segmentation network is trained on detecting objects' pickup points. We empirically obtain an approximate measure for the rate of success of a robotic pickup operation based on the accuracy of the object detection network. Our experiments show that adding an optimally weighted Euclidean distance loss to a network trained on the commonly used Intersection over Union (IoU) metric reduces the convergence time by 42.48%. The estimated pickup rate is improved by 39.90%. Compared to state-of-the-art task weighting methods, the improvement is 24.5% in convergence, and 15.8% on the estimated pickup rate.

This paper introduces the problem of long-range monocular depth estimation for outdoor urban environments. Range sensors and traditional depth estimation algorithms (both stereo and single view) predict depth for distances of less than 100 meters in outdoor settings and 10 meters in indoor settings. The shortcomings of outdoor single view methods that use learning approaches are, to some extent, due to the lack of long-range ground truth training data, which in turn is due to limitations of range sensors. To circumvent this, we first propose a novel strategy for generating synthetic long-range ground truth depth data. We utilize Google Earth images to reconstruct large-scale 3D models of different cities with proper scale. The acquired repository of 3D models and associated RGB views along with their long-range depth renderings are used as training data for depth prediction. We then train two deep neural network models for long-range depth estimation: i) a Convolutional Neural Network (CNN) and ii) a Generative Adversarial Network (GAN). We found in our experiments that the GAN model predicts depth more accurately. We plan to open-source the database and the baseline models for public use.

Handling safety is crucial to achieve lifelong autonomy for robots. Unsafe situations might arise during manipulation in unstructured environments due to noises in sensory feedback, improper action parameters, hardware limitations or external factors. In order to assure safety, continuous execution monitoring and failure detection procedures are mandatory. To this end, we present a multimodal failure monitoring and detection system to detect manipulation failures. Rather than relying only on a single sensor modality, we consider integration of different modalities to get better detection performance in different failure cases. In our system, high level proprioceptive, auditory and visual predicates are extracted by processing each modality separately. Then, the extracted predicates are fused. Experiments on a humanoid robot for tabletop manipulation scenarios indicate that the contribution of each modality is different depending on the action in execution and multimodal fusion results in an overall performance increase in detecting failures compared to the performance attained by unimodal processing.

Small-scale robots with soft joints and hinges have recently attracted interest because these components allow for more sophisticated locomotion mechanisms. Here, we investigate two different types of nanoscale swimmers as depicted in Figure 1. One consists of a rigid magnetic head linked to a semi-soft tail (1-link swimmer). Another consists of a rigid magnetic head and tail connected by a soft hinge (2-link swimmer). Both swimmers exhibit undulatory locomotion under an applied oscillating magnetic field. The speeds of the swimmers are assessed as a function of the oscillating magnetic field frequency and the sweeping angle. We find that a resonance-like frequency increases as the length decreases, and, in general, the speed increases as the sweeping angle increases. Last, we show that 2-link swimmers can also swim in a corkscrew-like pattern under rotating magnetic fields.

Line extraction is a preliminary step in various visual robotic tasks performed in low textured scenes such as city and indoor settings. Several efficient line segment detection algorithms such as LSD and EDLines have recently emerged. However, the state of the art segment grouping methods are not robust enough or not amenable for detecting lines in real-time. In this paper we present FSG, a fast and robust line detection algorithm. It is based on two independent components. A proposer that greedily cluster segments suggesting plausible line candidates and a probabilistic model that decides if a group of segments is an actual line. In the experiments we show that our procedure is more robust and faster than the best methods in the literature and achieves state-of-the art performance in a high level robot localization task such as vanishing points detection.

Robot control algorithms often rely on measurements of robot joint velocities, which can be estimated by measuring the time between encoder edges. When encoder edges occur infrequently, such as at low velocities and/or with low resolution encoders, this measurement delay may affect the stability of closed-loop control. This is evident in both the joint position control and Cartesian impedance control of the da Vinci Research Kit (dVRK), which contains several low-resolution encoders. We present a hardware-based method that gives more frequent velocity updates and is not affected by common encoder imperfections such as non-uniform duty cycles and quadrature phase error. The proposed method measures the time between consecutive edges of the same type but, unlike prior methods, is implemented for the rising and falling edges of both channels. Additionally, it estimates acceleration to enable software compensation of the measurement delay. The method is shown to improve Cartesian impedance control of the dVRK.

Both Optimal Control and Search-based Planning are used extensively for path planning and have their own set of advantages and disadvantages. In this paper, we propose an algorithm FOCS (Fusion of Optimal Control and Search) that combines these two classes of approaches together. FOCS finds a path exploiting the advantages of both approaches while providing a bound on the sub-optimality of its solution. The returned path is a concatenation of the path found in the implicit graph constructed by search and the path generated by following the negative gradient of the value function obtained as a solution of the Hamilton-Jacobi-Bellman equation. We analyze the algorithm and illustrate its effectiveness in finding a minimum-time path for a car-like vehicle in different environments.

In this paper, a method for dexterous manipulation of 3D soft objects for real-time deformation control is presented, relying on Finite Element modelling. The goal is to generate proper forces on the fingertips of an anthropomorphic device during in-hand manipulation to produce desired displacements of selected control points on the object. The desired motions of the fingers are computed in real-time as an inverse solution of a Finite Element Method (FEM), the forces applied by the fingertips at the contact points being modelled by Lagrange multipliers. The elasticity parameters of the model are preliminarly estimated using a vision system and a force sensor. Experimental results are shown with an underactuated anthropomorphic hand that performs a manipulation task on a soft cylindrical object.

Tracking and controlling the shape of continuum dexterous manipulators (CDM) in constraint environments is a challenging task. The imposed constraints and interaction with unknown obstacles may conform the CDM's shape and therefore demands for shape sensing methods which do not rely on direct line of sight. To address these issues, we integrate a novel Fiber Bragg Grating (FBG) shape sensing unit into a CDM, reconstruct the shape in real-time, and develop an optimization-based control algorithm using FBG tip position feedback. The CDM is designed for less-invasive treatment of osteolysis (bone degradation). To evaluate the performance of the feedback control algorithm when the CDM interacts with obstacles, we perform a set of experiments similar to the real scenario of the CDM interaction with soft and hard lesions during the treatment of osteolysis. In addition, we propose methods for identification of the CDM collisions with soft or hard obstacles using the jacobian information. Results demonstrate successful control of the CDM tip based on the FBG feedback and indicate repeatability and robustness of the proposed method when interacting with unknown obstacles.

For a deeper understanding of ecological functions and semantics of wild bird vocalizations (i.e., songs and calls), it is important to clarify the fine-scaled and detailed relationships among their characteristics of vocalizations and their behavioral contexts. However, it takes a lot of time and effort to obtain such data using conventional recordings or by human observation. Bringing out a robot to a field is our approach to solve this problem. We are developing a portable observation system called HARKBird using a robot audition HARK and microphone arrays to understand temporal patterns of vocalizations characteristics and their behavioral contexts. In this paper, we introduce a prototype system to 2D localize vocalizations of wild birds in real-time, and to classify their song types after recording. We show that the system can estimate the position of songs of a target individual and classify their songs with a reasonable quality to discuss their song - behavior relationships.

Financial and social elements of modern societies are closely connected to the cultivation of corn. Due to its massive production, deficiencies during the cultivation process directly translate to major financial losses. Since proper surveillance in a large scale is still very challenging, the companies that specialize in optimizing crop yield are trying to address the problem at its root by developing hybrid plants able to resist the harsh conditions of the field. The selection of the best hybrid is not easy and every year hundreds of test plants with different phenotypic characteristics are planted while their performance is quantified by inconsistent and rough measurements gathered by humans. We propose a pipeline that takes advantage of the structure from motion technology to create a detailed 3D point cloud of a few plants and segment it into the basic elements of the scene; the ground, the plants, the plant stems, and the plant leaves. The focus is on the segmentation process through which several phenotypic characteristics of individual plants can be extracted. As an example, we show the results for the plant counting and plant height estimation processes where we achieve an accuracy of 88.1% and 89.2%.

This paper describes a novel sensorless force estimation algorithm for the rigid link parallel wrist mechanism of a robotic surgical instrument. The method utilizes novel reaction force observers (RFOB) in joint space, which are modified disturbance observers (DOB) combined with Neural Networks (NN) for inverse dynamics calculations, to estimate external forces acting on the motors. External force/torque estimation in Cartesian space is achieved by the use of the robot Jacobian. The proposed algorithm is applicable to any back-drivable rigid-link wrist mechanism without the need for force sensors. In this paper, the method is implemented on a novel 3 degree-of-freedom (DOF) parallel robotic surgical wrist mechanism that is designed for high dexterity (±90 degrees pitch-yaw rotations, thrust motion) and force/torque estimation. The wrist is actuated extracorporally with 3 rigid push-pull rods and 3 linear motors. With a rigid transmission and high back-drivability, external force/torque estimation can be achieved from the motor position readings utilizing the proposed method. Several experiments were performed on the manufactured prototype of the instrument and results validate the efficacy of the wrist and estimation method with RMS force/torque estimation error values of 0.0024 Nm in pitch axis, 0.0043 Nm in yaw axis and 0.1866 N in thrust axis.

Studying and understanding the computational primitives of our neural system requires for a diverse and complementary set of techniques. In this work, we use the Neuro-robotic Platform (NRP)to evaluate the vestibulo ocular cerebellar adaptatIon (Vestibulo-ocular reflex, VOR)mediated by two STDP mechanisms located at the cerebellar molecular layer and the vestibular nuclei respectively. This simulation study adopts an experimental setup (rotatory VOR)widely used by neuroscientists to better understand the contribution of certain specific cerebellar properties (i.e. distributed STDP, neural properties, coding cerebellar topology, etc.)to r-VOR adaptation. The work proposes and describes an embodiment solution for which we endow a simulated humanoid robot (iCub)with a spiking cerebellar model by means of the NRP, and we face the humanoid to an r-VOR task. The results validate the adaptive capabilities of the spiking cerebellar model (with STDP)in a perception-action closed-loop (r- VOR)causing the simulated iCub robot to mimic a human behavior.

Tactile sensing of surface normals is essential for exploration of unknown objects. Many tactile sensors have been developed for contact measurement. However, few of these sensors provide surface orientation, and only up to a limited degree. This paper presents a novel contact and surface orientation sensor concept and its application for surface reconstruction of unknown objects. The sensor is comprised of an Inertial Measurement Unit (IMU) and a pressure sensor to accurately estimate the surface orientation in a wide range, while at the same time measuring contact force. We describe the developed sensor prototype and evaluate its performance regarding contact detection capability and normal estimation accuracy. We use this to reconstruct the surface of unknown objects using the humanoid robot ARMAR-III resulting in a mean reconstruction accuracy of 3.6 mm.

Camera relocalization plays a vital role in many robotics and computer vision applications, such as self-driving cars and virtual reality. Recent random forests based methods exploit randomly sampled pixel comparison features to predict 3D world locations for 2D image locations to guide the camera pose optimization. However, these point features are only sampled randomly in images, without considering geometric information such as lines, leading to large errors with the existence of poorly textured areas or in motion blur. Line segments are more robust in these environments. In this work, we propose to jointly exploit points and lines within the framework of uncertainty driven regression forests. The proposed approach is thoroughly evaluated on three publicly available datasets against several strong state-of-the-art baselines in terms of several different error metrics. Experimental results prove the efficacy of our method, showing superior or on-par state-of-the-art performance.

A common architecture for torque controlled humanoid robots consists in two nested loops. The outer loop generates desired joint/motor torques, and the inner loop stabilizes these desired values. In doing so, the inner loop usually compensates for joint friction phenomena, thus removing their inherent stabilizing property that may be also beneficial for high level control objectives. This paper shows how to exploit friction for joint and task space control of humanoid robots. Experiments are carried out on the humanoid robot iCub.

In this paper, an Expert-Guided Kinodynamic RRT algorithm (EGK-RRT) is presented. It aims to consider how a human pilot would navigate a kinodynamic robot. One of the characteristics of this algorithm is the fact that, unlike the original RRT for kinodynamic systems, it generates deterministic control sequences which can be reproduced as long as the sequence of references (sampled states) are known. Here, the performance of the proposed algorithm is tested against the basic RRT, showing that the EGK-RRT greatly improves in terms of execution speed. In addition to this, the influence of using a visibility check and an inertia estimation in order to select the nearest neighbor is also analyzed, demonstrating that a combination of both factors leads to a better overall performance, both in execution speed and in quality of the generated path.

Learning approaches have enabled significant performance improvements in robotic control allowing robots to execute motions that were previously impossible. The majority of the work to date, however, assumes that the parts to be learned are static or slowly changing, which limits their applicability in realistic scenarios with rapid changes in the conditions. This paper presents a method to extend an existing single-mode safe learning controller based on Gaussian Process Regression to learn an increasing number of non-linear models for the robot dynamics. We show that this approach enables a robot to re-use past experiences from a large number of previously visited operating conditions, and to safely adapt when a new and distinct operating condition is encountered. This allows the robot to achieve safety and high performance in a large number of operating conditions that do not have to be specified ahead of time. Our approach runs independently from the controller, imposing no additional computation time on the control loop regardless of the number of previous operating conditions considered. We demonstrate the effectiveness of our approach in experiment on a 900 kg ground robot with both physical and artificial changes to its dynamics. All of our experiments are conducted using vision for localization.

Hiroshi Ishiguro gave a lecture to a group of young students. We employed CommU, the desktop social robot, to manage the questions and answers for the talk. We encouraged the students to ask questions anytime. Half of the classroom was told to ask questions by raising their hand while the other half was shown an online messaging system developed for CommU, which allows the audience to post questions, which the robot would directly say. We had a gatekeeper to monitor for invalid sentences. In the middle of the presentation we asked the students to shift roles. The robot used a neural network based estimator of interruptibility to find the best time to speak. We did not expect too many questions, but the audience really embraced using the robot. They posted 44 questions to the presenter through CommU. On the other hand they asked 8 direct questions by raising their hands and standing up. Students thought that they gained more information from the lecturer using the robot than using the conventional method. In this instance we didn't stop the students from asking too many questions, but in a real-world application the gatekeeper will have to play an important role.