IROS 2018 Papers

To navigate through urban roads, an automated vehicle must be able to perceive and recognize objects in a three-dimensional environment. A high level contextual understanding of the surroundings is necessary to execute accurate driving maneuvers. This paper presents a novel approach to build three dimensional semantic octree maps from lidar scans and the output of a convolutional neural network (CNN) to obtain the labels of the environment. We present a heuristic method to associate uncertainties to the labels from the images based on a combination of the labels themselves, score maps retrieved by the CNN and the raw images. These uncertainties and the camera-lidar calibration parameters for multiple cameras are considered in the projection of the labels and their uncertainties into the point cloud. Every labeled lidar scan works as an input to an octree map building algorithm that calculates and updates the label probabilities of the voxels in the map. This paper also presents a qualitative and quantitative evaluation of accuracy, analyzing projection in single lidar scans and complete maps built with our probabilistic octree framework.

We propose a stereo-based obstacle detection approach for unmanned surface vehicles. Obstacle detection is cast as a scene semantic segmentation problem in which pixels are assigned a probability of belonging to water or non-water regions. We extend a single-view model to a stereo system by adding a constraint which prefers consistent class labels assignment to pixels in the left and right camera images corresponding to the same parts of a 3D scene. Our approach jointly fits a semantic model to both images, leading to an improved class-label posterior map from which obstacles and water edge are extracted. In overall F-measure, our approach outperforms the current state-of-the-art monocular approach by 0.495, a monocular CNN by 0.798 and their stereo extensions by 0.059 and 0.515, respectively on the task of obstacle detection while running real-time on a single CPU.

This paper investigates tactile object recognition with relatively densely distributed force vector measurements and evaluates what kind of tactile information is beneficial for object recognition. The uSkin tactile sensors are embedded in an Allegro Hand, and provide 240 triaxial force vector measurements in total in all fingers. Active object sensing is used to gather time-series training and testing data. A simple feedforward, a recurrent, and a convolutional neural network are used for recognizing objects. Evaluations with different number of employed measurements, static vs. time series data and force vector vs. only normal force vector measurements show that the high-dimensional information provided by the sensors is indeed beneficial. An object recognition rate of up to 95% for 20 objects was achieved.

Instance-based object detection and fine pose estimation is an active research problem in computer vision. While the traditional interest-point-based approaches for pose estimation are precise, their applicability in robotic tasks relies on controlled environments and rigid objects with detailed textures. CNN-based approaches, on the other hand, have shown impressive results in uncontrolled environments for more general object recognition tasks like category-based coarse pose estimation, but the need of large datasets of fully-annotated training images makes them unfavourable for tasks like instance-based pose estimation. We present a novel approach that combines the robustness of CNNs with a fine-resolution instance-based 3D pose estimation, where the model is trained with fully-annotated synthetic training data, generated automatically from the 3D models of the objects. We propose an experimental setup in which we can carefully examine how the model trained with synthetic data performs on real images of the objects. Results show that the proposed model can be trained only with synthetic renderings of the objects' 3D models and still be successfully applied on images of the real objects, with precision suitable for robotic tasks like object grasping. Based on the results, we present more general insights about training neural models with synthetic images for application on real-world images.

This work examines how and to what benefit an autonomous humanoid robot can supervise a child in an object assembly task. In order to understand the child's actions, a novel 3D object tracking algorithm for RGB-D data is employed. The tracker consists of two stages: the first performs a tracking-by-detection scheme on the color stream, to locate the objects on the image plane, while the second uses a particle filter that operates on the depth data stream to refine the first stage output and infer the objects' rotations. Given the six degrees-of-freedom of the assembly part poses, the system is able to recognize which connections have been completed at any given time. This information is then used to select an appropriate verbal or gestural response for the robot. Experimental results show that (a) the tracking algorithm is accurate, fast and robust to severe occlusions and fast movements, (b) the proposed method of assembly state estimation is indeed effective, and (c) the resulting Child-Robot Interaction scenario is educational and enjoyable for the children involved.

This paper explores the control strategy for a multi-limb robot nonprehensilely pushing an object to slide on the floor. The robot's limb distals perform point contacts with the object and the floor. The contact velocity constraint and force constraint are proposed to prevent separation and restrict the system forces. Then the constraints are combined with the system dynamic models to obtain bounds on the system states. We solve the motion planning problem by selecting a feasible path in the reduced-dimensional space and generating the system trajectory along the selected path. An example is provided to illustrate the application of our technique on the physical platform.

With the purpose of achieving stable and energy efficient locomotion on the slippery road surface, a sliding locomotion robot without joint torque but indirectly controlled by an active wobbling mass is recently proposed. In this paper, we deepen the analysis of the mechanism of the indirectly controlled sliding locomotion for further optimization and generalization. First, we derive the equations of dynamics and control. Second, we estimate the natural frequency of the robot, the moving speed and energy efficiency are also evaluated with respect to forcing amplitude and frequency of the wobbling mass. Third, the Arnol'd tongue is introduced to analyze the relationship between achieving efficient locomotion and being entrained. In addition, phase oscillation and synchronization phenomenon are analyzed via hysteresis plot to further interpret the unusual shapes of the Arnol'd tongues. Finally, we analyze the entrained, however, inefficient locomotion by reconfirming the rolling constraints from the mechanical energy dissipation point of view. Our results help better understanding of the indirectly controlling mechanism, and the methods can be applied to other indirectly controlled locomotion robots.

The dynamics of quadrotors are affected by time-varying torque latency, which can greatly alter the stability robustness and performance of the closed-loop control schemes employed for flight; this issue is especially relevant during the execution of aerobatic maneuvers such as high-speed multi-flips. To address this problem, we propose two controller synthesis methods associated with two different modeling approaches. In the first approach, we describe torque latency with a linear time-invariant (LTI)model, identified through ground experiments, which is then used to design a backstepping-based nonlinear controller. In the second approach, we employ an improved linear time-varying (LTV)model with a priori unknown parameters, which is used to synthesize and implement a novel nonlinear adaptive control scheme updated in real time using the recursive least-squares (RLS)algorithm. Empirical observations suggest that the torque delay affecting the system depends on the time-varying angular speed of the flyer and its derivative. This phenomenon is explained by the fact that the aerodynamic forces produced by, and acting on, the rotating propellers vary with the local velocity of the incident flows. Hence, in the proposed adaptive structure, we define the parameters of the LTV latency model as linear functions of the angular speed reference and its derivative. Experimental results compellingly demonstrate the efficacy of the methods introduced in this paper; compared to the highperformance linear controller in [1]-[3], the backstepping-based control scheme and adaptive controller decrease the average root mean square (RMS)value of the control error by 17.82 % and 38.42 %, respectively.

FES allows spinal cord injured individuals to propel tricycles by means of their own leg power. The stimulation patterns are in most of the cases predefined and muscle activation triggered on the basis of the pedal position. This requires an empirical tuning to fit the pattern to the pilot sitting position and distance to crank with no possible generalization and no adaptive properties. The aim of the present article is to introduce a new approach of motion segmentation based on inertial measurement units located on the cyclist legs with the final aim to predict the optimal pedaling force evolution. Results obtained with one healthy subject in different cycling conditions are presented and the application to FES-cycling discussed.

The present study highlights the benefits of using well-controlled experimental designs, grounded in experimental psychology research and objective neuroscientific methods, for generating progress in human-robot interaction (HRI) research. More specifically, we aimed at implementing a well-studied paradigm of attentional cueing through gaze (the so-called “joint attention” or “gaze cueing”) in an HRI protocol involving the iCub robot. Similarly to documented results in gaze-cueing research, we found faster response times and enhanced event-related potentials of the EEG signal for discrimination of cued, relative to uncued, targets. These results are informative for the robotics community by showing that a humanoid robot with mechanistic eyes and human-like characteristics of the face is in fact capable of engaging a human in joint attention to a similar extent as another human would do. More generally, we propose that the methodology of combining neuroscience methods with an HRI protocol, contributes to understanding mechanisms of human social cognition in interactions with robots and to improving robot design, thanks to systematic and well-controlled experimentation tapping onto specific cognitive mechanisms of the human, such as joint attention.

Motor functions of the biological system has been forged through 4 billion years evolution. From a neurorobotics view, it is important not only to know how well it works, but also how it fails. To quantitatively describe early onset symptoms of a neurodegenerative disease, we analyzed phenotypes of genetically engineered Huntington disease (HD) model mice, which reveal progressive impaired motor functions. We devised a simple yet sensitive paradigm called the crystalized motion profile (CMP), by which we successfully detected subtle difference between normal and abnormal mice in terms of whole-body level motor coordination. Our long-term objective is to remodel human mind and body to regain impaired motor and cognitive functions with ageing. To do so, we are developing a soft neurorobotic suit that provides integrated cognitive and physical interventions to users. Our analysis on the HD model mice is important as the first step to bridge between molecular mechanisms (altered genetic code) and the macroscopic neuro-musculoskeletal model. With this, we can extrapolate from knowledge of non-human mammals to human to derive the remodeling.

To generate dynamic motions such as hopping and running on legged robots, model-based approaches are usually used to embed the well studied spring-loaded inverted pendulum (SLIP) model into the whole-body robot. In producing controlled SLIP-like behaviors, existing methods either suffer from online incompatibility or resort to classical interpolations based on lookup tables. Alternatively, this paper presents the application of a data-driven approach which obviates the need for solving the inverse of the running return map online. Specifically, a deep neural network is trained offline with a large amount of simulation data based on the SLIP model to learn its dynamics. The trained network is applied online to generate reference foot placements for the humanoid robot. The references are then mapped to the whole-body model through a QP-based inverse dynamics controller. Simulation experiments on the WALK-MAN robot are conducted to evaluate the effectiveness of the proposed approach in generating bio-inspired and robust running motions.

We present a novel concept for teach-and-repeat visual navigation. The proposed concept is based on a mathematical model, which indicates that in teach-and-repeat navigation scenarios, mobile robots do not need to perform explicit localisation. Rather than that, a mobile robot which repeats a previously taught path can simply “replay” the learned velocities, while using its camera information only to correct its heading relative to the intended path. To support our claim, we establish a position error model of a robot, which traverses a taught path by only correcting its heading. Then, we outline a mathematical proof which shows that this position error does not diverge over time. Based on the insights from the model, we present a simple monocular teach-and-repeat navigation method. The method is computationally efficient, it does not require camera calibration, and it can learn and autonomously traverse arbitrarily-shaped paths. In a series of experiments, we demonstrate that the method can reliably guide mobile robots in realistic indoor and outdoor conditions, and can cope with imperfect odometry, landmark deficiency, illumination variations and naturally-occurring environment changes. Furthermore, we provide the navigation system and the datasets gathered at www.github.com/gestom/stroll_bearnav.

Albatross has an energy efficient flying pattern (dynamic soaring) among seabirds. This interesting point encourages us to exploit its flying natural dynamics so as to control the flying robots on energy efficient and robust gaits. In doing so, we study the albatross dynamic soaring from analytical and biological perspectives and realize that to generate the dynamic soaring instead of trajectory control, the mechanical energy should be regulated. Accordingly, the control objective is set to mechanical energy regulation, and the bank angle and lift coefficient are computed to satisfy this objective. The presented method is simulated on a standard albatross model and generates two different types of dynamic soaring; O-shaped and a-shaped patterns. In addition, by means of simulations, it is investigated that the presented method is robust in face of variations in initial conditions and unexpected disturbances in the environment's model; i.e., they cannot disturb the stability and cyclic behavior of the system. Moreover, the simulation results are compared with pieces of natural evidence from albatross and interesting similarities are observed.

In this manuscript, a NDVI point cloud generator tool based on low-cost active RGB-D sensor is presented. Taking advantage of currently available ROS point cloud generation tools and RGB-D sensor technology (like Microsoft Kinect), that includes an inbuilt active IR camera and a RGB camera, 3D NDVI maps can be quickly and easily generated for vegetation monitoring purposes. When using low-cost sensors for vegetation index estimation, it is necessary to apply a rigorous methodology for extracting reliable information. In this paper, the methodology for NDVI generation using a low-cost sensor as well as experiments to evaluate its performance is presented. The experiments performed show that it is possible to obtain a reliable NDVI point cloud from a Kinect V2.

Muscle activation signals are measured from the skin surface as surface electromyography (EMG) signals that contain information on human intentions; therefore, they are widely used in various robotics applications owing to their usability. However, selective muscle activation extraction is difficult because of the complexity of muscle structures. This study investigated muscle activation source model-based sEMG signal decomposition that considers the anatomical factors of muscle structures. The main advantage of the proposed model-based signal decomposition is that sEMG interface rotation can be recognized by comparing source parameters identified before and after rotation. To assess the performance of the proposed model-based decomposition method, hand motion estimation and rotation recognition were conducted. Additionally, two-dimensional simultaneous control was conducted with an inertial measurement unit to verify the usability of the proposed model. The results indicate that the proposed model decomposes an sEMG signal based on motion with good performance and demonstrate feasibility of motion estimation independent of sEMG interface rotation.

Human detection and tracking is an essential task for service robots, where the combined use of multiple sensors has potential advantages that are yet to be fully exploited. In this paper, we introduce a framework allowing a robot to learn a new 3D LiDAR-based human classifier from other sensors over time, taking advantage of a multisensor tracking system. The main innovation is the use of different detectors for existing sensors (i.e. RGB-D camera, 2D LiDAR) to train, online, a new 3D LiDAR-based human classifier based on a new “trajectory probability”. Our framework uses this probability to check whether new detection belongs to a human trajectory, estimated by different sensors and/or detectors, and to learn a human classifier in a semi-supervised fashion. The framework has been implemented and tested on a real-world dataset collected by a mobile robot. We present experiments illustrating that our system is able to effectively learn from different sensors and from the environment, and that the performance of the 3D LiDAR-based human classification improves with the number of sensors/detectors used.

Estimating motion from images is a well-studied problem in computer vision and robotics. Previous work has developed techniques to estimate the motion of a moving camera in a largely static environment (e.g., visual odometry) and to segment or track motions in a dynamic scene using known camera motions (e.g., multiple object tracking). It is more challenging to estimate the unknown motion of the camera and the dynamic scene simultaneously. Most previous work requires a priori object models (e.g., tracking-by-detection), motion constraints (e.g., planar motion), or fails to estimate the full SE (3) motions of the scene (e.g., scene flow). While these approaches work well in specific application domains, they are not generalizable to unconstrained motions. This paper extends the traditional visual odometry (VO) pipeline to estimate the full SE (3) motion of both a stereo/RGB-D camera and the dynamic scene. This multimotion visual odometry (MVO) pipeline requires no a priori knowledge of the environment or the dynamic objects. Its performance is evaluated on a real-world dynamic dataset with ground truth for all motions from a motion capture system.

Interactive robots offer improved performance in tasks with environmental uncertainty, but accommodating environment input weakens predictions of contact force or position trajectories, making the identification of subtask completion or faults difficult. This paper develops a task monitoring approach for complex assembly tasks that involve transitions between discrete environment dynamic modes. In semi-structured environments, these dynamic modes and their transitions are approximately known a priori, allowing task monitoring through estimation of the current mode and fault detection as a deviation from expected, desired dynamic mode transitions. This allows a more natural description of many interactive tasks, improving robustness to variations in force or position trajectories that impedance control seeks to address. The ability of impedance and admittance controlled robots to identify their environment is investigated, making consideration of joint and end-effector physical compliance. Prior information on environment dynamics and mode transitions allow recursive estimates of dynamic mode suitable for online use, under both full state knowledge and only force/position measurements. Experiments with an admittance controlled robot in a gear assembly task validate the approach.