IROS 2018 Papers

We assume that an intuitive means for the specification, re-use, modification and transfer of synchronized motions-both regarding the two arms of a dual-arm robotic system, as well as regarding the coordination of a user and a robot-is key in interactive and collaborative settings as they are currently targeted for industrial applications. We show, how our knowledge based approach to end-user programming of synchronized motions and other generalizable, robot-agnostic skills can support such specification of coordinated actions between two robot arms and explain how that could be extended to include coordination with a human user. We describe the underlying ontologies and possibilities to populate those with an interface for intuitive programming, and show the generality of our approach through a task transfer between different kinematics (different robots), where the user is supported through underlying reasoning about the fulfillment of certain parameters or constraints for the involved skills.

Fasteners assembly automation in different industries require flexible systems capable of dealing with faulty situations. Fault detection and isolation (FDI) techniques are used to detect failure and deal with them, avoiding losses on parts, tools or robots. However, FDI usually deals with the faults after or at the moment they occur. Thus, we propose a method that predicts potential failures online, based on the forces and torques signatures captured during the task. We demonstrate the approach experimentally using an industrial robot, equipped with a force-torque sensor and a pneumatic gripper, used to align and thread nuts into bolts. All effort information is fed into a supervised machine learning algorithm, based on a Convolutional Neural Network (CNN) classifier. The network was able to predict and classify the threading task outcomes in 3 groups: mounted, not mounted or jammed. Our approach was able to reduce in 10.9% the threading task execution time when compared to a reference without FDI, but had problem to predict jammed cases. The same experiment was also performed with other two additional learning algorithms, and the results were systematically compared.

For human-robot cooperation, inferring a hu-man's cognitive state is very important for an efficient and natural interaction. Similar to human-human cooperation, understanding what the partner plans and knowing, if he is situation aware, is necessary to prevent collisions, offer support at the right time, correct mistakes before they happen or choose the best actions for oneself as early as possible. We propose a model-based belief filter to extract relevant aspects of a human's mental state online during cooperation. It performs inference based on human actions and its own task knowledge, modeling cognitive processes like perception and action selection. In contrast to most prior work, we explicitly estimate the human belief instead of inferring only a single mode or intention. Since this is a double inference process, we focus on representing the human estimates of environmental state and task as well as corresponding uncertainties. We designed a human-robot cooperation experiment that allowed for a variety of cognitive states of both agents and collected data to test and evaluate the proposed belief filter. The results are promising, as our system can be used to provide reasonable predictions of the human action and insights into his situation awareness. At the same time it is inferring interpretable information about the underlying cognitive states - A belief about the human's belief about the environment.

Accurate state estimation is a fundamental module for various intelligent applications, such as robot navigation, autonomous driving, virtual and augmented reality. Visual and inertial fusion is a popular technology for 6-DOF state estimation in recent years. Time instants at which different sensors' measurements are recorded are of crucial importance to the system's robustness and accuracy. In practice, timestamps of each sensor typically suffer from triggering and transmission delays, leading to temporal misalignment (time offsets) among different sensors. Such temporal offset dramatically influences the performance of sensor fusion. To this end, we propose an online approach for calibrating temporal offset between visual and inertial measurements. Our approach achieves temporal offset calibration by jointly optimizing time offset, camera and IMU states, as well as feature locations in a SLAM system. Furthermore, the approach is a general model, which can be easily employed in several feature-based optimization frameworks. Simulation and experimental results demonstrate the high accuracy of our calibration approach even compared with other state-of-art offline tools. The VIO comparison against other methods proves that the online temporal calibration significantly benefits visual-inertial systems. The source code of temporal calibration is integrated into our public project, VINS-Mono1.

The paper proposes a spatial sound perception system for an autonomous mobile robot. The system performs three-dimensional position localization and recognition as online processing from a robot in motion. For online processing, the sound positions are estimated as probabilistic regions in three dimensional space, because the robot could observe only arrival direction of the sound at a moment. The detected sound signals are recognized using Convolutional Neural Network (CNN), for the adjustment to short input signals. The experimental results show our mobile robot could observe surrounding sound sources online and continuously update its position and sound label.

Precise robot manipulation of deformable objects requires an accurate and fast estimation of their shape as they deform. So far, visual sensing has been mostly used to solve this issue, but vision sensors are sensitive to occlusions, which might be inevitable when manipulating an object with robot. To address this issue, we present a modular pipeline to track the shape of a soft object in an online manner by coupling tactile sensing with a deformation model. Using a model of a tactile sensor, we compute the magnitude and location of a contact force and apply it as an external force to the deformation model. The deformation model then updates the nodal positions of a mesh that describes the shape of the deformable object. The proposed sensor model and pipeline, are evaluated using a Shadow Dexterous Hand equipped with BioTac sensors on its fingertips and an RGB-D sensor.

Tendon-driven musculoskeletal humanoids have many benefits in terms of the flexible spine, multiple degrees of freedom, and variable stiffness. At the same time, because of its body complexity, there are problems in controllability. First, due to the large difference between the actual robot and its geometric model, it cannot move as intended and large internal muscle tension may emerge. Second, movements which do not appear as changes in muscle lengths may emerge, because of the muscle route changes caused by softness of body tissue. To solve these problems, we construct two models: ideal joint-muscle model and muscle-route change model, using a neural network. We initialize these models by a man-made geometric model and update them online using the sensor information of the actual robot. We validate that the tendon-driven musculoskeletal humanoid Kengoro is able to obtain a correct self-body image through several experiments.

Recently, there have been numerous advances in the development of payload and power constrained lightweight Micro Aerial Vehicles (MAVs). As these robots aspire for highspeed autonomous flights in complex dynamic environments, robust scene understanding at long-range becomes critical. The problem is heavily characterized by either the limitations imposed by sensor capabilities for geometry-based methods, or the need for large-amounts of manually annotated training data required by data-driven methods. This motivates the need to build systems that have the capability to alleviate these problems by exploiting the complimentary strengths of both geometry and data-driven methods. In this paper, we take a step in this direction and propose a generic framework for adaptive scene segmentation using self-supervised online learning. We present this in the context of vision-based autonomous MAV flight, and demonstrate the efficacy of our proposed system through extensive experiments on benchmark datasets and realworld field tests.

This paper presents the path planning and coordinated control of a space robot with a manipulator for capturing a rotating large object. As the grasping point on a rotating large object is translationally moving fast, an appropriate strategy and coordinated motion control of the spacecraft base and robotic arm must be employed for approaching and tracking such a grasping point. In this paper, we propose a robust control scheme including the online path planning and compliance control for grasping such a target. The path planning is derived in a simple form that allows the desired end-effector trajectory to be easily modified in real-time using the newly updated states without complex numerical calculation. In addition, the compliance control allows the end-effector to track the planned trajectory or the moving grasping point, while using contact force feedback to reduce the end-effector position error from the grasping point when capturing the target. This end-effector motion is implemented by coordinated control on the spacecraft base and robotic arm, which can suitably alter their distribution of motion according to scenes using a weighted pseudoinverse matrix. Experiments are conducted to demonstrate the validity of the proposed path planning and compliance control.

How can high dimensional robots learn general sets of skills from experience in the real world? Many previous approaches focus on maximizing a single utility function and require large datasets of experience to do this, something that is not possible to collect outside of simulation as every data point is expensive both in time and in a potential wear down of the robot. This paper addresses this question using a newly developed framework called Finite Element Goal Babbling (FEGB). FEGB is an online learning method that aims at providing general control over some measurable feature, in contrast to optimizing it to some given utility function. It generalizes standard goal babbling by breaking down the full learning problem into local sub-problems, and combining it with a planner that learns how to navigate between these subproblems. We test FEGB using a real humanoid robot Nao, and find that it could quickly learn to robustly control its body orientation. After only 20-30 minutes of training, the robot could freely move into any body orientation between lying on either side and on its back. Rapid learning of body orientation control in high dimensional real robots is largely an unexplored field of robotics, and although many challenges remain, FEGB shows a feasible approach to the problem.

In this paper, we propose a novel method for selective management of muscle fatigue in human-robot co-manipulation. The proposed framework enables the detection of excessive fatigue levels of an individual muscle group while executing a certain task, and provides anticipatory robotic responses to distribute the effort among less-fatigued muscles of human arm. Our approach uses a machine learning technique to enable online predictions of muscle forces in different arm configurations and endpoint interaction forces. The estimated muscle forces are then used for the model-based estimation of muscle fatigue levels. Through optimisation, the fatigue management system can alter the task execution in a way that specific fatigued muscles are offloaded, while at the same time enables the production of task force using muscles with lower levels of fatigue. The main advantage of the proposed method is that it can operate online, and that all the measurements are performed by the robot sensory system, which can significantly increase the applicability in real-world scenarios. To validate the proposed method, we performed proof-of-concept experiments where the task of the human operator was to use a tool to polish an object that was manipulated by the robot.

Proprioceptive terrain sensing is essential for rough terrain traversal because it helps legged robots to negotiate individual steps by reacting to terrain irregularities. In this work, we propose to utilize inertial data in the detection of the contact between the leg and the terrain during the stride phase of the leg. We show that relatively cheap accelerometers can be utilized to reliably detect a foot-strike, and thus allow the robot to crawl irregular terrains. The continuous data processing is compared with the interrupt mode in which data are provided only around the foot-strike event. The interrupt mode exhibits significantly better performance, and it also supports generalization of the foot-strike event detector learned from data collected in slow locomotion to faster locomotion where the signals slightly change. The proposed solution is experimentally validated using a real hexapod walking robot for which the walking speed has been improved in comparison to the previous adaptive motion gait based on a force threshold-based position controller for the foot-strike detection.

Pushing is a common task in robotic scenarios. In real-world environments, robots need to manipulate various unknown objects without previous experience. We propose a data-driven approach for learning local inverse models of robot-object interaction for push manipulation. The robot makes observations of the object behaviour on the fly and adapts its movement direction. The proposed model is probabilistic, and we update it using maximum a posteriori (MAP) estimation. We test our method by pushing objects with a holonomic mobile robot base. Validation of results over a diverse object set demonstrates a high degree of robustness and a high success rate in pushing objects towards a fixed target and along a path compared to previous methods. Moreover, based on learned inverse models, the robot can learn object properties and distinguish between different object behaviours when they are pushed from different sides.

This video presents a microfluidic phenomenon called “Virtual Vortex Gear (VVG)” and an application of it. The video contains 4 parts and is described as follows: The 1st part shows an application of VVG as a controllable valve in a micro fluidic system and the on and off of the valve are controlled by different flow speeds. The valve is turned on when the flow speed is high enough, and vice versa. The 2nd part shows the generation of VVG and its mechanism. When the flow speed, which is proportional to Reynolds Number, is gradually increased, the flow pattern evolves in the order as (1)parallel streamlines, (2)one vortex, (3)two vortices and eventually (4)three vortices including the last vortex inside the circular chamber. The evolution indicates the transmission of flow energy from the main stream to the inside of the chamber when the flow speed is over a certain range. In addition, every two adjacent vortices rotate in opposite directions which is just like a set of gears, and that is why we named it “VVG”. In the 3rd part, an application of VVG for chemical injection is demonstrated. A colored liquid is represented for the chemical and is surrounded by different sheath flow for the control of injection locations. It is found that only the fluid in a particular pinpoint can be injected into the target chamber. Furthermore, the complex but stable 3D flow patterns are visualized from the video. The last part of the video shows that different amount of chemical injection can be performed in different chambers along the same main stream and the distribution of the color is gradually become uniform by spontaneous diffusing.

In multi-robot systems passive interconnections among agents are often exploited to achieve a desired and robustly stable cooperative behavior. Nevertheless, the passivity constraint limits the kinds of behaviors that can be achieved. In this paper, we exploit the concept of energy tank for building a novel generalized interconnection that allows to impose any kind of dynamic coupling between two passive systems in a flexible way while preserving the passivity of the overall coupled system. The proposed strategy is validated by simulations and experiments.

Speech emotion recognition (SER) is an important aspect of effective human-robot collaboration and received a lot of attention from the research community. For example, many neural network-based architectures were proposed recently and pushed the performance to a new level. However, the applicability of such neural SER models trained only on in-domain data to noisy conditions is currently under-researched. In this work, we evaluate the robustness of state-of-the-art neural acoustic emotion recognition models in human-robot interaction scenarios. We hypothesize that a robot's ego noise, room conditions, and various acoustic events that can occur in a home environment can significantly affect the performance of a model. We conduct several experiments on the iCub robot platform and propose several novel ways to reduce the gap between the model's performance during training and testing in real-world conditions. Furthermore, we observe large improvements in the model performance on the robot and demonstrate the necessity of introducing several data augmentation techniques like overlaying background noise and loudness variations to improve the robustness of the neural approaches.

This paper presents orientation planning algorithms respecting the requirements of task space trajectory generation, particularly in robotics applications. The proposed algorithms fulfill the following conditions: (i) permitting to impose constraints at angular velocity and acceleration in addition to orientation at endpoints; (ii) rendering continuous acceleration profiles even when interpolating multiple orientations; and (iii) being computationally fast enough for realtime implementation. The generated spline trajectories are essentially a concatenation of polynomial in time curves parameterized by quaternion coefficients. To impose the unitariness condition critically required for quaternion representation of orientation, we develop an on-line update mechanism which successively reparameterizes the polynomials constructing the spline, towards suppressing distortions that the normalization operation might incur. Experiments on an anthropomorphic robot upper-body are carried out to demonstrate the efficacy and real-time compatibility of the proposed algorithms in comparison with a standard spherical interpolation method.

Legged-wheeled robots combine the advantages of efficient wheeled mobility with the adaptability to real-world terrains through the legged locomotion. Due to this hybrid mobility skill, they can excel in many application scenarios where other mobile platforms are not suitable for. However, their versatile mobility increases the number of constraints in their motion control where both the properties of legged and wheeled systems need to be considered. Relevant schemes for legged-wheeled platforms so far have been developed exploiting separate motion control of the wheeled and legged functionalities. This paper discusses the legged-wheeled motion kinematics without constraining the camber angles of the wheels, and it proposes a first-order inverse kinematics scheme that stabilizes the legged-wheeled system in the wheeled motion. Furthermore, the work adopts a floating base model that allows to easily incorporate the legged motion to the scheme. The developed controller is tested in simulation and experiments on a legged-wheeled centaur-like robot - CENTAURO.

Spinal cord stimulation (SCS) has recently enabled humans with motor complete spinal cord injury (SCI) to independently stand and recover some lost autonomic function. However, the nature of the recovered motor activity and the interplay between SCS and motor training are not well understood. Understanding the effect of stand training and spinal stimulation on motor activity during bipedal standing is important for designing spinal rehabilitation therapies that seek to combine spinal stimulation and rehabilitative robots. In this study, we examined electromyography (EMG) data gathered from two SCI patients and six healthy subjects as they attempted standing. We analyzed the muscle activation patterns and EMG waveform shape to quantify both the changes in SCI patient motor activity with training, and the differences between healthy motor activity and SCI patient motor activity under stimulation. We also looked for correlations between the similarity in SCI patients' motor activity to healthy subjects and their overall standing ability. We found that good standing in SCI patients does not emulate healthy standing muscle activity. Furthermore, patient stand training heavily influenced motor activation patterns, but not in ways that improved standing ability. These results indicate that current training techniques do not optimally influence motor activity, and robotic rehabilitation strategies for SCI patients should target essential features of motor activity to optimize functional performance, rather than emulate healthy activity.