IROS 2016 Papers

Microneedle insertion is useful for elucidating the processes in living cells and their organelles. However, mechanical puncture by the needle causes traumatic damage to the cell. As a less invasive process, we have developed intracellular needle insertion using femtosecond laser cell ablation. To quickly and precisely position the spot ablated by the laser, we accurately located the needle tip in a 2D image plane. When the needle approaches the cell surface, the only accessible information is the microscope image; the vertical approach of the needle is unknown and must be detected. To this end, we propose an image process that integrates needle recognition, needle-tip localization, image focus measure, and vertical approach detection. The proposed image process was tested in 15 experimental sessions at various locations. The needle tip was reasonably and consistently positioned in the image, and the vertical approach was detected within the safe and plausible ranges.

We present the localization approach for Astrobee, a new free-flying robot designed to navigate autonomously on the International Space Station (ISS). Astrobee will accommodate a variety of payloads and enable guest scientists to run experiments in zero-g, as well as assist astronauts and ground controllers. Astrobee will replace the SPHERES robots which currently operate on the ISS, whose use of fixed ultrasonic beacons for localization limits them to work in a 2 meter cube. Astrobee localizes with monocular vision and an IMU, without any environmental modifications. Visual features detected on a pre-built map, optical flow information, and IMU readings are all integrated into an extended Kalman filter (EKF) to estimate the robot pose. We introduce several modifications to the filter to make it more robust to noise, and extensively evaluate the localization algorithm.

Complex indoor and outdoor missions for autonomous micro aerial vehicles (MAV) require fast generation of collision-free paths in 3D space. Often not all obstacles in an environment are known prior to the mission execution. Consequently, the ability for replanning during a flight is key for success. Our approach locally optimizes trajectories of grid-based path planning. It preserves obstacle-freeness of the path and ensures smoothness with continuous curvature transition segments. Fast optimization and frequent reoptimization is made possible by means of local multiresolution time discretization. With our extensions, high dimensional flight trajectories incorporating velocities and accelerations can be planned with a time discretization of 100 Hz within the prediction horizon of the underlying controller.

This paper presents the design and experimental validation of a compliant and lightweight 3-DOF robotic arm - shoulder yaw, shoulder pitch and elbow pitch joints - equipped with a compliant finger module intended for aerial inspection and manipulation in contact with the environment. A simple transmission mechanism consisting in a pair of compression springs and a flange bearing is integrated in the shoulder pitch and elbow pitch joints between the servo shaft and the output frame. Joint deflection measurement with potentiometer allows joint torque but also contact force estimation and control. The low stiffness of the compliant finger has been exploited for soft collision detection and obstacle localization, in such a way that the contact forces do not significantly disturb the UAV. Fixed-base experiments have been performed with the arm, including the characterization of the compliant joints and the control of the contact force at wrist point.

No two robots are exactly the same—even for a given model of robot, different units will require slightly different controllers. Furthermore, because robots change and degrade over time, a controller will need to change over time to remain optimal. This paper leverages lifelong learning in order to learn controllers for different robots. In particular, we show that by learning a set of control policies over robots with different (unknown) motion models, we can quickly adapt to changes in the robot, or learn a controller for a new robot with a unique set of disturbances. Furthermore, the approach is completely model-free, allowing us to apply this method to robots that have not, or cannot, be fully modeled.

One of the central tasks for a household robot is searching for specific objects. It does not only require localizing the target object but also identifying promising search locations in the scene if the target is not immediately visible. As computation time and hardware resources are usually limited in robotics, it is desirable to avoid expensive visual processing steps that are exhaustively applied over the entire image. The human visual system can quickly select those image locations that have to be processed in detail for a given task. This allows us to cope with huge amounts of information and to efficiently deploy the limited capacities of our visual system. In this paper, we therefore propose to use human fixation data to train a top-down saliency model that predicts relevant image locations when searching for specific objects. We show that the learned model can successfully prune bounding box proposals without rejecting the ground truth object locations. In this aspect, the proposed model outperforms a model that is trained only on the ground truth segmentations of the target object instead of fixation data.

Robots are still not able to grasp all unforeseen objects. Finding a proper grasp configuration, i.e. the position and orientation of the arm relative to the object, is still challenging. One approach for grasping unforeseen objects is to recognize an appropriate grasp configuration from previous grasp demonstrations. The underlying assumption in this approach is that new objects that are similar to known ones (i.e. they are familiar) can be grasped in a similar way. However finding a grasp representation and a grasp similarity metric is still the main challenge in developing an approach for grasping familiar objects. In this paper, interactive object view learning and recognition capabilities are integrated in the process of learning and recognizing grasps. The object view recognition module uses an interactive incremental learning approach to recognize object view labels. The grasp pose learning approach uses local and global visual features of a demonstrated grasp to learn a grasp template associated with the recognized object view. A grasp distance measure based on Mahalanobis distance is used in a grasp template matching approach to recognize an appropriate grasp pose. The experimental results demonstrate the high reliability of the developed template matching approach in recognizing the grasp poses. Experimental results also show how the robot can incrementally improve its performance in grasping familiar objects.

We propose in this work a new model of how the hidden human knowledge (HHK) of UAV pilots can be incorporated in the UAVs path planning generation. We intuitively know that human's pilots barely manage or even attempt to drive the UAV through a path that is optimal attending to some criteria as an optimal planner would suggest. Although human pilots might get close but not reach the optimal path proposed by some planner that optimizes over time or distance, the final effect of this differentiation could be not only surprisingly better, but also desirable. In the best scenario for optimality, the path that human pilots generate would deviate from the optimal path as much as the hidden knowledge that its perceives is injected into the path. The aim of our work is to use real human pilot paths to learn the hidden knowledge using repulsion fields and to incorporate this knowledge afterwards in the environment obstacles as cause of the deviation from optimality. We present a strategy of learning this knowledge based on attractor and repulsors, the learning method and a modified RRT* that can use this knowledge for path planning. Finally we do real-life tests and we compare the resulting paths with and without this knowledge.

In this paper, we propose a hybrid methodology based on a combination of analytical, numerical and machine learning methods for performing dexterous, in-hand manipulation with simple, adaptive robot hands. A constrained optimization scheme utilizes analytical models that describe the kinematics of adaptive hands and classic conventions for modelling quasistatically the manipulation problem, providing intuition about the problem mechanics. A machine learning (ML) scheme is used in order to split the problem space, deriving task-specific models that account for difficult to model, dynamic phenomena (e.g., slipping). In this respect, the ML scheme: 1) employs the simulation module in order to explore the feasible manipulation paths for a specific hand-object system, 2) feeds the feasible paths to an experimental setup that collects manipulation data in an automated fashion, 3) uses clustering techniques in order to group together similar manipulation trajectories, 4) trains a set of task-specific manipulation models and 5) uses classification techniques in order to trigger a task-specific model based on the user provided task specifications. The efficacy of the proposed methodology is experimentally validated using various adaptive robot hands in 2D and 3D in-hand manipulation tasks.

The goal of this paper is to develop a robot with a grounded spatial vocabulary. Such a vocabulary would allow it to give and follow directions, and would give it valuable additional information in aiding localization and navigation. We approach the problem by defining an ontology of space (including corridor, doorway, and room) and by creating a Convolutional Neural Network (CNN) that allows the robot to classify LIDAR sensor data accordingly. In particular, we propose a CNN architecture that performs comparably or better than existing methods based on engineered features. Training CNNs can be fickle; we describe several specific aspects of our approach that are important for good performance in this task.

For robotic agents to perform manipulation tasks in human environments at a human level or higher, they need to be able to relate the physical effects of their actions to how they are executing them; small variations in execution can have very different consequences. This paper proposes a framework for acquiring and applying action knowledge from naive user demonstrations in an interactive simulation environment under varying conditions. The framework combines a flexible constraint-based motion control approach with games-with-a-purpose-based learning using Random Forest Regression. The acquired action models are able to produce context-sensitive constraint-based motion descriptions to perform the learned action. A pouring experiment is conducted to test the feasibility of the suggested approach and shows the learned system can perform comparable to its human demonstrators.

Learning from demonstration is a popular approach for teaching robots as it allows service robots to acquire new skills without explicit programming. However, for manipulation actions mostly kinesthetic teaching is used as these actions require precise knowledge about the interactions between the robot and the object. In this paper, we present a novel approach that allows a robot to learn actions carried out by a teacher from observations. We achieve this by first transforming RGBD observations to consistent hand-object trajectories, which are then adapted to the robot's grasping capabilities. Experimental results show that the robot is able to learn complex tasks such as opening doors or drawers.

Learning to perform tasks like pulling a door handle or pushing a button, inherently easy for a human, can be surprisingly difficult for a robot. A crucial problem in these kinds of in-contact tasks is the context specificity of pose and force requirements. In this paper, a robot learns in-contact tasks from human kinesthetic demonstrations. To address the need to balance between the position and force constraints, we propose a model based on the hidden semi-Markov model (HSMM) and Cartesian impedance control. The model captures uncertainty over time and space and allows the robot to smoothly satisfy a task's position and force constraints by online modulation of impedance controller stiffness according to the HSMM state belief. In experiments, a KUKA LWR 4+ robotic arm equipped with a force/torque sensor at the wrist successfully learns from human demonstrations how to pull a door handle and push a button.

We present an approach to generate rapid and fluid drawing movements on a compliant Baxter robot, by taking advantage of the kinematic redundancy and torque control capabilities of the robot. We concentrate on the task of reproducing graffiti-stylised letter-forms with a marker. For this purpose, we exploit a compact lognormal-stroke based representation of movement to generate natural drawing trajectories. An Expectation-Maximisation (EM) algorithm is used to iteratively improve tracking performance with low gain feedback control. The resulting system captures the aesthetic and dynamic features of the style under investigation and permits its reproduction with a compliant controller that is safe for users surrounding the robot.

Dexterous multi-fingered hands can accomplish fine manipulation behaviors that are infeasible with simple robotic grippers. However, sophisticated multi-fingered hands are often expensive and fragile. Low-cost soft hands offer an appealing alternative to more conventional devices, but present considerable challenges in sensing and actuation, making them difficult to apply to more complex manipulation tasks. In this paper, we describe an approach to learning from demonstration that can be used to train soft robotic hands to perform dexterous manipulation tasks. Our method uses object-centric demonstrations, where a human demonstrates the desired motion of manipulated objects with their own hands, and the robot autonomously learns to imitate these demonstrations using reinforcement learning. We propose a novel algorithm that allows us to blend and select a subset of the most feasible demonstrations, which we use with an extension of the guided policy search framework that learns generalizable neural network policies. We demonstrate our approach on the RBO Hand 2, with learned motor skills for turning a valve, manipulating an abacus, and grasping.

Automating assembly processes outside controlled factory environments is still rare, mostly because of the inherent position uncertainties. The use of compliant motions allows robustness against the uncertainty, but automatic planning of compliant motion sequences is not computationally feasible. In this paper, we show how compliant assembly motions can be learned from human demonstrations. A human teacher will kinesthetically demonstrate compliant motions where the physical shapes of assembled parts guide the motion. From these demonstrations, the proposed method identifies desired direction of movement, the number of compliant axes and their directions. We use this information to construct an impedance controller which can reproduce the assembly motion despite uncertainty in the starting position. The method is studied with a KUKA LWR4+ arm in two test setups with different number of physically constrained degrees of freedom. The experimental study shows that the method is able to correctly identify the motion parameters and allows the robot to successfully perform the demonstrated assembly motion from various unseen starting positions.

Human-powered lower exoskeletons have gained considerable interests from both academia and industry over the past few decades, and thus have seen increasing applications in areas of human locomotion assistance and strength augmentation. One of the most important aspects in those applications is to achieve robust control of lower exoskeletons, which, in the first place, requires the proactive modeling of human movement trajectories through physical Human-Robot Interaction (pHRI). As a powerful representation tool for motion trajectories, Dynamic Movement Primitive (DMP) has been used extensively to model human movement trajectories. However, canonical DMPs only offers a general offline representation of human movement trajectory and neglects the real-time interaction term, therefore it cannot be directly applied to lower exoskeletons which need to model human motion trajectories online since different pilots have different trajectories and even one pilot might change his/her intended trajectory during walking. This paper presents a novel Coupled Cooperative Primitives (CCPs) scheme, which models the motion trajectories online. Besides maintaining canonical motion primitives, we also model the interaction term between the pilot and exoskeletons through impedance models and apply a reinforcement learning method based on Policy Improvement and Path Integrals (PI2) to learn the parameters online. Experimental results on both a single Degree-Of-Freedom (DOF) platform and a HUman-powered Augmentation Lower EXoskeleton (HUALEX) system demonstrate the advantages of our proposed CCP scheme.

3D scene reconstruction involves the volumetric modeling of space, and it is a fundamental step in a wide variety of robotic applications, including grasping, obstacle avoidance, path planning, mapping and many others. Nowadays, sensors are able to quickly collect vast amounts of data, and the challenge has become one of storing and processing all this information in a timely manner, especially if real-time performance is required. Recently, a novel technique for the stochastic learning of discriminative models through continuous occupancy maps was proposed: Hilbert Maps [18], that is able to represent the input space at an arbitrary resolution while capturing statistical relationships between measurements. The original framework was proposed for 2D environments, and here we extend it to higher-dimensional spaces, addressing some of the challenges brought by the curse of dimensionality. Namely, we propose a method for the automatic selection of feature coordinate locations, and introduce the concept of localized automatic relevance determination (LARD) to the Hilbert Maps framework, in which different dimensions in the projected Hilbert space operate within independent length-scale values. The proposed technique was tested against other state-of-the-art 3D scene reconstruction tools in three different datasets: a simulated indoors environment, RIEGL laser scans and dense LSD-SLAM pointclouds. The results testify to the proposed framework's ability to model complex structures and correctly interpolate over unobserved areas of the input space while achieving real-time training and querying performances.

We present a kinetostatic model for a needle-sized notched tube continuum joint design, and examine the impact that the design parameters have on both stiffness and joint kinematics. The joint is fabricated by removing a series of asymmetric notches from a metal tube. By fixing a cable to the distal end of the tube, and routing this tendon through the tube's lumen, the joint can be actuated in bending. This simple cutting pattern allows for impressive performance compared to other mechanisms of similar size. However, selecting the cutting geometry using kinematics alone results in many redundant solutions. Since the notches significantly affect the overall structure's stiffness, and limit the maximum forces that it can transmit, a stiffness model can be used to constrain the design-space. Further, because the notch geometry represents a non-prismatic beam configuration, modelling the force-deflection behavior is nontrivial. We have approached this problem using a variation of Castigliano's 2nd Theorem, experimentally validated its performance with several test specimens, and analyzed the behavior of the model over a design space specific to neurosurgical applications. The outcomes of this study aim to aid in the design of joints given task-specific constraints, particularly within the field of surgical robotics.