ICRA 2017 Papers

For the past few years, unmanned aerial vehicles (UAVs) have been successfully employed in several investigations and exploration tasks such as aerial inspection and manipulations. However, most of these UAVs are limited to open spaces distant from any obstacles because of the high risk of falling as a result of an exposed propeller or not enough protection. On the other hand, a UAV with a passive rotating spherical shell can fly over a complex environment but cannot engage in physical interaction and perform power tethering because of the passive rotation of the spherical shell. In this study, we propose a new mechanism that allows physical interaction and power tethering while the UAV is well-protected and has a good flight stability, which enables exploration in a complex environment such as disaster sites. We address the current problem by dividing the whole shell into two separate hemispherical shells that provide a gap unaffected by passive rotation. In this paper, we mainly discuss the concept, general applications, and design of the proposed system. The capabilities of the proposed system for physical interaction and power tethering in a complex space were initially verified through laboratory-based test flights of our experimental prototype.

In this paper, we present an effective and accurate gaze estimation method based on two-eye model of a subject with the tolerance of free head movement from a Kinect sensor. To accurately and efficiently determine the point of gaze, i) we employ two-eye model to improve the estimation accuracy; ii) we propose an improved convolution-based means of gradients method to localize the iris center in 3D space; iii) we present a new personal calibration method that only needs one calibration point. The method approximates the visual axis as a line from the iris center to the gaze point to determine the eyeball centers and the Kappa angles. The final point of gaze can be calculated by using the calibrated personal eye parameters. We experimentally evaluate the proposed gaze estimation method on eleven subjects. Experimental results demonstrate that our gaze estimation method has an average estimation accuracy around 1.99°, which outperforms many leading methods in the state-of-the-art.

This paper studies a reconfiguration algorithm for heterogeneous cubic modular robots in environments with obstacles. Tunneling is suitable for the transformation of cubic modular robots in such an environment because a tunneling robot only passes spaces that are occupied by the robot in the start and goal configurations. We have also designed a permutation algorithm that can be executed in the space used by the tunneling robot. Considering the application in environments with obstacles, we assumed a motion primitive with only sliding motion along another module's surface. This differs from the previously studied one, which allows convex motions. The proposed algorithm uses a three-dimensional 2 × 2 × 2 meta-module to guarantee the existence of mobile modules and maintain the connectivity of the robot structure during the tunneling and permutation processes. We show that the algorithm is complete for a connected robot structure with more than one meta-module and that the reconfiguration in an environment with obstacles is executed in quadratic operating time cost.

Many areas of robotics, particularly locomotion and grasping, can benefit from the ability to modulate friction on surfaces that come into contact with objects in the environment. Previous research has tried to address the challenge of tunable friction; however, those efforts only provide modest gains or are difficult to integrate. We propose a tunable friction mechanism that relies on pneumatic actuation and is easily integrated into pre-existing soft actuators. We characterize the performance of our friction mechanism with quantitative force data for varying preload forces, substrate materials, and inflation pressures. Testing results show that our tunable friction mechanism achieves an order of magnitude differentiation in friction forces between high and low friction states. We demonstrate its potential application in a one degree-of-freedom soft crawler and a soft gripper with actuatable finger friction pads. The crawler successfully propelled itself forward by leveraging asymmetric strokes and the gripper achieved a factor of five differentiation of grip force between engaged and disengaged states of the friction tuning mechanism.

Collaborative robots represent a clear added value to manufacturing, as they promise to increase productivity and improve working conditions of such environments. Although modern robotic systems have become safe and reliable enough to operate close to human workers on a day-to-day basis, the workload is still skewed in favor of a limited contribution from the robot's side, and a significant cognitive load is allotted to the human. We believe the transition from robots as recipients of human instruction to robots as capable collaborators hinges around the implementation of transparent systems, where mental models about the task are shared between peers, and the human partner is freed from the responsibility of taking care of both actors. In this work, we implement a transparent task planner able to be deployed in realistic, near-future applications. The proposed framework is capable of basic reasoning capabilities for what concerns role assignment and task allocation, and it interfaces with the human partner at the level of abstraction he is most comfortable with. The system is readily available to non-expert users, and programmable with high-level commands in an intuitive interface. Our results demonstrate an overall improvement in terms of completion time, as well as a reduced cognitive load for the human partner.

We present trajectory tracking and balance control of an autonomous bikebot. The bikebot is a single-track autonomous mobile robot that is designed to study unstable physical human-robot interactions. The controller is built on the property of the external-internal convertible (EIC) structure for the bikebot dynamics. We present two types of the designs and analyses of the control systems and also demonstrate their performance through extensive experiments. The comparison results with the human riding experiments show that the rider's motor skills generate the similar strategies as the proposed EIC-based balance control.

We present a trajectory optimization framework for legged locomotion on rough terrain. We jointly optimize the center of mass motion and the foothold locations, while considering terrain conditions. We use a terrain costmap to quantify the desirability of a foothold location. We increase the gait's adaptability to the terrain by optimizing the step phase duration and modulating the trunk attitude, resulting in motions with guaranteed stability. We show that the combination of parametric models, stochastic-based exploration and receding horizon planning allows us to handle the many local minima associated with different terrain conditions and walking patterns. This combination delivers robust motion plans without the need for warm-starting. Moreover, we use soft-constraints to allow for increased flexibility when searching in the cost landscape of our problem. We showcase the performance of our trajectory optimization framework on multiple terrain conditions and validate our method in realistic simulation scenarios and experimental trials on a hydraulic, torque controlled quadruped robot.

We present an object-tracking framework that fuses point cloud information from an RGB-D camera with tactile information from a GelSight contact sensor. GelSight can be treated as a source of dense local geometric information, which we incorporate directly into a conventional point-cloud-based articulated object tracker based on signed-distance functions. Our implementation runs at 12 Hz using an online depth reconstruction algorithm for GelSight and a modified second-order update for the tracking algorithm. We present data from hardware experiments demonstrating that the addition of contact-based geometric information significantly improves the pose accuracy during contact, and provides robustness to occlusions of small objects by the robot's end effector.

Weed scouting is an important part of modern integrated weed management but can be time consuming and sparse when performed manually. Automated weed scouting and weed destruction has typically been performed using classification systems able to classify a set group of species known a priori. This greatly limits deployability as classification systems must be retrained for any field with a different set of weed species present within them. In order to overcome this limitation, this paper works towards developing a clustering approach to weed scouting which can be utilized in any field without the need for prior species knowledge. We demonstrate our system using challenging data collected in the field from an agricultural robotics platform. We show that considerable improvements can be made by (i) learning low-dimensional (bottleneck) features using a deep convolutional neural network to represent plants in general and (ii) tying views of the same area (plant) together. Deploying this algorithm on in-field data collected by AgBotII, we are able to successfully cluster cotton plants from grasses without prior knowledge or training for the specific plants in the field.

In this paper we lay the foundation of the first heterogeneous multi-robot system of the Multiple Aerial-Ground Manipulator System (MAGMaS) type. A MAGMaS consists of a ground manipulator and a team of aerial robots equipped with a simple gripper manipulator the same object. The idea is to benefit from the advantages of both kinds of platforms, i.e., physical strength versus large workspace. The dynamic model of such robotic systems is derived, and its characteristic structure exhibited. Based on the dynamical structure of the system a nonlinear control scheme, augmented with a disturbance observer is proposed to perform trajectory tracking tasks in presence of model inaccuracies and external disturbances. The system redundancy is exploited by solving an optimal force/torque allocation problem that takes into account the heterogeneous system constraints and maximizes the force manipulability ellipsoid. Simulation results validated the proposed control scheme for this novel heterogeneous robotic system. We finally present a prototypical mechanical design and preliminary experimental evaluation of a MAGMaS composed by a kuka LWR4 and quadrotor based aerial robot.

In this paper, we present a method for real-time 3D sound sources mapping using an off-the-shelf robotic perception sensor equipped with a linear microphone array. Conventional approaches to map sound sources in 3D scenarios use dedicated 3D microphone arrays, as this type of arrays provide two degrees of freedom (DOF) observations. Our method addresses the problem of 3D sound sources mapping using a linear microphone array, which only provides one DOF observations making the estimation of the sound sources location more challenging. In the proposed method, multi hypotheses tracking is combined with a new sound source parametrisation to provide with a good initial guess for an online optimisation strategy. A joint optimisation is carried out to estimate 6 DOF sensor poses and 3 DOF landmarks together with the sound sources locations. Additionally, a dedicated sensor model is proposed to accurately model the noise of the Direction of Arrival (DOA) observation when using a linear microphone array. Comprehensive simulation and experimental results show the effectiveness of the proposed method. In addition, a real-time implementation of our method has been made available as open source software for the benefit of the community.

Vision-based grasping for humanoid robots is a challenging problem due to a multitude of factors. First, humanoid robots use an “eye-to-hand” kinematics configuration that, on the contrary to the more common “eye-in-hand” configuration, demands a precise estimate of the position of the robot's hand. Second, humanoid robots have a long kinematic chain from the eyes to the hands, prone to accumulate the calibration errors of the kinematics model, which offsets the measured hand-to-object relative pose from the real one. In this paper, we propose a method able to solve these two issues jointly. A robust pose estimation of the robot's hand is achieved via a 3D model-based stereo-vision algorithm, using an edge-based distance transform metric and synthetically generated images of a robot's arm-hand internal computer-graphics model (kinematics and appearance). Then, a particle-based optimization method adapts on-line the robot's internal model to match the real and the synthetically generated images, effectively compensating the kinematics calibration errors. We evaluate the proposed approach using a position-based visual-servoing method on the iCub robot, showing the importance of the continuous visual feedback in humanoid grasping tasks.

This work is towards targeted drug delivery using microrobots functionalized to navigate towards naturally occurring pH gradients caused by cancer cells, and to release a payload in response to a light stimulus. Stimuli-responsive microrobots for the localization of specific cell types and targeted drug delivery could provide a new and promising therapy to prevent and treat the spread of cancer. In this work, we present two novel biocompatible photoresists for the fabrication of hybrid microrobots using two-photon polymerization (TPP) for medical applications. One biomarker for cancerous cells is that they exhibit lower pH compared to surrounding healthy tissue. In this work, a pH-responsive resist was developed and demonstrated to automatically seek a low-pH solid in a microfluidic channel, simulating metastatic cells within a vessel. The second resist, a hydrogel-based photoresist, was created to contract in response to light. The two resists were combined together in a two-step printing process to create a microswimmer with potential for tumor localization and drug release capabilities in the human circulatory system.

Inference and decision making under uncertainty are essential in numerous robotics problems. In recent years, the similarities between inference and control triggered much work, from developing unified computational frameworks to pondering about the duality between the two. In spite of the aforementioned efforts, inference and control, as well as inference and belief space planning (BSP) are still treated as two separate processes. In this paper we propose a novel approach that utilizes the similarities between inference and BSP and make the key observation that inference can be efficiently updated using the precursory planning stage, thus paving the way towards a joint inference and BSP paradigm. We develop four different methods that implement our novel approach under simplifying assumptions and validate them in the context of autonomous navigation in unknown environment. Results indicate that not only our methods improve running time by at least two orders of magnitude, compared to iSAM2 paradigm, they also found to be less sensitive to state dimensionality and loop closures.

Variable stiffness for robotics is attracting increasing attention from researchers in the field of surgical robots. A surgical robot that can access the human colon or stomach via natural orifices must be flexible enough to pass through tortuous paths and to work in a confined space. Meanwhile, the robot must also be stiff enough to ensure pushability and to hold high payloads during the surgery. Thus, surgical robots with variable stiffness are desirable. This paper presents a new design concept for variable stiffness manipulators using a thermoplastic material - Polyethylene Terephthalate (PET) - and a flexible stainless steel sheath as a heating solution. The stiffness of PET can be flexibly adjusted through temperature. Experiments and validations were carried out at different conditions. The results showed that our proposed design is at least as flexible as a typical commercial endoscope when flexibility is desired and meanwhile at least 9 times stiffer than the endoscope when stiffness is desired (Flexural modulus was compared). A tendon-driven manipulator based on the proposed concept was also developed. Validation tests showed that the manipulator in compliant mode can be significantly bent through cable actuation, and the manipulator in stiff mode is able to maintain its shape against considerably large loads.

Manual palpation is commonly used to localize tumors and other features buried deep inside organs during open surgery. This approach is not feasible in minimally invasive or robotic surgery, as the contact with the tissue is mediated by instruments. To address this problem, we propose a soft robotic skin (SRS) that can be deployed from a small incision and create a stiffness map in a single step. Such a skin is composed of a matrix of soft robotic tactile elements (SRTEs), each one able to expand and record the tissue response during expansion. In this paper, we firstly prove the feasibility of palpation using a single SRTE. Then, we present and test a soft-suction based anchoring mechanism able to keep the SRS in the desired position in contact with the tissue, allowing surgeons to palpate different sides of the organ. Finally, we detail a calibration method for the SRTE, and assess the feasibility of identifying lumps buried inside a soft tissue phantom, and then inside a chicken liver during an ex-vivo trial. Experimental results show that the SRTE was able to differentiate simulated lumps (up to 3.25 mm deep) from healthy tissue in both the phantom and the ex-vivo trials. These results, added to the ability of the suction gripper to compensate for the expansion forces of each SRTE, are paving the way for soft robotic autonomous tools that can be used for intraoperative mapping of tissue cancers.

The use of magnetic resonance (MR) images for needle-based interventions offers several advantages over other types of imaging modalities (e.g., high tissue contrast and no radiation). However, MR-guided interventions face challenges related to electromagnetic compatibility of medical devices and real-time tracking of surgical instruments. This work presents a flexible needle steering system that combines an MR-compatible robot and a Fiber Bragg Grating (FBG)-based needle tip tracker. The MR images are used to localize obstacles and targets, while the FBG sensors provide strain measurements for online estimation of the needle tip position. A pre-operative planner defines the needle entry point and desired path, while a model predictive controller calculates the needle rotation during the insertion. To the best of the authors knowledge, this is the first work that fuses MR images and FBG-based tracking to steer a flexible needle in closed-loop inside the MR bore. The system is validated by steering a bevel-tipped flexible needle towards a physical target in gelatin phantoms and biological tissues. The needle reaches the target in all trials with an average targeting error of 2.76 mm. Disregarding the target displacement during the insertion, the average targeting error drops to 1.74 mm. The preliminary results demonstrate the feasibility of combining MR images and FBG-based needle tip tracking to steer a flexible needle in clinical procedures. In order to move towards to a clinically-relevant application, the design of a flexible Nitinol biopsy needle is also presented and evaluated by experiments in a prostate of a bull. The flexible needle presented a curvature 2.5 times larger than a conventional biopsy needle while maintaining the ability to collect tissue samples.

Human in-hand dexterity can be highly fluid and unstructured, but in contrast, prevailing research in robotic manipulation has focused on highly structured, well-controlled motions where contact points are carefully characterized. Maintaining grasp stability through traditional closure conditions during complex within-hand manipulation motions can be difficult, even with highly-articulated end-effectors. However, simple grippers can still achieve an effective range of in-hand manipulation tasks without strict closure conditions, as long as the object can be bounded locally relative to the hand frame. The end-effector can be utilized as a tool to limit the range of possible object poses. We show that the hand-object system's configuration space can be sampled to find a set of manipulation primitives that can reliably constrain the object inside the hand workspace even without feedback, a strategy proposed as whole-hand caging manipulation. Experimental results with a planar (gravity into the page), two-finger underactuated gripper (Yale OpenHand) are presented to validate this manipulation strategy, and it is shown that even though contacts are regularly broken and reformed, the object can be repeatability manipulated within the hand workspace without ejection, enabling challenging behaviors such as sliding and gaiting.

In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.