ICRA 2017 Papers

Existing control techniques for optical trapping of cells or micro-objects can only perform either positioning control or separate control of orientation and position in a sequential manner. In this paper, we propose a robotic control technique to achieve simultaneous orientation and positioning control of a microscopic object using multiple laser-driven fingertips and robotic motorized stage control. Several optically trapped micro-particles are first utilized as the laser-driven fingertips to grasp a target micro-object. Simultaneous control of the laser-driven fingertips and the robotic motorized stage is then performed to achieve the control objective, in which the target object is oriented to a desired angular position by controlling the laser-driven fingertips, and at the same time, it is manipulated to follow a desired trajectory by maneuvering the robotic stage. Rigorous mathematical formulation and solution are developed for simultaneous orientation and positioning control of a micro-object using optical tweezers.

Robot awareness of human actions is an essential research problem in robotics with many important real-world applications, including human-robot collaboration and teaming. Over the past few years, depth sensors have become a standard device widely used by intelligent robots for 3D perception, which can also offer human skeletal data in 3D space. Several methods based on skeletal data were designed to enable robot awareness of human actions with satisfactory accuracy. However, previous methods treated all body parts and features equally important, without the capability to identify discriminative body parts and features. In this paper, we propose a novel simultaneous Feature And Body-part Learning (FABL) approach that simultaneously identifies discriminative body parts and features, and efficiently integrates all available information together to enable real-time robot awareness of human behaviors. We formulate FABL as a regression-like optimization problem with structured sparsity-inducing norms to model interrelationships of body parts and features. We also develop an optimization algorithm to solve the formulated problem, which possesses a theoretical guarantee to find the optimal solution. To evaluate FABL, three experiments were performed using public benchmark datasets, including the MSR Action3D and CAD-60 datasets, as well as a Baxter robot in practical assistive living applications. Experimental results show that our FABL approach obtains a high recognition accuracy with a processing speed of the order-of-magnitude of 101 Hz, which makes FABL a promising method to enable real-time robot awareness of human behaviors in practical robotics applications.

We propose a simulation framework for gait assistance with ankle pathologies. We first construct the neu-romuscular walking model, then design the parameters for assistance torques for stance and swing legs. The parameter values are determined by performing dynamic optimizations which takes into account the human-exoskeleton interactive dynamics. The simulated energy expenditure and kinematic data are compared with the real data. Case studies involve abnormal gaits with 1) foot drop, 2) foot drop and plantarflexion failure. We evaluate the gait efficiency and walking speed for the different gait types. Our result shows that each gait type should have a different assistance strategy (timing and magnitude) compared to the assistance strategy of a normal gait.

Planetary rovers face mobility hazards associated with various classes of terrains they traverse: sand, bedrock, and rock-strewn terrain. This work develops visual classifiers for these 3 terrain types for single monochrome navigation images from the NASA Mars Exploration Rover missions. The classifiers are based primarily on visual texture, captured in histograms of edges filter responses at various scales and orientations. Monochrome image intensity is further used to distinguish between confusing rock and bedrock cases. Three approaches are investigated: a gradient-based simplified HOG descriptor, a simplified GIST descriptor, and MR8 textons. Local rotational invariance is implemented in each approach, as validation tests demonstrate its benefit to performance. K-Nearest Neighbors is used for the final classification. No major differences in performance are observed between the three approaches, leading to the adoption of the HOG approach due to its lower computational complexity and thus highest applicability to planetary missions. Final tests demonstrate an accuracy between 70% and 93% (81% average) for the 3-way classification using the simplified HOG descriptor.

For a safe, natural and effective human-robot social interaction, it is essential to develop a system that allows a robot to demonstrate the perceivable responsive behaviors to complex human behaviors. We introduce the Multimodal Deep Attention Recurrent Q-Network using which the robot exhibits human-like social interaction skills after 14 days of interacting with people in an uncontrolled real world. Each and every day during the 14 days, the system gathered robot interaction experiences with people through a hit-and-trial method and then trained the MDARQN on these experiences using end-to-end reinforcement learning approach. The results of interaction based learning indicate that the robot has learned to respond to complex human behaviors in a perceivable and socially acceptable manner.

Small Unmanned Aerial Vehicles (UAVs) are some of the most promising robotic platforms in a variety of applications due to their high mobility. Their restricted computational and payload capabilities, however, translate into significant challenges in automating their navigation. With Simultaneous Localization And Mapping (SLAM) systems recently demonstrated to be employable onboard UAVs, the focus fall on path-planning on the quest of achieving autonomous navigation. With the vast body of path-planning literature often assuming perfect maps or maps known a priori, the biggest challenge lies in dealing with the robustness and accuracy limitations of onboard SLAM in real missions. In this spirit, this paper proposes a path-planning algorithm designed to work in the loop of the SLAM estimation of a monocular-inertial system. This point-to-point planner is demonstrated to navigate in an unknown environment using the incrementally generated SLAM map, while dictating the navigation strategy for preferable acquisition of sensor data for better estimations within SLAM. A thorough evaluation testbed of both simulated and real data is presented, demonstrating the robustness of the proposed pipeline against the state-of-the-art and its dramatically lower computational complexity, revealing its suitability to UAV navigation.

Retinal diseases including age-related macular degeneration and diabetic retinopathy are leading causes of visual impairment and a growing medical crisis worldwide. As a result, new therapies and methods of safely delivering these new therapies can be of significant benefit to society. In this paper, we present a proof-of-concept for performing drug delivery using flexible magnetic microcatheters. The potential benefits of these catheters are that they are safer to use than existing rigid vitreoretinal tools, are precisely manipulable, and can be used to deliver lower dosages of therapeutic agents to precisely targeted locations on the retina with higher efficacy. The task of positioning the catheter tip is shared between a human operator and a control algorithm, which have complementary skills and can separately address different aspects of the overall task. The system was tested in an eye-phantom by an untrained operator with a 93% success rate in reaching 43 target points uniformly dispersed over the retina.

Hardness is among the most important attributes of an object that humans learn about through touch. However, approaches for robots to estimate hardness are limited, due to the lack of information provided by current tactile sensors. In this work, we address these limitations by introducing a novel method for hardness estimation, based on the GelSight tactile sensor, and the method does not require accurate control of contact conditions or the shape of objects. A GelSight has a soft contact interface, and provides high resolution tactile images of contact geometry, as well as contact force and slip conditions. In this paper, we try to use the sensor to measure hardness of objects with multiple shapes, under a loosely controlled contact condition. The contact is made manually or by a robot hand, while the force and trajectory are unknown and uneven. We analyze the data using a deep constitutional (and recurrent) neural network. Experiments show that the neural net model can estimate the hardness of objects with different shapes and hardness ranging from 8 to 87 in Shore 00 scale.

Metal detectors are widely used to detect and localize land mines, as well as metallic clutter that causes many false alarms. These false alarms are handled by manual inspection with a prodder or using extra features like size and depth as well as fusion with other sensors like ground penetrating radar or chemical sensors. Directly shape itself has never been used, mainly due to infeasibility of the results of the few existing studies, which require controlled environments and dense sampling of the target objects. In this paper, we propose a new method for shape reconstruction that works with sparse data, and show its feasibility using field data collected by a mobile robot equipped with a commercial pulse induction metal detector carried by a custom 2DoF arm. This paper also describes the method employed to sweep natural terrains with the mobile manipulator and provides results of imaging a set of different metallic objects.

This article develops a set space visual servoing method that is quiet different from state-of-the-art approaches. Our approach does not require complex image processing techniques for the extraction, matching and tracking of image features. Instead, it only requires a simple matching algorithm and builds visual errors in set space. Each error is mainly related to one degree of freedom of the camera; therefore, we can design a decoupled control law. This control law is robust and does not require calibrated inner parameters of the camera. Our approach has been validated in 4-degree-of-freedom (DOF) visual servoing simulations with common image patterns and 6-DOF visual servoing experiments with specific image patterns. These visual servoing tasks are properly achieved even when partial occlusions occur.

We describe a new series pneumatic artificial muscle (sPAM) and its application as an actuator for a soft continuum robot. The robot consists of three sPAMs arranged radially around a tubular pneumatic backbone. Analogous to tendons, the sPAMs exert a tension force on the robot's pneumatic backbone, causing bending that is approximately constant curvature. Unlike a traditional tendon driven continuum robot, the robot is entirely soft and contains no hard components, making it safer for human interaction. Models of both the sPAM and soft continuum robot kinematics are presented and experimentally verified. We found a mean position accuracy of 5.5 cm for predicting the end-effector position of a 42 cm long robot with the kinematic model. Finally, closed-loop control is demonstrated using an eye-in-hand visual servo control law which provides a simple interface for operation by a human. The soft continuum robot with closed-loop control was found to have a step-response rise time and settling time of less than two seconds.

Bayesian Optimization has gained much popularity lately, as a global optimization technique for functions that are expensive to evaluate or unknown a priori. While classical BO focuses on where to gather an observation next, it does not take into account practical constraints for a robotic system such as where it is physically possible to gather samples from, nor the sequential nature of the problem while executing a trajectory. In field robotics and other real-life situations, physical and trajectory constraints are inherent problems. This paper addresses these issues by formulating Bayesian Optimization for continuous trajectories within a Partially observable Markov Decision Process (POMDP) framework. The resulting POMDP is solved using Monte-Carlo Tree Search (MCTS), which we adapt to using a reward function balancing exploration and exploitation. Experiments on monitoring a spatial phenomenon with a UAV illustrate how our BO-POMDP algorithm outperforms competing techniques.

Apprenticeship learning has recently attracted a wide attention due to its capability of allowing robots to learn physical tasks directly from demonstrations provided by human experts. Most previous techniques assumed that the state space is known a priori or employed simple state representations that usually suffer from perceptual aliasing. Different from previous research, we propose a novel approach named Sequence-based Multimodal Apprenticeship Learning (SMAL), which is capable to simultaneously fusing temporal information and multimodal data, and to integrate robot perception with decision making. To evaluate the SMAL approach, experiments are performed using both simulations and real-world robots in the challenging search and rescue scenarios. The empirical study has validated that our SMAL approach can effectively learn plans for robots to make decisions using sequence of multimodal observations. Experimental results have also showed that SMAL outperforms the baseline methods using individual images.

In modern day industry, robots are indispensable for achieving high production rates and competitiveness. In small and medium scale enterprises, where the production may shift rapidly, it is vital to be able to reprogram robots quickly. Kinesthetic teaching, also known as lead-through programming (LTP), provides a fast approach for teaching a trajectory. In this approach, a trajectory is demonstrated by physical interaction with the robot, i.e., the user manually guides the manipulator. This paper presents a sensorless approach to LTP for redundant robots that eliminates the need for expensive force/torque sensors. The active implementation enhances the passive LTP by an admittance control in joint space based on the external forces applied by the user, estimated with a Kalman filter using the generalized momentum formulation. To improve the quality of the estimation and hence LTP, we use a dithering technique. The active LTP has been implemented on ABB YuMi robot and experimental comparison with an earlier passive LTP is presented.

This paper presents the design and experimental validation of a soft pneumatic artificial muscle with position and force sensing capabilities. Conductive liquid-based soft sensors are embedded in a fiber-reinforced contractile actuator to measure two modes of deformation - axial strain and diametral expansion - which, together, are used to determine the stroke length and contractile force generated under internal pressure. We validate the proposed device by using data from the embedded sensors to estimate the force output of the actuator at fixed lengths and the stiffness and force output of a one degree-of-freedom hinge joint driven by an antagonist pair of the sensorized pneumatic muscles.

Understanding the semantic relations between vision and language data has become a research trend in artificial intelligence and robotic systems. The lack of training data is an essential issue for vision-language understanding. We address the problem of image and sentence cross-modal retrieval when paired training samples are not sufficient. Inspired by recent works in variational inference, in this paper, the autoencoding variational Bayes framework is novelly extended to a semi-supervised model for image-sentence mapping task. Our method does not require all training images and sentences to be paired. The proposed model is an end-to-end system, and consists of a two-level variational embedding structure where unpaired data are involved in the first level embedding to give support to intra-modality statistics so that the lower bound of the joint marginal likelihood of paired data embeddings can be better approximated. The proposed retrieval model is evaluated on two popular datasets, i.e. Flickr30K and Flickr8K, producing superior performances compared with related state-of-the-art methods.

In recent years in Japan, over half of all workers suffered from lower back pain. This has become a social problem that needs to be addressed. To reduce its occurrence, we developed a flexible, high-output waist assist suit called “AB-Wear” in a previous study. The AB-Wear suit can assist human motion and reduce muscular fatigue of the waist. However, the assistive forces of the device generate compressive forces on the backbone, which have adverse effects on the body. Hence, in this study, we propose an exoskeleton-type AB-Wear equipped with a compressive force reduction mechanism, called “semi-endoskeleton-type AB-Wear”. This device has a reduction mechanism similar to a flexible flat spring behind the upper body. Because of this structure, this device can generate an effective assistive force. First, we explain the difference between the semi-endoskeleton-type AB-Wear and the previous device. Then, we model the semi-endoskeleton-type AB-Wear because the model is used for its operation. Moreover, its effectiveness is confirmed using musculoskeletal simulation. Finally it is evaluated by measuring surface electromyography (EMG) on a subject's body to confirm its effectiveness with a real body. The EMGs of the wearer with and without the suit are compared. The usefulness of the AB-Wear is confirmed by simulation and experiment.

This paper presents a robust and efficient semidense visual odometry solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline which estimates the pose of the sensor by registering the projection of a 3D semidense map of a reference frame with the 2D semi-dense region extracted in the current frame. The processing is speeded up by efficiently implemented approximate nearest neighbour fields under the Euclidean distance criterion, which permits the use of compact Gauss-Newton updates in the optimization. The registration is formulated as a maximum a posterior problem to deal with outliers and sensor noise, and the equivalent weighted least squares problem is consequently solved by iteratively reweighted least squares method. A variety of robust weight functions are tested and the optimum is determined based on the probabilistic characteristics of the sensor model. Extensive evaluation on publicly available RGB-D datasets shows that the proposed method predominantly outperforms existing state-of-the-art methods.

Visual place recognition under difficult perceptual conditions remains a challenging problem due to changing weather conditions, illumination and seasons. Long-term visual navigation approaches for robot localization should be robust to these dynamics of the environment. Existing methods typically leverage feature descriptions of whole images or image regions from Deep Convolutional Neural Networks. Some approaches also exploit sequential information to alleviate the problem of spatially inconsistent and non-perfect image matches. In this paper, we propose a novel approach for learning a discriminative holistic image representation which exploits the image content to create a dense and salient scene description. These salient descriptions are learnt over a variety of datasets under large perceptual changes. Such an approach enables us to precisely segment the regions of an image which are geometrically stable over large time lags. We combine features from these salient regions and an off-the-shelf holistic representation to form a more robust scene descriptor. We also introduce a semantically labeled dataset which captures extreme perceptual and structural scene dynamics over the course of 3 years. We evaluated our approach with extensive experiments on data collected over several kilometers in Freiburg and show that our learnt image representation outperforms off-the-shelf features from the deep networks and hand-crafted features.