ICRA 2017 Papers

Provides an abstract for each of the presentations and a brief professional biography of each presenter. The complete presentations were not made available for publication as part of the conference proceedings.

The tension control of wires can benefit from Series Elastic Actuator (SEA), which is a potential emerging actuator system. In this paper, it is verified that the tension of wires can be successfully controlled by a novel SEA, called Compact Planetary-geared Elastic Actuator (cPEA), which incorporates a planetary gear and a spring to achieve precise force sensing and compactness at the same time. The design of model-based tension control and impedance compensator using cPEA is introduced, and the experiments that show the effectiveness of the proposed control are provided.

Autonomous legged robots will be required to handle a wide range of tasks in complex environments. While a lot of research has focused on developing their abilities for periodic locomotion tasks, less effort has been invested in devising generalized strategies for dynamic, non-periodic movements. Motion design approaches are frequently enlisted in the form of teleoperation or predefined heuristics in such scenarios. We employ a realistic simulation of the hydraulically actuated HyQ2Max quadrupedal system for investigations on two distinctive tasks: rearing and posture recovery. We present a whole-body optimization methodology for non-periodic tasks on quadrupedal systems. This approach delivers solutions involving multiple contacts without the need for predefined feet placements. The results obtained show the potential of optimization approaches for motion synthesis in the context of complex tasks.

In this paper, we introduce the achievement of the aerial manipulation by using the whole body of a transformable aerial robot, instead of attaching an additional manipulator. The aerial robot in our work is composed by two-dimensional multilinks which enable a stable aerial transformation and can be employed as an entire gripper. We propose a planning method to find the optimized grasping form for the multilinks while they are on the air, which is based on the original planar enveloping algorithm, along with the optimization of the internal force and joint torque for the force-closure. We then propose the aerial approach and grasp motion strategy, which is devoted to the determination of the form and position of the aerial robot to approach and grasp effectively the object from the air. Finally we present the experimental results of the aerial manipulation which involves grasping, carrying and dropping different types of object. These results validate the performance of aerial grasping based on our proposed whole-body grasp planning and motion control method.

We investigate the task of recognizing objects of daily use in human environments purely based on object descriptions given in natural language. In particular, we present an approach to transform phrases stated in natural language that describe such objects by their visual appearance into formal, semantic representations of their perceptual characteristics, which in turn can be used in a robot perception system in order to identify objects that the robot has never encountered before. To this end, we learn probabilistic first-order knowledge bases from encyclopedic articles and online dictionaries, which contain textual descriptions of a vast amount of everyday objects. We demonstrate the applicability of the approach on a robotic system in a proof-of-concept evaluation on a selected set of object descriptions acquired from the internet.

During robot-assisted dressing, a robot manipulates a garment in contact with a person's body. Inferring the forces applied to the person's body by the garment might enable a robot to provide more effective assistance and give the robot insight into what the person feels. However, complex mechanics govern the relationship between the robot's end effector and these forces. Using a physics-based simulation and data-driven methods, we demonstrate the feasibility of inferring forces across a person's body using only end effector measurements. Specifically, we present a long short-term memory (LSTM) network that at each time step takes a 9-dimensional input vector of force, torque, and velocity measurements from the robot's end effector and outputs a force map consisting of hundreds of inferred force magnitudes across the person's body. We trained and evaluated LSTMs on two tasks: pulling a hospital gown onto an arm and pulhng shorts onto a leg. For both tasks, the LSTMs produced force maps that were similar to ground truth when visualized as heat maps across the limbs. We also evaluated their performance in terms of root-mean-square error. Their performance degraded when the end effector velocity was increased outside the training range, but generalized well to limb rotations. Overall, our results suggest that robots could learn to infer the forces people feel during robot-assisted dressing, although the extent to which this will generalize to the real world remains an open question.

For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition. To perform actual tasks, it is critical for the robot to have a functional understanding of the visual scene. Here, we address the problem of localization and recognition of functional areas in an arbitrary indoor scene, formulated as a two-stage deep learning based detection pipeline. A new scene functionality testing-bed, which is compiled from two publicly available indoor scene datasets, is used for evaluation. Our method is evaluated quantitatively on the new dataset, demonstrating the ability to perform efficient recognition of functional areas from arbitrary indoor scenes. We also demonstrate that our detection model can be generalized to novel indoor scenes by cross validating it with images from two different datasets.

Wearable haptic feedback devices for virtual reality, human-robot interaction, and motion guidance require lightweight actuators that display clearly discernible cues to the user. These goals motivate the design of WRAP, a wearable, pneumatically actuated haptic feedback device. WRAP displays a variety of tactile sensations to the user by inflating a thermoplastic pneumatic actuator in direct contact with the skin. This paper describes the design and construction of WRAP, shows its effectiveness in indicating direction cues to users, and demonstrates two other appUcations for WRAP in image-guided medical interventions and human-computer interaction. Users were able to identify translation and rotation cues from WRAP with 99.4% accuracy. Our results suggest that WRAP is suitable for a variety of wearable and portable applications in which direction cues are beneficial.

This paper proposes a computationally efficient approach to detecting objects natively in 3D point clouds using convolutional neural networks (CNNs). In particular, this is achieved by leveraging a feature-centric voting scheme to implement novel convolutional layers which explicitly exploit the sparsity encountered in the input. To this end, we examine the trade-off between accuracy and speed for different architectures and additionally propose to use an L1 penalty on the filter activations to further encourage sparsity in the intermediate representations. To the best of our knowledge, this is the first work to propose sparse convolutional layers and L1 regularisation for efficient large-scale processing of 3D data. We demonstrate the efficacy of our approach on the KITTI object detection benchmark and show that Vote3Deep models with as few as three layers outperform the previous state of the art in both laser and laser-vision based approaches by margins of up to 40% while remaining highly competitive in terms of processing time.

Environmental conditions and external effects, such as shocks, have a significant impact on the calibration parameters of visual-inertial sensor systems. Thus long-term operation of these systems cannot fully rely on factory calibration. Since the observability of certain parameters is highly dependent on the motion of the device, using short data segments at device initialization may yield poor results. When such systems are additionally subject to energy constraints, it is also infeasible to use full-batch approaches on a big dataset and careful selection of the data is of high importance. In this paper, we present a novel approach for resource efficient self-calibration of visual-inertial sensor systems. This is achieved by casting the calibration as a segment-based optimization problem that can be run on a small subset of informative segments. Consequently, the computational burden is limited as only a predefined number of segments is used. We also propose an efficient information-theoretic selection to identify such informative motion segments. In evaluations on a challenging dataset, we show our approach to significantly outperform state-of-the-art in terms of computational burden while maintaining a comparable accuracy.

Nowadays, one of the largest environmental challenges that European countries must face consists in dealing with the past half century of nuclear waste. In order to optimize maintenance costs, nuclear waste must be sorted, segregated and stored according to its radiation level. Towards this end, in [1] we have recently proposed a visual-based shared control architecture meant to facilitate a human operator in controlling two remote robotic arms (one equipped with a gripper and another with a camera) during remote manipulation tasks of nuclear waste via a master device. The operator could then receive force cues informative of the feasibility of her/his motion commands during the task execution. The strategy presented in [1], albeit effective, suffers however from a locality issue since the operator can only provide instantaneous velocity commands (in a suitable task space), and receive instantaneous force feedback cues. On the other hand, the ability to `steer' a whole future trajectory in task space, and to receive a corresponding integral force feedback along the whole planned trajectory (because of any constraint of the considered system), could significantly enhance the operator's performance, especially when dealing with complex manipulation tasks. The aim of this work is to then extend [1] towards a planning-based shared control architecture able to take into account the mentioned requirements. A human/hardware-in-the-loop experiment with simulated slave robots and a real master device is reported for demonstrating the feasibility and effectiveness of the proposed approach.

Our work builds upon Visual Teach & Repeat 2 (VT&R2): a vision-in-the-loop autonomous navigation system that enables the rapid construction of route networks, safely built through operator-controlled driving. Added routes can be followed autonomously using visual localization. To enable long-term operation that is robust to appearance change, its Multi-Experience Localization (MEL) leverages many previously driven experiences when localizing to the manually taught network. While this multi-experience method is effective across appearance change, the computation becomes intractable as the number of experiences grows into the tens and hundreds. This paper introduces an algorithm that prioritizes experiences most relevant to live operation, limiting the number of experiences required for localization. The proposed algorithm uses a visual Bag-of-Words description of the live view to select relevant experiences based on what the vehicle is seeing right now, without having to factor in all possible environmental influences on scene appearance. This system runs in the loop, in real time, does not require bootstrapping, can be applied to any pointfeature MEL paradigm, and eliminates the need for visual training using an online, local visual vocabulary. By picking a subset of visually similar experiences to the live view, we demonstrate safe, vision-in-the-loop route following over a 31 hour period, despite appearance as different as night and day.

Percutaneous needle procedures are mostly carried out with the guidance of 2D ultrasound (US) imaging. US images are inherently noisy and their resolutions are low. Hence, target tracking can be challenging. Image based tracking methods can be used to track the needle and the target. This paper proposes visual tracking of multiple moving points, such as biopsy needles and targets, in 2D US images using normalized cross correlation and mutual information similarity functions. Both moving and deformable targets can be tracked. An affine motion model is used for small and moving target tracking and a thin plate spline motion model is used for deformable target tracking. During the tracking, needle and target template images are updated with a template update strategy. Also, tracking outputs of normalized cross correlation and mutual information are fused using the Kalman filter to reduce the tracking error. During the experiments, needle is inserted using a needle insertion robot. 2D US probe is attached to a robotic arm's end effector to servo the probe along the needle insertion path. Proposed needle and target tracking methods were tested with phantoms. Accuracies of the needle tip and moving target tracking methods were measured using an optical tracking system. Experimental results showed that the proposed tracking method could be used to simultaneously track the needle tip and the targets in real-time in 2D US guided percutaneous needle procedures.

Understanding physical phenomena is a key competence that enables humans and animals to act and interact under uncertain perception in previously unseen environments containing novel objects and their configurations. Developmental psychology has shown that such skills are acquired by infants from observations at a very early stage. In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability from appearance. We ask the question if and to what extent and quality such a skill can directly be acquired in a data-driven way — bypassing the need for an explicit simulation at run-time. We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers. We first evaluate the approach on synthetic data and compared the results to human judgments on the same stimuli. Further, we extend this approach to reason about future states of such towers that in return enables successful stacking.

We propose in this paper a new active perception scheme based on Model Predictive Control under constraints for generating a sequence of visual servoing tasks. The proposed control scheme is used to compute the motion of a camera whose task is to successively observe a set of robots for measuring their position and improving the accuracy of their localization. This method is based on the prediction of an uncertainty model (due to actuation and measurement noise) for determining which robot has to be observed by the camera. Simulation results are presented for validating the approach.

Apart the use of catadioptric cameras, only few visual servoing works exploit the use of mirror. Such a configuration is however interesting since it allows to overpass the limited camera field of view. Based on the known projection equations involved in such a system, this paper introduces the theoretical background that allows the use of planar mirror for visual servoing in different configurations. Limitations intrinsic to such systems, such as the number of d.o.f actually controllable, is then discussed. Experiments using a mirror mounted on the end-effector of a 6 d.o.f robot validate the proposed approach.

The Remote Centre of Motion (RCM) is an essential movement for most of the medical robotic systems during the minimal invasive surgeries. In literature, there are many references which had analysed the techniques for modelling the problem of RCM constraint. However, only very few have discussed the RCM with Path Following Controller which ensures the complexity of surgical tasks. Therefore, this article is focusing on presenting the Task Priority Controller. Such task controller deploys exteroceptive sensor (visual servoing scheme) in order that the surgical tool follows the reference path while maintaining the RCM constraints. The experimental results showed good performance of the proposed controller.

We propose a novel scoring concept for visual place recognition based on nearest neighbor descriptor voting and demonstrate how the algorithm naturally emerges from the problem formulation. Based on the observation that the number of votes for matching places can be evaluated using a binomial distribution model, loop closures can be detected with high precision. By casting the problem into a probabilistic framework, we not only remove the need for commonly employed heuristic parameters but also provide a powerful score to classify matching and non-matching places. We present methods for both a 2D-2D image matching and a 2D-3D landmark matching based on the above scoring. The approach maintains accuracy while being efficient enough for online application through the use of compact (low-dimensional) descriptors and fast nearest neighbor retrieval techniques. The proposed methods are evaluated on several challenging datasets in varied environments, showing state-of-the-art results with high precision and high recall.

Pouring a specific amount of liquid is a challenging task. In this paper we develop methods for robots to use visual feedback to perform closed-loop control for pouring liquids. We propose both a model-based and a model-free method utilizing deep learning for estimating the volume of liquid in a container. Our results show that the model-free method is better able to estimate the volume. We combine this with a simple PID controller to pour specific amounts of liquid, and show that the robot is able to achieve an average 38ml deviation from the target amount. To our knowledge, this is the first use of raw visual feedback to pour liquids in robotics.