ICRA 2018 Papers

Programming by demonstration has recently gained much attention due to its user-friendly and natural way to transfer human skills to robots. In order to facilitate the learning of multiple demonstrations and meanwhile generalize to new situations, a task-parameterized Gaussian mixture model (TP-GMM) has been recently developed. This model has achieved reliable performance in areas such as human-robot collaboration and dual-arm manipulation. However, the crucial task frames and associated parameters in this learning framework are often set by the human teacher, which renders three problems that have not been addressed yet: (i) task frames are treated equally, without considering their individual importance, (ii) task parameters are defined without taking into account additional task constraints, such as robot joint limits and motion smoothness, and (iii) a fixed number of task frames are pre-defined regardless of whether some of them may be redundant or even irrelevant for the task at hand. In this paper, we generalize the task-parameterized learning by addressing the aforementioned problems. Moreover, we provide a novel learning perspective which allows the robot to refine and adapt previously learned skills in a low dimensional space. Several examples are studied in both simulated and real robotic systems, showing the applicability of our approach.

We propose a novel approach for generating high-quality, synthetic data for domain-specific learning tasks, for which training data may not be readily available. We leverage recent progress in image-to-image translation to bridge the gap between simulated and real images, allowing us to generate realistic training data for real-world tasks using only unlabeled real-world images and a simulation. GeneSIS-Rtameliorates the burden of having to collect labeled real-world images and is a promising candidate for generating high-quality, domain-specific, synthetic data. To show the effectiveness of using GeneSIS-Rtto create training data, we study two tasks: semantic segmentation and reactive obstacle avoidance. We demonstrate that learning algorithms trained using data generated by GeneSIS-RT make high-accuracy predictions and outperform systems trained on raw simulated data alone, and as well or better than those trained on real data. Finally, we use our data to train a quadcopter to fly 60 meters at speeds up to 3.4 m/s through a cluttered environment, demonstrating that our GeneSIS-RT images can be used to learn to perform mission-critical tasks.

We introduce an interactive system for extracting the geometries of generalized cylinders and cuboids from single-or multiple-view point clouds. Our proposed method is intuitive and only requires the object's silhouettes to be traced by the user. Leveraging the user's perceptual understanding of what an object looks like, our proposed method is capable of extracting accurate models, even in the presence of occlusion, clutter or incomplete point cloud data, while preserving the original object's details and scale. We demonstrate the merits of our proposed method through a set of experiments on a public RGB-D dataset. We extracted 16 objects from the dataset using at most two views of each object. Our extracted models represent a high degree of visual similarity to the original objects. Further, we achieved a mean normalized Hausdorff distance of 5.66% when comparing our extracted models with the dataset's ground truths.

A key component to the success of master-slave surgical systems is the quality of the master interface used to relay the surgeon's instructions to the slave robot. In previous work the authors developed a gaze-assisted intention recognition scheme, allowing the system to dynamically adapt the motion scaling based on where the user is trying to reach. This allowed users to perform tasks significantly more quickly and with less need for clutching. However, the system possessed a number of core parameters that were manually optimized, potentially providing a non-optimal solution depending on the user. This paper presents a Bayesian approach to the problem of optimizing a human-robot interface in a user-specific manner. Two Bayesian optimization methods are studied: one in which users are asked to grade robot behavior for a given set of parameters, and one where only preference relative to other parameter sets is expressed. The performance of these optimizations is evaluated in a blind comparison user study, demonstrating that the optimized parameters are preferred to the manually optimized ones in over 90 % of cases after only 12 test samples. These parameters are further shown to perform at least as well as the manually optimized ones in all cases, and showing statistically significant improvement in the case of the graded optimization.

In this paper, we focus on adaptive sampling on a Gaussian Processes (GP) using the receding-horizon Cross-Entropy (CE) trajectory optimization. Specifically, we employ the GP upper confidence bound (GP-UCB) as the optimization criteria to adaptively plan sampling paths that balance the exploitation-exploration trade-off. Path planning at the initial stage focuses on exploring and learning a model of the environment, and later, on exploiting the learned model to focus sampling around regions that exhibit extreme sensory measurements and much higher spatial variability, denoted as the Region of Interest (ROI). The integration of the CE trajectory optimization allows the sampling density to be dynamically adjusted based on the latest sensory measurements, thus providing an efficient sampling strategy for sensing and localizing the ROI. We demonstrate the effectiveness of the proposed method in exploring simulated scalar fields with single or multiple ROIs. Field experiments with an Unmanned Surface Vehicle (USV) in a coastal bathymetry mapping mission validate the approach's capability in quickly exploring and mapping the given area, and then focusing and increasing the sampling density around the deepest region, as a surrogate for e.g. the extremal concentration of a pollutant in the environment.

Achieving accurate, high-rate pose estimates from proprioceptive and/or exteroceptive measurements is the first step in the development of navigation algorithms for agile mobile robots such as Unmanned Aerial Vehicles (UAVs). In this paper, we propose a decoupled Graph-Optimization based Multi-Sensor Fusion approach (GOMSF) that combines generic 6 Degree-of-Freedom (DoF) visual-inertial odometry poses and 3 DoF globally referenced positions to infer the global 6 DoF pose of the robot in real-time. Our approach casts the fusion as a real-time alignment problem between the local base frame of the visual-inertial odometry and the global base frame. The alignment transformation that relates these coordinate systems is continuously updated by optimizing a sliding window pose graph containing the most recent robot's states. We evaluate the presented pose estimation method on both simulated data and large outdoor experiments using a small UAV that is capable to run our system onboard. Results are compared against different state-of-the-art sensor fusion frameworks, revealing that the proposed approach is substantially more accurate than other decoupled fusion strategies. We also demonstrate comparable results in relation with a finely tuned Extended Kalman Filter that fuses visual, inertial and GPS measurements in a coupled way and show that our approach is generic enough to deal with different input sources in a straightforward manner. Video - https//youtu.be/GIZNSZ2soL8.

We study how autonomous robots can learn by themselves to improve their depth estimation capability. In particular, we investigate a self-supervised learning setup in which stereo vision depth estimates serve as targets for a convolutional neural network (CNN) that transforms a single still image to a dense depth map. After training, the stereo and mono estimates are fused with a novel fusion method that preserves high confidence stereo estimates, while leveraging the CNN estimates in the low-confidence regions. The main contribution of the article is that it is shown that the fused estimates lead to a higher performance than the stereo vision estimates alone. Experiments are performed on the KITTI dataset, and on board of a Parrot SLAMDunk, showing that even rather limited CNNs can help provide stereo vision equipped robots with more reliable depth maps for autonomous navigation.

A cognitive robot usually needs to perform multiple tasks in practice and needs to locate the desired area for each task. Since deep learning has achieved substantial progress in image recognition, to solve this area detection problem, it is straightforward to label a functional area (affordance) image dataset and apply a well-trained deep-model-based classifier on all the potential image regions. However, annotating the functional area is time consuming and the requirement of large amount of training data limits the application scope. We observe that the functional area are usually related to the surrounding object context. In this work, we propose to use the existing object detection dataset and employ the object context as effective prior to improve the performance without additional annotated data. In particular, we formulate a two-stream network that fuses the object-related and functionality-related feature for functional area detection. The whole system is formulated in an end-to-end manner and easy to implement with current object detection framework. Experiments demonstrate that the proposed network outperforms current method by almost 20% in terms of precision and recall.

We build upon the functional object-oriented network (FOON), a structured knowledge representation which is constructed from observations of human activities and manipulations. A FOON can be used for representing object-motion affordances. Knowledge retrieval through graph search allows us to obtain novel manipulation sequences using knowledge spanning across many video sources, hence the novelty in our approach. However, we are limited to the sources collected. To further improve the performance of knowledge retrieval as a follow up to our previous work, we discuss generalizing knowledge to be applied to objects which are similar to what we have in FOON without manually annotating new sources of knowledge. We discuss two means of generalization: 1) expanding our network through the use of object similarity to create new functional units from those we already have, and 2) compressing the functional units by object categories rather than specific objects. We discuss experiments which compare the performance of our knowledge retrieval algorithm with both expansion and compression by categories.

Road detection from images is a key task in autonomous driving. The recent advent of deep learning (and in particular, CNN or convolutional neural networks) has greatly improved the performance of road detection algorithms. In this paper, we show how to fuse multiple different cues under the same convolutional network framework. Specifically, we adopt a pre-trained Resnet-lOl to extract feature maps from RGB images; we then connect it with three extra deconvolution layers. These deconvolution layers is trained conditioning on appropriate image cues, and in our case they are a height image (i.e. elevation map obtained by e.g. Lidar scanner), image gradient, and position map. We also design two skip layers to speed up the convergence. Experiments on KITTI benchmark show competitive performance of our new networks.

Swarm robotics carries the potential of solving complex tasks using simple devices. To do so, however, one must define distributed control algorithms capable of producing globally coordinated behaviours. We propose a methodology to address the problem of the spatial coverage of multiple tasks with a swarm of robots that must not lose global connectivity. Our methodology comprises two layers: (i) a distributed Robot Navigation Controller (RNC) is responsible for simultaneously guaranteeing connectivity and pursuit of multiple tasks; and (ii) a global Task Scheduling Controller approximates the optimal strategy for the RNC with minimal computational load. Our contributions include: (i) a qualitative analysis of the literature on connectivity assessment, (ii) our proposed methodology, (iii) simulations in a multi-physics environment, (iv) real-life robot experiments, and (v) the experimental validation of connectivity, coverage optimality, and fault-tolerance.

Many running animals, unlike their robotic counterparts, have distinct morphologies and functional roles for their front and rear legs. In this paper we present a new control approach for a 5kg autonomous dynamic quadruped that explicitly encodes separate roles for each contralateral pair of legs. This controller utilizes a functional dynamic decomposition similar to Raibert's three part control law, but focuses on fore-aft leg specialization to regulate the robot's performance. The velocity of this controller, which exceeds 5 body lengths per sec, is compared with an improved trajectory-based controller and shown to be significantly more robust to changes in environment.

Soft robotic devices have been utilized in a number of biomedical applications involving human interaction. An emerging opportunity for soft robotic wearable devices is in mechanotherapeutic applications for the recovery and regeneration of soft tissues. Previous studies have implied that judicious force application during mechanotherapy plays an important role in the functional outcome of tissue regeneration. In this paper, we propose soft robotic devices with closed-loop force control to precisely manipulate muscular tissue. The developed devices incorporate fully soft sensors and actuators using textile-based materials and fabrication methods. The closed-loop force control system is demonstrated in bench studies to regulate massage-magnitude forces at frequencies akin to those expected in manual mechanotherapy practices. Testing of the device on human limbs demonstrates the precision and accuracy of the closed-loop force control methodology across different body shapes and types. When commanded to regulate sinusoidal force profiles (with amplitudes of 30N, 45N and 60N), the soft robotic force control device could regulate peak compressive loads to within 0.7N of the desired force. Conversely, open-loop pressure-based control resulted in up to +/-6.6N force tracking variability between participants. A soft robotic system with independently actuatable modules was also fabricated to demonstrate force-controlled actuation patterns to mimic manual massage techniques.

This paper proposes a novel parallel robot - Virtual Ground Robot (VGR) - that is driven by three Series Elastic Actuators (SEAs) to interact with a human. The proposed Virtual Ground Robot provides a virtual ground on which a human can stand on and interact in three directions: the pitch, the roll and the height directions. The most significant features of the proposed VGR are that 1) it is driven by RFSEAs (Reaction Force-sensing Series Elastic Actuator), and thus it can provide precise forces and torques, 2) the size of the VGR is small enough for a human to stand on with ease, and 3) it can generate torque/force large to support a weight of a human. Taking advantage of RFSEAs utilized in the proposed VGR, Spatial Force control algorithm is proposed in this paper. In order to design this controller, the motions of VGR are defined in the task space, the joint space and the RFSEA level. Based on the Kinematics, force control of VGR in the task level, which is named Spatial Force Control is designed and verified using experiments.

Bipedal robots have gained a lot of locomotion capabilities the past few years, especially in the control level. Navigation over complex and unstructured environments using exteroceptive perception, is still an active research topic. In this paper, we present a footstep planning system to produce foothold placements, using visual perception and proper environment modeling, given a black box walking controller. In particular, we extend a state-of-the-art search-based planning approach (ARA*) that produces 6DoF footstep sequences in 3D space for flat uneven terrain, to also handle rough curved surfaces, e.g. rocks. This is achieved by integrating both a curved patch modeling system for rough local terrain surfaces and a flat foothold contact analysis based on visual range input data, into the existing planning framework. The system is experimentally validated using real-world point clouds, while rough terrain stepping demonstrations are presented on the WALK-MAN humanoid robot, in simulation.

This paper proposes a computationally efficient method to estimate the time-varying relative pose between two visual-inertial sensor rigs mounted on the flexible wings of a fixed-wing unmanned aerial vehicle (UAV). The estimated relative poses are used to generate highly accurate depth maps in real-time and can be employed for obstacle avoidance in low-altitude flights or landing maneuvers. The approach is structured as follows: Initially, a wing model is identified by fitting a probability density function to measured deviations from the nominal relative baseline transformation. At runtime, the prior knowledge about the wing model is fused in an Extended Kalman filter (EKF) together with relative pose measurements obtained from solving a relative perspective N-point problem (PNP), and the linear accelerations and angular velocities measured by the two inertial measurement units (IMU) which are rigidly attached to the cameras. Results obtained from extensive synthetic experiments demonstrate that our proposed framework is able to estimate highly accurate baseline transformations and depth maps.

One of the emerging tasks for Micro Air Vehicles (MAVs) is autonomous indoor navigation. While commonly employed platforms for such tasks are micro-quadrotors, insect-inspired flapping wing MAVs can offer many advantages, such as being inherently safe due to their low inertia, reciprocating wings bouncing of objects or potentially lower noise levels compared to rotary wings. Here, we present the first flapping wing MAV to perform an autonomous multi-room exploration task. Equipped with an on-board autopilot and a 4 g stereo vision system, the DelFly Explorer succeeded in combining the two most common tasks of an autonomous indoor exploration mission: room exploration and door passage. During the room exploration, the vehicle uses stereo-vision based droplet algorithm to avoid and navigate along the walls and obstacles. Simultaneously, it is running a newly developed monocular color based Snake-gate algorithm to locate doors. A successful detection triggers the heading-based door passage algorithm. In the real-world test, the vehicle could successfully navigate, multiple times in a row, between two rooms separated by a corridor, demonstrating the potential of flapping wing vehicles for autonomous exploration tasks.

Nature offers many examples of organisms coming together to form self-assembling structures. The attachment methods these organisms employ allow them to grab onto others' bodies, often without need for specific alignment or orientation, an ability absent from most existing robotic self-assembling structures, which require complicated sensing and specific alignment. This paper presents FireAnt, a modular 2D robot that demonstrates full-body continuous docks, an attachment mechanism able to attach anywhere onto other robots at any orientation, eliminating the need for alignment mechanisms and complex sensors. Such docks allow FireAnt to climb over copies of itself, something critical to self-assembling structures. This paper first discusses the design of FireAnt before presenting test results that show the strength and reliability of the continuous docks and demonstrate FireAnt's ability to traverse an environment consisting of inert FireAnt robots. The work presented in this paper provides a docking mechanism that can minimize the mechanical complexity of modular robots and will allow the creation of swarms of rigid and adaptable self-assembling structures.

This paper focuses on tracking failure avoidance during vision-based navigation to a desired goal in unknown environments. While using feature-based Visual Simultaneous Localization and Mapping (VSLAM), continuous identification and association of map points are required during motion. Thus, we discuss a motion planning framework that takes into account sensory constraints for a reliable navigation. We use information available in the SLAM and propose a data-driven approach to predict the number of map points associated in a given pose. Then, a distance-optimal path planner utilizes the model to constrain paths such that the number of associated map points in each pose is above a threshold. We also include an online mapping of the environment for collision avoidance. Overall, we propose an iterative motion planning framework that enables real-time replanning after the acquisition of more information. Experiments in two environments demonstrate the performance of the proposed framework.