ICRA 2019 Papers

Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and data collection is expensive, making retraining undesirable. Simulation training allows for feasible training times, but on the other hand suffer from a reality-gap when applied in real-world settings. This raises the need of efficient adaptation of policies acting in new environments.We consider the problem of transferring knowledge within a family of similar Markov decision processes. We assume that Q-functions are generated by some low-dimensional latent variable. Given such a Q-function, we can find a master policy that can adapt given different values of this latent variable. Our method learns both the generative mapping and an approximate posterior of the latent variables, enabling identification of policies for new tasks by searching only in the latent space, rather than the space of all policies. The low-dimensional space, and master policy found by our method enables policies to quickly adapt to new environments. We demonstrate the method on both a pendulum swing-up task in simulation, and for simulation-to-real transfer on a pushing task.

Surgeons performing laparoscopic surgery experience distortion when perceiving the stiffness of a patient's tissues. This is due to the lever effect induced by the introduction of instruments in their patient's body through a fulcrum. To address this problem, we propose to use the comanipulation paradigm. A robotic device is connected to the handle of the instrument while simultaneously being held by the surgeon. This device applies a force on the handle that reflects the force measured at the tool tip, with a gain that depends on the lever ratio. The implementation of this method is presented on an experimental setup and a preliminary assessment experiment is presented.

This paper addresses the challenge of humanoid robot teleoperation in a natural indoor environment via a Brain-Computer Interface (BCI). We leverage deep Convolutional Neural Network (CNN) based image and signal understanding to facilitate both real-time object detection and dry-Electroencephalography (EEG) based human cortical brain bio-signals decoding. We employ recent advances in dry-EEG technology to stream and collect the cortical waveforms from subjects while they fixate on variable Steady State Visual Evoked Potential (SSVEP) stimuli generated directly from the environment the robot is navigating. To these ends, we propose the use of novel variable BCI stimuli by utilising the real-time video streamed via the on-board robot camera as visual input for SSVEP, where the CNN detected natural scene objects are altered and flickered with differing frequencies (10Hz, 12Hz and 15Hz). These stimuli are not akin to traditional stimuli - as both the dimensions of the flicker regions and their on-screen position changes depending on the scene objects detected. Onscreen object selection via such a dry-EEG enabled SSVEP methodology, facilitates the on-line decoding of human cortical brain signals, via a specialised secondary CNN, directly into teleoperation robot commands (approach object, move in a specific direction: right, left or back). This SSVEP decoding model is trained via a priori offline experimental data in which very similar visual input is present for all subjects. The resulting classification demonstrates high performance with mean accuracy of 85% for the real-time robot navigation experiment across multiple test subjects.

Sampling-based planners are effective in many real-world applications such as robotics manipulation, navigation, and even protein modeling. However, it is often challenging to generate a collision-free path in environments where key areas are hard to sample. In the absence of any prior information, sampling-based planners are forced to explore uniformly or heuristically, which can lead to degraded performance. One way to improve performance is to use prior knowledge of environments to adapt the sampling strategy to the problem at hand. In this work, we decompose the workspace into local primitives, memorizing local experiences by these primitives in the form of local samplers, and store them in a database. We synthesize an efficient global sampler by retrieving local experiences relevant to the given situation. Our method transfers knowledge effectively between diverse environments that share local primitives and speeds up the performance dramatically. Our results show, in terms of solution time, an improvement of multiple orders of magnitude in two traditionally challenging high-dimensional problems compared to state-of-the-art approaches.

In this paper we define two feature representations for grasping. These representations capture hand-object geometric relationships at the near-contact stage - before the fingers close around the object. Their benefits are: 1) They are stable under noise in both joint and pose variation. 2) They are largely hand and object agnostic, enabling direct comparison across different hand morphologies. 3) Their format makes them suitable for direct application of machine learning techniques developed for images. We validate the representations by: 1) Demonstrating that they can accurately predict the distribution of ε-metric values generated by kinematic noise. I.e., they capture much of the information inherent in contact points and force vectors without the corresponding instabilities. 2) Training a binary grasp success classifier on a real-world data set consisting of 588 grasps.

Learning controllers for bipedal robots is a challenging problem, often requiring expert knowledge and extensive tuning of parameters that vary in different situations. Recently, deep reinforcement learning has shown promise at automatically learning controllers for complex systems in simulation. This has been followed by a push towards learning controllers that can be transferred between simulation and hardware, primarily with the use of domain randomization. However, domain randomization can make the problem of finding stable controllers even more challenging, especially for under actuated bipedal robots. In this work, we explore whether policies learned in simulation can be transferred to hardware with the use of high-fidelity simulators and structured controllers. We learn a neural network policy which is a part of a more structured controller. While the neural network is learned in simulation, the rest of the controller stays fixed, and can be tuned by the expert as needed. We show that using this approach can greatly speed up the rate of learning in simulation, as well as enable transfer of policies between simulation and hardware. We present our results on an ATRIAS robot and explore the effect of action spaces and cost functions on the rate of transfer between simulation and hardware. Our results show that structured policies can indeed be learned in simulation and implemented on hardware successfully. This has several advantages, as the structure preserves the intuitive nature of the policy, and the neural network improves the performance of the hand-designed policy. In this way, we propose a way of using neural networks to improve expert designed controllers, while maintaining ease of understanding.

This work develops a technique for using robot motion trajectories to create a high quality stochastic dynamics model that is then leveraged in simulation to train control policies with associated performance guarantees. We demonstrate the idea by collecting dynamics data from a 1/5 scale agile ground vehicle, fitting a stochastic dynamics model, and training a policy in simulation to drive around an oval track at up to 6.5 m/s while avoiding obstacles. We show that the control policy can be transferred back to the real vehicle with little loss in predicted performance. We compare this to an approach that uses a simple analytic car model to train a policy in simulation and show that using a model with stochasticity learned from data leads to higher performance in terms of trajectory tracking accuracy and collision probability. Furthermore, we show empirically that simulation-derived performance guarantees transfer to the actual vehicle when executing a policy optimized using a deep stochastic dynamics model fit to vehicle data.

The detection of curved lanes is still challenging for autonomous driving systems. Although current cutting-edge approaches have performed well in real applications, most of them are based on strict model assumptions. Similar to other visual recognition tasks, lane detection can be formulated as a two-dimensional graph searching problem, which can be solved by finding several optimal paths along with line segments and boundaries. In this paper, we present a directed graph model, in which dynamic programming is used to deal with a specific shortest path problem. This model is particularly suitable to represent objects with long continuous shape structure, e.g., lanes and roads. We apply the designed model and proposed an algorithm for detecting lanes by formulating it as the shortest path problem. To evaluate the performance of our proposed algorithm, we tested five sequences (including 1573 frames) from the KITTI database. The results showed that our method achieves an average successful detection precision of 97.5%.

Robotic-Assisted Minimally Invasive Surgery allows for easy recording of kinematic data, and presents excellent opportunities for data-intensive approaches to assessment of surgical skill, system design, and automation of procedures. However, typical surgical cases result in long data streams, and therefore, automated segmentation into gestures is important. The public release of the JIGSAWS dataset allowed for developing and benchmarking data-intensive segmentation algorithms. However, this dataset is small and the gestures are similar in their structure and directions. This may limit the generalization of the algorithms to real surgical data that are characterized by movements in arbitrary directions. In this paper, we use a recurrent neural network to segment a suturing task, and demonstrate one such generalization problem-limited generalization to rotation. We propose a simple augmentation that can solve this problem without collecting new data, and demonstrate its benefit using: (1) the JIGSAWS dataset, and (2) a new dataset that we recorded with a da Vinci Research Kit. Our study highlights the prospect of using data augmentation in the analysis of kinematic data in surgical data science.

We present an offline method to generate smooth, feasible motion for robot arms such that end-effector pose goals of a 6-DoF path are matched within acceptable limits specified by the user. Our approach aims to accurately match the position and orientation goals of the given path, and allows deviation from these goals if there is danger of self-collisions, joint-space discontinuities or kinematic singularities. Our method generates multiple candidate trajectories, and selects the best by incorporating sparse user input that specifies what kinds of deviations are acceptable. We apply our method to a range of challenging paths and show that our method generates solutions that achieve smooth, feasible motions while closely approximating the given pose goals and adhering to user specifications.

Alongside sweeping transformations in healthcare, a timeless drive to make surgical interventions less invasive and more effective has led to the integration of disparate technologies into surgical navigation systems. Fusions of device tracking and medical imaging modalities have been comprehensively investigated for opportunities to improve care. Such composite systems provide more and better information, enabling clinicians to operate less invasively and more effectively. Because of these merits, the preoperative ritual of harmonizing multiple information sources has been tacitly adopted. In this paper, we challenge the paradigm of preoperative registration. Proposed herein is a technique in which a clinician registers an interventional device to a navigation system simply by gesturing the device through a strategically designed fixture. In the background, the system continuously monitors the device path for this registration gesture. We demonstrate generality by applying the method to both robotic and electromagnetically tracked devices, and exhibit versatility by repeating the registration at multiple device base locations. Experiments indicate sub-millimeter accuracy versus conventional approaches on the same setup. Consequently, clinicians can register devices on the fly, increasing flexibility in setup and redefining workflow possibilities in surgery.

Modern cities are growing ecosystems that face new challenges due to the increasing population demands. One of the many problems they face nowadays is waste management, which has become a pressing issue requiring new solutions. Swarm robotics systems have been attracting an increasing amount of attention in the past years and they are expected to become one of the main driving factors for innovation in the field of robotics. The research presented in this paper explores the feasibility of a swarm robotics system in an urban environment. By using bio-inspired foraging methods such as multi-place foraging and stigmergy-based navigation, a swarm of robots is able to improve the efficiency and autonomy of the urban waste management system in a realistic scenario. To achieve this, a diverse set of simulation experiments was conducted using real-world GIS data and implementing different garbage collection scenarios driven by robot swarms. Results presented in this research show that the proposed system outperforms current approaches. Moreover, results not only show the efficiency of our solution, but also give insights about how to design and customize these systems.

The paper presents an unsupervised out-of-context action (O2CA) paradigm that is based on facilitating understanding by separately presenting both human action and context within a video sequence. As a means of generating an unsupervised label, we comprehensively evaluate responses from action-based (ActionNet) and context-based (ContextNet) convolutional neural networks (CNNs). Additionally, we have created three synthetic databases based on the human action (UCF101, HMDB51) and motion capture (mocap) (SURREAL) datasets. We then conducted experimental comparisons between our approach and conventional approaches. We also compared our unsupervised learning method with supervised learning using an O2CA ground truth given by synthetic data. From the results obtained, we achieved a 96.8 score on Synth-UCF, a 96.8 score on Synth-HMDB, and 89.0 on SURREAL-O2CA with F-score.

A new unsupervised learning method of depth and ego-motion using multiple masks from monocular video is proposed in this paper. The depth estimation network and the ego-motion estimation network are trained according to the constraints of depth and ego-motion without truth values. The main contribution of our method is to carefully consider the occlusion of the pixels generated when the adjacent frames are projected to each other, and the blank problem generated in the projection target imaging plane. Two fine masks are designed to solve most of the image pixel mismatch caused by the movement of the camera. In addition, some relatively rare circumstances are considered, and repeated masking is proposed. To some extent, the method is to use a geometric relationship to filter the mismatched pixels for training, making unsupervised learning more efficient and accurate. The experiments on KITTI dataset show our method achieves good performance in terms of depth and ego-motion. The generalization capability of our method is demonstrated by training on the low-quality uncalibrated bike video dataset and evaluating on KITTI dataset, and the results are still good.

This paper presents a novel method by which an assistive aerial robot can learn the relevant camera views within a task domain through tracking the head motions of a human collaborator. The human's visual field is modeled as an anisotropic spherical sensor, which decays in acuity towards the periphery, and is integrated in time throughout the domain. This data is resampled and fed into an expectation maximization solver in order to estimate the environment's visual interest as a mixture of Gaussians. A dynamic coverage control law directs the robot to capture camera views of the peaks of these Gaussians which is broadcast to an augmented reality display worn by the human operator. An experimental study is presented that assesses the influence of the assistive robot on reflex time, head motion, and task completion time.

Contact detection is an important topic in contemporary humanoid robotic research. Up to date control and state estimation schemes readily assume that feet contact status is known in advance. In this work, we elaborate on a broader question: in which gait phase is the robot currently in? We introduce an unsupervised learning framework for gait phase estimation based solely on proprioceptive sensing, namely joint encoder, inertial measurement unit and force/torque data. Initially, a meaningful physical explanation on data acquisition is presented. Subsequently, dimensionality reduction is performed to obtain a compact low-dimensional feature representation followed by clustering into three groups, one for each gait phase. The proposed framework is qualitatively and quantitatively assessed in simulation with ground-truth data of uneven/rough terrain walking gaits and insights about the latent gait phase dynamics are drawn. Additionally, its efficacy and robustness is demonstrated when incorporated in leg odometry computation. Since our implementation is based on sensing that is commonly available on humanoids today, we release an open-source ROS/Python package to reinforce further research endeavors.

In this paper, we propose a novel approach for underwater simultaneous localization and mapping using a multibeam imaging sonar for 3D terrain mapping tasks. The high levels of noise and the absence of elevation angle information in sonar images present major challenges for data association and accurate 3D mapping. Instead of repeatedly projecting extracted features into Euclidean space, we apply optical flow within bearing-range images for tracking extracted features. To deal with degenerate cases, such as when tracking is interrupted by noise, we model the subsea terrain as a Gaussian Process random field on a Chow-Liu tree. Terrain factors are incorporated into the factor graph, aimed at smoothing the terrain elevation estimate. We demonstrate the performance of our proposed algorithm in a simulated environment, which shows that terrain factors effectively reduce estimation error. We also show ROV experiments performed in a variable-elevation tank environment, where we are able to construct a descriptive and smooth height estimate of the tank bottom.

In this paper we consider inter-robot communication in the context of joint activities. In particular, we focus on convoying and passive communication for radio-denied environments by using whole-body gestures to provide cues regarding future actions. We develop a communication protocol whereby information described by codewords is transmitted by a series of actions executed by a swimming robot. These action sequences are chosen to optimize robustness and transmission duration given the observability, natural activity of the robot and the frequency of different messages. Our approach uses a convolutional network to make core observations of the pose of the robot being tracked, which is sending messages. The observer robot then uses an adaptation of classical decoding methods to infer a message that is being transmitted. The system is trained and validated using simulated data, tested in the pool and is targeted for deployment in the open ocean. Our decoder achieves.94 precision and.66 recall on real footage of robot gesture execution recorded in a swimming pool.

Although most robots use probabilistic algorithms to solve state estimation problems, path planning is often performed without considering the uncertainty about the robot's position. Uncertainty, however, matters in planning, but considering it often leads to computationally expensive algorithms. In this paper, we investigate the problem of path planning considering the uncertainty in the robot's belief about the world, in its perceptions and in its action execution. We propose the use of an uncertainty-augmented Markov Decision Process to approximate the underlying Partially Observable Markov Decision Process, and we employ a localization prior to estimate how the belief about the robot's position propagates through the environment. This yields to a planning approach that generates navigation policies able to make decisions according to the degree of uncertainty while being computationally tractable. We implemented our approach and thoroughly evaluated it on different navigation problems. Our experiments suggest that we are able to compute policies that are more effective than approaches that ignore the uncertainty, and that also outperform policies that always take the safest actions.