ICRA 2017 Papers

Developmental robotics seeks to build robots that learn to interact with the environment largely autonomously. These robots can calibrate their sensorimotor competencies on their own, much like developing children. In this paper, we build a developmental model of image stabilization based on the active efficient coding (AEC) framework and apply the model to a real robotic platform. In the visual system of primates, the optokinetic response (OKR) and the vestibulo-ocular reflex (VOR) cooperate to ensure image stabilization during relative motion between the observer and the environment. Inspired by these biological findings, our model integrates visual, inertial and motor encoder sensory cues. The sensory processing and the motor policy co-develop. The visual processing is based on a sparse coding algorithm. Motor behavior is learned using reinforcement learning. Our results show that the stabilization performance is improved by integrating visual and inertial inputs. Importantly, the weighting between the two inputs is learned automatically as the robot interacts with the environment.

For decades, robots have been essential allies alongside humans in controlled industrial environments like heavy manufacturing facilities. However, without the guidance of a trusted human operator to shepherd a robot safely through a wide range of conditions, they have been barred from the complex, ever changing environments that we live in from day to day. Safe learning control has emerged as a promising way to start bridging algorithms based on first principles to complex real-world scenarios by using data to adapt, and improve performance over time. Safe learning methods rely on a good estimate of the robot dynamics and of the bounds on modelling error in order to be effective. Current methods focus on either a single adaptive model, or a fixed, known set of models for the robot dynamics. This limits them to static or slowly changing environments. This paper presents a method using Gaussian Processes in a Dirichlet Process mixture model to learn an increasing number of non-linear models for the robot dynamics. We show that this approach enables a robot to re-use past experience from an arbitrary number of previously visited operating conditions, and to automatically learn a new model when a new and distinct operating condition is encountered. This approach improves the robustness of existing Gaussian Process-based models to large changes in dynamics that do not have to be specified ahead of time.

Reinforcement learning (RL) can automate a wide variety of robotic skills, but learning each new skill requires considerable real-world data collection and manual representation engineering to design policy classes or features. Using deep reinforcement learning to train general purpose neural network policies alleviates some of the burden of manual representation engineering by using expressive policy classes, but exacerbates the challenge of data collection, since such methods tend to be less efficient than RL with low-dimensional, hand-designed representations. Transfer learning can mitigate this problem by enabling us to transfer information from one skill to another and even from one robot to another. We show that neural network policies can be decomposed into “task-specific” and “robot-specific” modules, where the task-specific modules are shared across robots, and the robot-specific modules are shared across all tasks on that robot. This allows for sharing task information, such as perception, between robots and sharing robot information, such as dynamics and kinematics, between tasks. We exploit this decomposition to train mix-and-match modules that can solve new robot-task combinations that were not seen during training. Using a novel approach to train modular neural networks, we demonstrate the effectiveness of our transfer method for enabling zero-shot generalization with a variety of robots and tasks in simulation for both visual and non-visual tasks.

A robot performs a sequence of motions to animate a given input, e.g., dancing to music or telling a story. Each input is pre-processed to determine labels, e.g., emotions of the music or words in the story. Each label corresponds to multiple motions, and each motion has multiple labels. Therefore, the robot can choose one sequence from multiple motion sequences to animate the input. We aim to choose the best sequence to animate based on the audience's preferences. The audience prefers some motions over others, and each motion has an initially unknown preference value. At the end of the motion sequence, the audience provides feedback which is the sum of the motions' preference values. However, the observation of the feedback is noisy due to the device used to capture the audience's feedback. To select the most preferred sequence, the robot has to determine the sequence to query the audience with, so as to learn the preference values of individual motions from noisy observations of the audience's feedback. By learning the individual motion preference values, the most preferred sequence can be determined. Moreover, the audience may get bored of watching the same single motion in multiple sequences and the preference value will degrade based on the number of times the motion is viewed. We contribute MAK (Multi-Armed bandit and Kalman filter) and show that MAK outperforms least squares regression in selecting the best sequence with lower degradation in our simulation experiments.

Understanding the dynamics of urban environments is crucial for path planning and safe navigation. However, the dynamics might be extremely complex making learning the environment an unfathomable task. Within the methods available for learning dynamic environments, dynamic Gaussian process occupancy maps (DGPOM) are very attractive because they can produce spatially-continuous occupancy maps taking into account neighborhood information, and provide probabilistic estimates, naturally inferring the uncertainty of predictions. Despite these properties, they are extremely slow, especially in dynamic mapping where the parameters of the map have to be updated as new data arrive from range sensors such as LiDARs. In this work, we leverage recent advancements in stochastic variational inference (SVI) to quickly learn dynamic areas in an online fashion. Further, we propose an information-driven technique to “intelligently” select inducing points required for SVI without relying on any object trackers which essentially improves computational time as well as robustness. These long-term occupancy maps entertain all attractive properties of DGPOM while the learning process is significantly faster, yet accurate. Our experiments with both simulation and real robot data on road intersections show a significant improvement in speed while maintaining a comparable or better accuracy.

Model predictive control (MPC) is a popular control method that has proved effective for robotics, among other fields. MPC performs re-planning at every time step. Re-planning is done with a limited horizon per computational and real-time constraints and often also for robustness to potential model errors. However, the limited horizon leads to suboptimal performance. In this work, we consider the iterative learning setting, where the same task can be repeated several times, and propose a policy improvement scheme for MPC. The main idea is that between executions we can, offline, run MPC with a longer horizon, resulting in a hindsight plan. To bring the next real-world execution closer to the hindsight plan, our approach learns to re-shape the original cost function with the goal of satisfying the following property: short horizon planning (as realistic during real executions) with respect to the shaped cost should result in mimicking the hindsight plan. This effectively consolidates long-term reasoning into the short-horizon planning. We empirically evaluate our approach in contact-rich manipulation tasks both in simulated and real environments, such as peg insertion by a real PR2 robot.

With the advancement of robotics, machine learning, and machine perception, increasingly more robots will enter human environments to assist with daily tasks. However, dynamically-changing human environments requires reactive motion plans. Reactivity can be accomplished through re-planning, e.g. model-predictive control, or through a reactive feedback policy that modifies on-going behavior in response to sensory events. In this paper, we investigate how to use machine learning to add reactivity to a previously learned nominal skilled behavior. We approach this by learning a reactive modification term for movement plans represented by nonlinear differential equations. In particular, we use dynamic movement primitives (DMPs) to represent a skill and a neural network to learn a reactive policy from human demonstrations. We use the well explored domain of obstacle avoidance for robot manipulation as a test bed. Our approach demonstrates how a neural network can be combined with physical insights to ensure robust behavior across different obstacle settings and movement durations. Evaluations on an anthropomorphic robotic system demonstrate the effectiveness of our work.

There has been a great deal of work on learning new robot skills, but very little consideration of how these newly acquired skills can be integrated into an overall intelligent system. A key aspect of such a system is compositionality: newly learned abilities have to be characterized in a form that will allow them to be flexibly combined with existing abilities, affording a (good!) combinatorial explosion in the robot's abilities. In this paper, we focus on learning models of the preconditions and effects of new parameterized skills, in a form that allows those actions to be combined with existing abilities by a generative planning and execution system.

3D hand pose estimation is a hot research topic in recent years. It's been widely used in many advanced applications for virtual reality and human-computer interaction, since it provides a natural interface for communication between human and cyberspace. Despite the fast development of this field, it is still a difficult task due to the various challenges. In this paper, we aim to build a 3D hand pose estimation system which can correctly detect human hands and accurately estimate its pose using depth images. To guarantee the robustness of our system, we design a hand model called spherical part model (SPM), and train a deep convolutional neural network using this model. Moreover, to reduce the influence of human's omissions, we use a data-driven approach to integrate them together. Our network can more accurately estimate hand pose based on prior knowledge of human hand. To demonstrate the superiority of our method, a complete experiment is conducted on two public and one self-built datasets. The results show that our system can detect human hands with average precision at almost 90% and the average error distance of the pose estimation is about 10 millimeters, and is better than the other state of the art works.

Solutions to real world robotic tasks often require complex behaviors in high dimensional continuous state and action spaces. Reinforcement Learning (RL) is aimed at learning such behaviors but often fails for lack of scalability. To address this issue, Hierarchical RL (HRL) algorithms leverage hierarchical policies to exploit the structure of a task. However, many HRL algorithms rely on task specific knowledge such as a set of predefined sub-policies or sub-goals. In this paper we propose a new HRL algorithm based on information theoretic principles to autonomously uncover a diverse set of sub-policies and their activation policies. Moreover, the learning process mirrors the policys structure and is thus also hierarchical, consisting of a set of independent optimization problems. The hierarchical structure of the learning process allows us to control the learning rate of the sub-policies and the gating individually and add specific information theoretic constraints to each layer to ensure the diversification of the sub-policies. We evaluate our algorithm on two high dimensional continuous tasks and experimentally demonstrate its ability to autonomously discover a rich set of sub-policies.

We present LS-ELAS, a line segment extension to the ELAS algorithm, which increases the performance and robustness. LS-ELAS is a binocular dense stereo matching algorithm, which computes the disparities in constant time for most of the pixels in the image and in linear time for a small subset of the pixels (support points). Our approach is based on line segments to determine the support points instead of uniformly selecting them over the image range. This way we find very informative support points which preserve the depth discontinuity. The prior of our Bayesian stereo matching method is based on a set of line segments and a set of support points. Both sets are given to a constrained Delaunay triangulation to generate a triangulation mesh which is aware of possible depth discontinuities. We further increased the accuracy by using an adaptive method to sample candidate points along edge segments. We evaluated our algorithm on the Middlebury benchmark.

This paper introduces a new four-limbed robot, LEMUR 3, that has demonstrated climbing on cliff faces and smooth glass. Each limb on the robot consists of seven identical actuators in a serial chain. Each limb terminates in a single axis force sensor that allows various end effectors to be mounted and connected to the robot's power and communication system. Microspine grippers were used for climbing the rocky surface and gecko adhesive grippers were used for the glass solar panels. All other hardware and much of the software was common for the two demonstrations. The robot's mechanical, electrical, and software systems, various gripping devices, and field demonstrations are described. Limbed mobility is of interest to JPL and NASA because of its potential to access extreme terrain, including that on Mars and in microgravity environments.

We present a mobile robot motion planning approach under kinodynamic constraints that exploits learned perception priors in the form of continuous Gaussian mixture fields. Our Gaussian mixture fields are statistical multi-modal motion models of discrete objects or continuous media in the environment that encode e.g. the dynamics of air or pedestrian flows. We approach this task using a recently proposed circular linear flow field map based on semi-wrapped GMMs whose mixture components guide sampling and rewiring in an RRT* algorithm using a steer function for non-holonomic mobile robots. In our experiments with three alternative baselines, we show that this combination allows the planner to very efficiently generate high-quality solutions in terms of path smoothness, path length as well as natural yet minimum control effort motions through multi-modal representations of Gaussian mixture fields.

A new four degrees-of-freedom (DOF) parallel manipulator that can produce 3-DOF translations and 1-DOF rotation (3T1R), has been proposed in this paper. It has two identical limbs connected to the moving platform through passive revolute joints, and each limb has two identical branches driven by a pair of base mounted collinear prismatic joints. Due to such a unique “4-2-1” kinematic structure, the 4-DOF parallel manipulator has the advantages of simple kinematics, large workspace, high speed, and high positioning accuracy. These advantages make it an appropriate candidate for high-speed and high-precision pick-and-place operations. To validate the proposed parallel manipulator design, mobility analysis is conducted based on the screw theory. Other critical design analysis issues, such as displacement, singularity, and workspace analyses, have been addressed in details.

In this work, we develop a novel keyframe-based dense planar SLAM (KDP-SLAM) system, based on CPU only, to reconstruct large indoor environments in real-time using a hand-held RGB-D sensor. Our keyframe-based approach applies a fast dense method to estimate odometry, fuses depth measurements from small baseline images, extracts planes from the fused depth map, and optimizes the poses of the keyframes and landmark planes in a global factor graph using incremental smoothing and mapping (iSAM). Using the fast odometry estimation, correct plane correspondences may be found projectively, and the pose of each frame can be estimated accurately even without sufficient planes to fully constrain the 6 degree-of-freedom transformation. The depth map generated from the local fusion process generates higher quality reconstructions and plane segmentations by eliminating noise. Moreover, explicitly modeling plane landmarks in the fully probabilistic global optimization significantly reduces the drift that plagues other dense SLAM algorithms. We test our system on standard RGB-D benchmarks as well as additional indoor environments, demonstrating its state-of-the-art performance as a real-time dense 3D SLAM algorithm, without the use of GPU.

In this paper we present a novel joint approach for optimising surface curvature and pose alignment. We present two implementations of this joint optimisation strategy, including a fast implementation that uses two frames and an offline multi-frame approach. We demonstrate an order of magnitude improvement in simulation over state of the art dense relative point-to-plane Iterative Closest Point (ICP) pose alignment using our dense joint frame-to-frame approach and show comparable pose drift to dense point-to-plane ICP bundle adjustment using low-cost depth sensors. Additionally our improved joint quadric based approach can be used to more accurately estimate surface curvature on noisy point clouds than previous approaches.

Robotic assembly tasks are subject to uncertainties arising from part tolerances. A popular approach to deal with such partially unstructured environments is to introduce compliance by using admittance control schemes. For such schemes, contact forces and torques are speed dependent, which often limits the achievable assembly speed. To overcome this limitation, we present a new method for increasing the achievable speed of compliant manipulators by iteratively reducing contact forces. The presented concept of Iteratively Learned and Temporally Scaled Force Control (ILTSFC) is based on two coupled iterative learning controllers where one increases the assembly speed and the other one adjusts the reference trajectory to reduce contact forces. The approach is verified by an experimental peg-in-hole study on an ABB YuMi, a dual-arm collaborative robot with 7DOF each arm.

The unified approach algorithm to control a mobile robot is useful in simplifying path planning and inverse kinematics by considering a mobile platform as hypothetical joints. Applying the damped least square method by using the Jacobian prevents the divergence of the joint angle. However, it is possible to violate the nonholonomic constraint on mobility by distorting the Jacobian. In robot manipulation, the end-effector position error due to a violation of the constraint can be fatal, because such an error can damage a nearby object or the robot itself. In this paper, we propose a method that strictly guarantees nonholonomic constraints on a mobile manipulator. We also propose a compensation technique for faster convergence of the position and orientation of the end effector. The effectiveness of the proposed methods was confirmed through simulations.

We address the problem of finding useful invariant funnels for dynamic walking robots, i.e. sets of initial conditions from which continued walking in a stable manner is guaranteed. The construction is based on transverse dynamics and sum-of-squares verification. This paper makes two main contributions: firstly, we show that for typical models of walking robots the construction of such funnels can be significantly simplified by use of a new phase variable. Secondly, we provide the first hardware validation of the resulting funnels on an experimental testbed.