ICRA 2017 Papers

Autonomous learning of robotic skills can allow general-purpose robots to learn wide behavioral repertoires without extensive manual engineering. However, robotic skill learning must typically make trade-offs to enable practical real-world learning, such as requiring manually designed policy or value function representations, initialization from human demonstrations, instrumentation of the training environment, or extremely long training times. We propose a new reinforcement learning algorithm that can train general-purpose neural network policies with minimal human engineering, while still allowing for fast, efficient learning in stochastic environments. We build on the guided policy search (GPS) algorithm, which transforms the reinforcement learning problem into supervised learning from a computational teacher (without human demonstrations). In contrast to prior GPS methods, which require a consistent set of initial states to which the system must be reset after each episode, our approach can handle random initial states, allowing it to be used even when deterministic resets are impossible. We compare our method to existing policy search algorithms in simulation, showing that it can train high-dimensional neural network policies with the same sample efficiency as prior GPS methods, and can learn policies directly from image pixels. We also present real-world robot results that show that our method can learn manipulation policies with visual features and random initial states.

In this paper we present a passive and reliable explicit discrete integrator, which allows to preserve the energy and dynamic properties of a physical body rendered on a hardware-in-the-loop simulator. Starting from the standard Euler integrator, we identify the energy generation that results from the integration process. This energy makes the time discrete dynamics deviate from the ideal one, resulting in position drifts or stability issues. By exploiting the time domain passivity approach, the simulated dynamics is reshaped in order to preserve its physical energy properties. The proposed integration method allows precise simulation of virtual bodies on industrial robot facilities. The method has been validated in simulation and experimentally tested on the DLR OOS-SIM facility.

A fundamental problem in cooperative HumanRobot Interaction is object handover. Existing works in this area assume the human can reliably grasp the object from the robot hand. However, in some situations the human can produce perturbing forces in the object that are not meant to end in a handover. These perturbations can result in the object being dropped or the robot hand being damaged. This paper addresses this problem and presents a mechanism for reliable robot to human object handover implemented in a Shadow Robot hand endowed with tactile sensing. Given a stable grasping configuration, using BioTAC sensors we are able to estimate the contact forces applied to the object, and provide a feedback signal to a joint effort controller to maintain grasp forces despite perturbations. Our system is able to identify between object pulling forces which should result in an object handover, and other disturbances. Experimental results show that the hand releases the object only when the object is pulled, validating the proposed algorithm.

In this paper, we present a regression for predicting 3D rigid transformations from real-valued vectors. We use a unit dual quaternion to represent the transformation. The regression is formulated as blending unit dual quaternions. To formulate it in a closed form, we introduce an approximation based on error metrics according to geometric algebra. Finally, we take an articulated motion and an elastic deformation as examples to present the descriptive power of our method in modeling the motion and the deformation.

Fetching items is an important problem for a social robot. It requires a robot to interpret a person's language and gesture and use these noisy observations to infer what item to deliver. If the robot could ask questions, it would help the robot be faster and more accurate in its task. Existing approaches either do not ask questions, or rely on fixed question-asking policies. To address this problem, we propose a model that makes assumptions about cooperation between agents to perform richer signal extraction from observations. This work defines a mathematical framework for an item-fetching domain that allows a robot to increase the speed and accuracy of its ability to interpret a person's requests by reasoning about its own uncertainty as well as processing implicit information (implicatures). We formalize the item-delivery domain as a Partially Observable Markov Decision Process (POMDP), and approximately solve this POMDP in real time. Our model improves speed and accuracy of fetching tasks by asking relevant clarifying questions only when necessary. To measure our model's improvements, we conducted a real world user study with 16 participants. Our method achieved greater accuracy and a faster interaction time compared to state-of-the-art baselines. Our model is 2.17 seconds faster (25% faster) than a state-of-the-art baseline, while being 2.1% more accurate.

We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible. We leverage recent advances in Bayesian Convolutional Neural Networks to train and implement a sun detection model that infers a three-dimensional sun direction vector from a single RGB image. Crucially, our method also computes a principled uncertainty associated with each prediction, using a Monte Carlo dropout scheme. We incorporate this uncertainty into a sliding window stereo visual odometry pipeline where accurate uncertainty estimates are critical for optimal data fusion. Our Bayesian sun detection model achieves a median error of approximately 12 degrees on the KITTI odometry benchmark training set, and yields improvements of up to 42% in translational ARMSE and 32% in rotational ARMSE compared to standard VO. An open source implementation of our Bayesian CNN sun estimator (Sun-BCNN) using Caffe is available at https://github.com/utiasSTARS/sun-bcnn-vo.

Various model-based control schemes, e.g. feedback-linearizing control of robotic manipulators with series elastic actuators require time derivatives of the control forces, i.e. of the equations of motion. This is referred to as the higher-order inverse dynamics problem. As this must be evaluated on the controller hardware, computationally efficient formulations are crucial. While recursive O (n) inverse dynamics algorithms are well-established, such for the higherorder inverse dynamics are relatively new. O (n) algorithms for the latter problem were published recently building upon the known recursive inverse dynamics algorithms in terms of Denavit-Hartenber (DH) parameters. The DH convention is restrictive, however. Also the higher-order formulations are naturally very complex. Both issues are addresses in this paper with concepts from screw and Lie group theory. An O (n) algorithm for determining the first and second time derivative of the inverse dynamics solution is presented. The Lie group approach provides a high level of compactness while ensuring the computational efficiency. Being frame invariant this formulation allows for using representations in different frame. Two formulations are presented, one using the body-fixed and the other the so-called hybrid representation of twists. The latter is deemed to be computationally more efficient that the classical body-fixed version. Results are shown for a 6-DOF industrial manipulator.

We present an approach for reconstructing vehicles from a single (RGB) image, in the context of autonomous driving. Though the problem appears to be ill-posed, we demonstrate that prior knowledge about how 3D shapes of vehicles project to an image can be used to reason about the reverse process, i.e., how shapes (back-)project from 2D to 3D. We encode this knowledge in shape priors, which are learnt over a small keypoint-annotated dataset. We then formulate a shape-aware adjustment problem that uses the learnt shape priors to recover the 3D pose and shape of a query object from an image. For shape representation and inference, we leverage recent successes of Convolutional Neural Networks (CNNs) for the task of object and keypoint localization, and train a novel cascaded fully-convolutional architecture to localize vehicle keypoints in images. The shape-aware adjustment then robustly recovers shape (3D locations of the detected keypoints) while simultaneously filling in occluded keypoints. To tackle estimation errors incurred due to erroneously detected keypoints, we use an Iteratively Re-weighted Least Squares (IRLS) scheme for robust optimization, and as a by-product characterize noise models for each predicted keypoint. We evaluate our approach on autonomous driving benchmarks, and present superior results to existing monocular, as well as stereo approaches.

Deep learning approaches have been used to perform classification in several applications with high-dimensional input data. In this paper, we investigate the potential for deep learning for classifying affective touch on robotic skin in a social setting. Three models are considered, a convolutional neural network, a convolutional-recurrent neural network and an autoencoder-recurrent neural network. These models are evaluated on two publicly available affective touch datasets, and compared with models built to classify the same datasets. The deep learning approaches provide a similar level of accuracy, and allows gestures to be predicted in real-time at a rate of 6 to 9 Hertz. The memory requirements of the models demonstrate that they can be implemented on small, inexpensive microcontrollers, demonstrating that classification can be performed in the skin itself by collocating computing elements with the sensor array.

A Spring-Loaded-Inverted Pendulum (SLIP) model has been applied to many legged robots, such as quadruped robots, for realizing trotting, bounding, and galloping motions. The indecipherable damping factors, however, hindered the implementation of the SLIP model in practice. In this paper, a control algorithm is proposed to realize the ideal springy motion of a robotic leg. A Kalman filter with a damped SLIP model as the reference system is utilized for estimating a longitudinal velocity of the robotic leg (i.e., the length change rate between the proximal joint and the tip toe). By cancelling the undesired damping factors through positive feedback based on the Kalman filter estimate, the robotic leg is controlled to realize an undamped SLIP model. The proposed method is verified by simulation and experiment. The results showed that the proposed control algorithm enabled the robotic leg to keep continuously hopping as a spring even in the presence of nonlinear frictions.

There has been an increasing interest in the use of correlation filters for visual object tracking due to their impressive tracking performance. However, existing correlation filter based tracking methods, such as Struck and Kernelized Correlation Filter (KCF), cannot always solve tracking problems in complicated conditions such as heavy occlusion and aggressive motion. In this paper, we proposed a real-time visual tracker via a robust KCF. We start by implementing a search window alignment, based on a motion model with uncertainty, which increases the tracking accuracy for fast moving targets and reduces the padding value to accelerate tracking speed. Next, we establish a combined confidence measurement including occlusion information, which is utilized for robust updating. Then we apply an adaptive Kalman filter to improve the tracking accuracy. Qualitative and quantitative experimental results show that the proposed algorithm outperforms the state-of-the-art methods such as KCF and Struck.

In this paper we present the PUMP (Parallel Uncertainty-aware Multiobjective Planning) algorithm for addressing the stochastic kinodynamic motion planning problem, whereby one seeks a low-cost, dynamically-feasible motion plan subject to a constraint on collision probability (CP). To ensure exhaustive evaluation of candidate motion plans (as needed to tradeoff the competing objectives of performance and safety), PUMP incrementally builds the Pareto front of the problem, accounting for the optimization objective and an approximation of CP. This is performed by a massively parallel multiobjective search, here implemented with a focus on GPUs. Upon termination of the exploration phase, PUMP searches the Pareto set of motion plans to identify the lowest cost solution that is certified to satisfy the CP constraint (according to an asymptotically exact estimator). We introduce a novel particle-based CP approximation scheme, designed for efficient GPU implementation, which accounts for dependencies over the history of a trajectory execution. We present numerical experiments for quadrotor planning wherein PUMP identifies solutions in ~100 ms, evaluating over one hundred thousand partial plans through the course of its exploration phase. The results show that this multiobjective search achieves a lower motion plan cost, for the same CP constraint, compared to a safety buffer-based search heuristic and repeated RRT trials.

Estimating the depth from two images with a baseline has a well-known regular problem: When a line is parallel to the epipolar geometry it is not possible to estimate the depth from pixels on this line. Moreover, the classic measure for the certainty of the depth estimate fails as well: The matching score between the template and any pixel on the epipolar line is perfect. This results for common scenes in incorrect matches with very high confidence, some even resistant to left-right image checks. It is straightforward to try to address this by adding a second stereo head in a perpendicular direction. However, it is nontrivial to identify the failure and fuse the two depth maps in a real-time system. A simple weighted average will alleviate the problem but still result in a very large error in the depth map. Our contributions are: 1) We derive a model to predict the failure of stereo by leveraging the matching scores and 2) we propose a combined cost function to fuse two depth maps from orthogonal stereo heads using the failure prediction, matching score and consistency. We show the resulting system in real-time operation on a low-latency system in indoor, urban and natural environments.

In this work we propose a new method for online motion planning in the task-space for hydraulic actuated soft robots. Our solution relies on the interactive resolution of an inverse kinematics problem, that takes into account the properties (mass, stiffness) of the deformable material used to build the robot. An accurate modeling of the mechanical behavior of hydraulic components is based on a novel GPU parallel method for the real-time computation of fluid weight distribution. The efficiency of the method is further increased by a novel GPU parallel leveraging mechanism. Our complete solution has been integrated within the open-source SOFA framework. In our results, we validate our simulation with a fabricated silicone cylinder and we demonstrate the usage of our approach for direct control of hydraulic soft robots.

In this paper, we propose a real-time spatial shape estimation method for an elastic rod. Our proposed method does not use any image sensor nor shape sensor, but does use a six-axis force/torque sensor placed near one end of the elastic rod. In the proposed shape estimation method, the occlusion problem never occurs principally because the shape of a flexible object will be estimated based on an elastic rod model. We utilize the discretized Kirchhoff elastic rod for shape estimation, which can be viewed as a serial chain of rigid bodies connected with three-degree-of-freedom rotational spring joints. We show that real-time spatial shape estimation is possible based on real-time sensing of force/torque information. The computational time of the proposed shape estimation algorithm is linear in the number of rigid-body partition. Moreover, the required calculations are very simple owing to deep understanding of robot manipulator kinematics. Thus we can compute the spatial rod shape in realtime. We show experimental results in order to validate our proposed method.

We consider a pursuit-evasion problem between humanoids. In our scenario, the pursuer enters the safety area of the evader headed for collision, while the latter executes a fast evasive motion. Control schemes are designed for both the pursuer and the evader. They are structurally identical, although the objectives are different: the pursuer tries to align its direction of motion with the line-of-sight to the evader, whereas the evader tries to move in a direction orthogonal to the line-of-sight to the pursuer. At the core of the control scheme is a maneuver planning module which makes use of closed-form expressions exclusively. This allows its use in a replanning framework, where each robot updates its motion plan upon completion of a step to account for the perceived motion of the other. Simulation and experimental results on NAO humanoids reveal an interesting asymptotic behavior which was predicted using unicycle as template models for trajectory generation.

In this work, we present a solution to real-time monocular dense mapping. A tightly-coupled visual-inertial localization module is designed to provide metric and high-accuracy odometry. A motion stereo algorithm is proposed to take the video input from one camera to produce local depth measurements with semi-global regularization. The local measurements are then integrated into a global map for noise filtering and map refinement. The global map obtained is able to support navigation and obstacle avoidance for aerial robots through our indoor and outdoor experimental verification. Our system runs at 10Hz on an Nvidia Jetson TX1 by properly distributing computation to CPU and GPU. Through onboard experiments, we demonstrate its ability to close the perception-action loop for autonomous aerial robots. We release our implementation as open-source software1.

On the quest of automating the navigation of challenging and promising Robotics platforms such as small Unmanned Aerial Vehicles (UAVs), the community has been increasingly active in developing perception capabilities able to run onboard such platforms in real-time. Despite that vision-based techniques have been at the heart of recent advancements, the realistic employment onboard UAVs is still in its infancy. Inspired by some of the most recent breakthroughs in online dense scene estimation and borrowing fundamental concepts from Computer Vision, in this work we propose a new pipeline for real-time, local scene reconstruction using a single camera for aerial navigation. Aiming for denser scene estimation than traditional feature-based maps with the ability to run onboard a small UAV in real-time, the proposed approach is demonstrated to achieve unprecedented performance producing rich maps of the camera's workspace, timely enough to serve in obstacle avoidance and real-time interaction of a robot with its direct surroundings. Evaluation on benchmarking datasets and on challenging aerial footage captured with a UAV featuring a conventional camera, reveals dramatic speed-ups, as well as denser and more accurate local reconstructions with respect to the state of the art.

The ability to accurately capture locomotion is relevant in various use cases, in particular in the sports and health area. With the major goal of providing a measurement system that can deliver different types of relevant information (3D body segment kinematics, spatiotemporal locomotion parameters, and locomotion patterns) in-field and in real-time, we propose a novel probabilistic (single-plane) ground contact estimation method, using four contact points defined through a biomechanical foot model, and integrate this into an existing inertial motion capturing method. The resulting method is quantitatively evaluated on simulated and real IMU data in comparison to an optical motion capture system on walking, running, and jumping sequences. The results show its ability to maintain a good average 3D kinematics estimation error on low- and high-acceleration locomotion, whereas many previous accuracy studies restrict themselves to movements with low to moderate global accelerations, such as upper body activities or slow locomotion. Moreover, a qualitative evaluation of the estimated ground contact probabilities demonstrates the method's ability to also provide consistent information also for deriving spatiotemporal locomotion parameters as well as locomotion patterns (e.g., over-pronation/-supination) simultaneously with the 3D kinematics.