ICRA 2017 Papers

Belief space planning is concerned with the problem of finding the control policy under process and measurement uncertainties. Formulated as a stochastic control problem, the solution of a general Decentralized Partially Observed Markov Decision Process (Dec-POMDP) is a collection of feedback policies for individual agents, maximizing a joint value function. In this paper, we design (m) number of Linear Quadratic Gaussian (LQG) policies for (m) number of agents maximizing the joint performance of the team. Casting the problem as a NonLinear Program (NLP), we propose a framework that reduces the optimization dimension from ((mn)2 + mn) to (mn) with (n) referring to the dimension of each individual agent's state space. As a result, the proposed method reduces the formidable generic Dec-POMDP to a computationally tractable multi-agent planning under uncertainty. Our results in 2D and 3D environments demonstrate the performance of the algorithm and its ability to predict and avoid inter-agent collisions.

Simultaneous Localisation and Mapping (SLAM) systems that recover the trajectory of a robot or mobile device are characterised by a front-end and back-end. The front-end uses sensor observations to identify loop closures; the back-end optimises the estimated trajectory to be consistent with these closures. The GraphSLAM framework formulates the back-end problem as a graph-based optimisation on a pose graph. This paper describes a back-end system optimised for very dense sequence-based loop closures. This arises when the frontend generates magnetic loop closures, among other things. Magnetic measurements are fast varying, which is good for localisation, but the requirement for high sampling rates (50 Hz+) produces many more loop closures than conventional systems. To date, however, there is no study optimising GraphSLAM back-end for sequence-based magnetic loop closures. Hence we introduce a novel variant of the Stochastic Gradient Descent-based SLAM algorithm called MSGD (Magnetic-SGD). We use high-accuracy groundtruth system and extensive real datasets to evaluate MSGD against state-of-the-art back-end algorithms. We demonstrate MSGD is at least as good as the best competitor algorithm in terms of quality, while being faster and more scalable.

We present a novel model-free method for online 3D semantic scene parsing from video sequences. MF3D (Model-Free 3D) is different from conventional methods for 3D scene parsing in that voxel labelling is approached via search-based label transfer instead of discriminative classification. This non-parametric approach makes MF3D easy to scale with an online growth in the database, as no model re-training is required with the addition of new examples or categories. Experimental results on the KITTI benchmark demonstrate that our model-free approach enables accurate online 3D scene parsing while retaining extensibility to new categories. In addition, we show that unsupervised binary encoding (hashing) techniques can be easily incorporated into our framework for scalability to larger databases.

In this paper we present a low-cost and easy to fabricate 3-axis tactile sensor based on magnetic technology. The sensor consists in a small magnet immersed in a silicone body with an Hall-effect sensor placed below to detect changes in the magnetic field caused by displacements of the magnet, generated by an external force applied to the silicone body. The use of a 3-axis Hall-effect sensor allows to detect the three components of the force vector, and the proposed design assures high sensitivity, low hysteresis and good repeatability of the measurement: notably, the minimum sensed force is about 0.007N. All components are cheap and easy to retrieve and to assemble; the fabrication process is described in detail and it can be easily replicated by other researchers. Sensors with different geometries have been fabricated, calibrated and successfully integrated in the hand of the human-friendly robot Vizzy. In addition to the sensor characterization and validation, real world experiments of object manipulation are reported, showing proper detection of both normal and shear forces.

This paper presents a method for predicting slip using Gaussian process regression. Slip models are learned for visually classified terrain types as a function of terrain geometry. Spatial correlations between terrain properties are leveraged for on-line slip model adaptation. Results show that regression-based modeling using in-situ rover data outperforms the state-of-practice, terrestrially-calibrated slip curves in both mean prediction and uncertainty bounds. Local adaptation improves slip prediction results, particularly in high-slip sand areas that pose the greatest threat to rovers. Slip estimates made using a visual classifier to identify terrain type are compared to estimates using on-line model selection with only proprioceptive slip measurements as inputs. The proprioceptive results nearly match the visual results, showing that this approach could work even when a visual classifier is not available.

There is an increasing interest in the use of soft technologies for robotic application, however, the lack of advanced sensory motor capabilities is currently a significant limitation. Sensing of soft robots, in particular, is still not fully understood and methods by which a large deformable continuum body can be effectively sensed without loosing the intrinsic soft body dynamics are limited. This paper proposes a novel soft body sensing method, localized differential sensing, whereby the sensing of localized deformation on a large soft structure can be achieved with a pair of strain sensors. The design principles for this method are given. To demonstrate this sensing method Conductive Thermoplastic Elastomer (CTPE) is used for the sensing of deformation of soft body structures. A feasibility study of the approach is also presented, with strain sensors incorporated into the universal gripper to allow detection of grasped objects.

Active assistive mobility systems are largely limited to a-priori mapped environments, whereas their reactive assistive counterparts are in general location independent and focus on the provision of collision avoidance in the immediate space surrounding the platform. This paper presents a framework capable of providing active short-term navigation, combining the intelligence of active assistance with the freedom of location independence. Demonstration data from an able expert while driving the mobility aid in a standard indoor setting is used off-line to learn reference behavioral models of navigation given perceptual information from the platform surroundings and the input controls exerted by the user while navigating. These serve as the foundation for on-line probabilistic short-term destination inference using the instantaneously available data from the user and on-board sensors. This is coupled with a real-time stochastic optimal path generation able to exploit the same short term demonstration paths from the expert with the belief they capture both the driver's awareness of the platform's physical geometry and appropriate behaviors for their surroundings. Experimental results with users of varying proficiency in a setting unvisited in training data show promise in using the framework in assisting users experiencing difficulty in safe power mobility aid use.

Detection of traversable road regions, positive and negative obstacles, and water hazards is a fundamental task for autonomous driving vehicles, especially in off-road environment. This paper proposes an efficient method, called Lidar-histogram. It can be used to integrate the detection of traversable road regions, obstacles and water hazards into one single framework. The weak assumption of the Lidar-histogram is that a decent-sized area in front of the vehicle is flat. The Lidar-histogram is derived from an efficient organized map of Lidar point cloud, called Lidar-imagery, to index, describe and store Lidar data. The organized point-cloud map can be easily obtained by indexing the original unordered 3D point cloud to a Lidar-specific 2D coordinate system. In the Lidar-histogram representation, the 3D traversable road plane in front of vehicle can be projected as a straight line segment, and the positive and negative obstacles are projected above and below the line segment, respectively. In this way, the problem of detecting traversable road and obstacles is converted into a simple linear classification task in 2D space. Experiments have been conducted in different kinds of off-road and urban scenes, and we have obtained very promising results.

Urban environments are characterised by the presence of distinctive audio signals which alert the drivers to events that require prompt action. The detection and interpretation of these signals would be highly beneficial for smart vehicle systems, as it would provide them with complementary information to navigate safely in the environment. In this paper, we present a framework that spots the presence of acoustic events, such as horns and sirens, using a two-stage approach. We first model the urban soundscape and use anomaly detection to identify the presence of an anomalous sound, and later determine the nature of this sound. As the audio samples are affected by copious non-stationary and unstructured noise, which can degrade classification performance, we propose a noise-removal technique to obtain a clean representation of the data we can use for classification and waveform reconstruction. The method is based on the idea of analysing the spectrograms of the incoming signals as images and applying spectrogram segmentation to isolate and extract the alerting signals from the background noise. We evaluate our framework on four hours of urban sounds collected driving around urban Oxford on different kinds of road and in different traffic conditions. When compared to traditional feature representations, such as Mel-frequency cepstrum coefficients, our framework shows an improvement of up to 31% in the classification rate.

This work presents Experience-driven Predictive Control (EPC) as a fast technique for solving nonlinear model predictive control (NMPC) problems with uncertain system dynamics. EPC leverages an affine dynamics model that is updated online via Locally Weighted Projection Regression (LWPR) to capture nonlinearities, uncertainty, and changes in the system dynamics. This model enables the NMPC problem to be re-cast as a quadratic program (QP). The QP can then be solved via multi-parametric techniques to generate a mapping from state, reference, and dynamics model to a locally optimal, affine feedback control law. These mappings, in conjunction with the basis functions learned via LWPR, define a notion of experience for the controller as they capture the full input-output relationship for previous actions the controller has taken. The resulting experience database allows EPC to avoid solving redundant optimization problems, and as it is constructed online, enables the system to operate more efficiently over time. We demonstrate the performance of EPC through a set of hardware-in-the-loop simulation studies of a quadrotor micro air vehicle that is subjected to unmodeled exogenous perturbations.

3D reconstruction methods based on active stereo technique have been widely used for many practical systems. Many of these systems are configured with a single camera and a single projector. Since such systems can only capture one side of the target object, several attempts have been conducted to enlarge the captured area, especially multi-projector systems attract many researchers. For multi-projector based systems, overlap between multiple pattern projections is a serious problem. Even if different color channels are used for each projector, complete separation is not possible because of color crosstalks. Another open problem is decoding errors of the projected patterns, which causes a failure on extracting positional information of the projected pattern form the captured image. Among several reasons for such errors, color crosstalks are crucial because their features are similar to the main signal and difficult to be decomposed. In this paper, we solve these problems by utilizing machine learning techniques where a convolutional neural network is trained to extract low dimensional pattern features for each projector. In addition, it is trained to suppress the color crosstalks from different projectors. Using this new technique, we succeeded in reconstructing 3D shapes from images where multiple patterns are overlapped.

The sense of touch, being the earliest sensory system to develop in a human body [1], plays a critical part of our daily interaction with the environment. In order to successfully complete a task, many manipulation interactions require incorporating haptic feedback. However, manually designing a feedback mechanism can be extremely challenging. In this work, we consider manipulation tasks that need to incorporate tactile sensor feedback in order to modify a provided nominal plan. To incorporate partial observation, we present a new framework that models the task as a partially observable Markov decision process (POMDP) and learns an appropriate representation of haptic feedback which can serve as the state for a POMDP model. The model, that is parametrized by deep recurrent neural networks, utilizes variational Bayes methods to optimize the approximate posterior. Finally, we build on deep Q-learning to be able to select the optimal action in each state without access to a simulator. We test our model on a PR2 robot for multiple tasks of turning a knob until it clicks.

Recently, end-to-end learning frameworks are gaining prevalence in the field of robot control. These frameworks input states/images and directly predict the torques or the action parameters. However, these approaches are often critiqued due to their huge data requirements for learning a task. The argument of the difficulty in scalability to multiple tasks is well founded, since training these tasks often require hundreds or thousands of examples. But do end-to-end approaches need to learn a unique model for every task? Intuitively, it seems that sharing across tasks should help since all tasks require some common understanding of the environment. In this paper, we attempt to take the next step in data-driven end-to-end learning frameworks: move from the realm of task-specific models to joint learning of multiple robot tasks. In an astonishing result we show that models with multi-task learning tend to perform better than task-specific models trained with same amounts of data. For example, a deep-network learned with 2.5K grasp and 2.5K push examples performs better on grasping than a network trained on 5K grasp examples.

The varied and complex dynamics of deformable terrain are significant impediments toward real-world viability of locomotive robotics, particularly for legged machines. We explore vertical jumping on granular media (GM) as a model task for legged locomotion on uncharacterized deformable terrain. By integrating (Gaussian process) GP-based regression and evaluation to estimate ground forcing as a function of state, a one-dimensional jumper acquires the ability to learn forcing profiles exerted by its environment in tandem to achieving its control objective. The GP-based dynamical model initially assumes a baseline rigid, non-compliant surface. As part of an iterative procedure, the optimizer employing this model generates an optimal control to achieve a target jump height while respecting known hardware limitations of the robot model. Trajectory and forcing data recovered from evaluation on the true GM surface model simulation is applied to train the GP, and in turn, provide the optimizer a more richly informed dynamical model of the environment. After three iterations, predicted optimal control trajectories coincide with execution results, within 1.2% jumping height error, as the GP-based approximation converges to the true GM model.

In this paper, we propose a learning algorithm that speeds up the search in task and motion planning problems. Our algorithm proposes solutions to three different challenges that arise in learning to improve planning efficiency: what to predict, how to represent a planning problem instance, and how to transfer knowledge from one problem instance to another. We propose a method that predicts constraints on the search space based on a generic representation of a planning problem instance, called score space, where we represent a problem instance in terms of performance of a set of solutions attempted so far. Using this representation, we transfer knowledge, in the form of constraints, from previous problems based on the similarity in score space. We design a sequential algorithm that efficiently predicts these constraints, and evaluate it in three different challenging task and motion planning problems. Results indicate that our approach perform orders of magnitudes faster than an unguided planner.

The budgeted information gathering problem - where a robot with a fixed fuel budget is required to maximize the amount of information gathered from the world - appears in practice across a wide range of applications in autonomous exploration and inspection with mobile robots. Although there is an extensive amount of prior work investigating effective approximations of the problem, these methods do not address the fact that their performance is heavily dependent on distribution of objects in the world. In this paper, we attempt to address this issue by proposing a novel data-driven imitation learning framework. We present an efficient algorithm, EXPLORE, that trains a policy on the target distribution to imitate a clairvoyant oracle - an oracle that has full information about the world and computes non-myopic solutions to maximize information gathered. We validate the approach on a spectrum of results on a number of 2D and 3D exploration problems that demonstrates the ability of EXPLORE to adapt to different object distributions. Additionally, our analysis provides theoretical insight into the behavior of EXPLORE. Our approach paves the way forward for efficiently applying data-driven methods to the domain of information gathering.

Recent studies suggest that reinforcement learning has great potential for generating assistive strategies in exoskeletons through physical interactions between a user and a robot. Previous methods focused on a task-specific assistive strategy, where for every single task (situation/context), the user needs to interact with a robot to learn an appropriate assistive strategy. Therefore, the learned strategies cannot be generalized for a new task. Since the sampling cost is expensive for such human-in-the-loop systems as exoskeletons, generalization must be enabled. In this paper, we propose to learn task-parametrized assistive strategies for exoskeleton robots. Our method employs an assistive strategy, which depends on the task parameter and the state variable, that can be learned from multiple sets of human-robot interaction data across different tasks and generalized even for an unseen task, given the task parameter without additional learning. To alleviate the user's burden in the learning process across multiple tasks, we exploit a data-efficient multi-task reinforcement learning framework. To verify the effectiveness of our method, we developed an experimental platform with an exoskeleton robot. We conducted a series of experiments whose experimental results show that our method can learn such a task-parametrized assistive strategy and be generalized for unseen tasks to reduce the user's electromyography signals (EMGs) during tasks.

Many human skills can be described in terms of performing a set of prioritised tasks. While a number of tools have become available that recover the underlying control policy from constrained movements, few have explicitly considered learning how constraints should be imposed in order to perform the control policy. In this paper, a method for learning the self-imposed constraints present in movement observations is proposed. The problem is formulated into the operational space control framework, where the goal is to estimate the constraint matrix and its null space projection that decompose the task space and any redundant degrees of freedom. The proposed method requires no prior knowledge about either the dimensionality of the constraints nor the underlying control policies. The techniques are evaluated on a simulated three degree-of-freedom arm and on the AR10 humanoid hand.

In this paper, we present a general framework for learning social affordance grammar as a spatiotemporal AND-OR graph (ST-AOG) from RGB-D videos of human interactions, and transfer the grammar to humanoids to enable a real-time motion inference for human-robot interaction (HRI). Based on Gibbs sampling, our weakly supervised grammar learning can automatically construct a hierarchical representation of an interaction with long-term joint sub-tasks of both agents and short term atomic actions of individual agents. Based on a new RGB-D video dataset with rich instances of human interactions, our experiments of Baxter simulation, human evaluation, and real Baxter test demonstrate that the model learned from limited training data successfully generates human-like behaviors in unseen scenarios and outperforms both baselines.