IROS 2017 Papers

We present a novel method for classifying 3D objects that is particularly tailored for the requirements in robotic applications. The major challenges here are the comparably small amount of available training data and the fact that often data is perceived in streams and not in fixed-size pools. Traditional state-of-the-art learning methods, however, require a large amount of training data, and their online learning capabilities are usually limited. Therefore, we propose a modality-specific selection of convolutional neural networks (CNN), pre-trained or fine-tuned, in combination with a classifier that is designed particularly for online learning from data streams, namely the Mondrian Forest (MF). We show that this combination of trained features obtained from a CNN can be improved further if a feature selection algorithm is applied. In our experiments, we use the resulting features both with a MF and a linear Support Vector Machine (SVM). With SVM we beat the state of the art on an RGB-D dataset, while with MF a strong result for active learning is achieved.

Discrete submovements are the building blocks of any complex movement. When robots collaborate with humans, extraction of such submovements can be very helpful in applications such as robot-assisted rehabilitation. Our work aims to segment these submovements based on the invariant geometric information embedded in segment kinematics. Moreover, this segmentation is achieved without any explicit kinematic representation. Our work demonstrates the usefulness of this invariant framework in segmenting a variety of humeral movements, which are performed at different speeds across different subjects. Our results indicate that this invariant framework has high computational reliability despite the inherent variability in human motion.

Recent robotic manipulation competitions have highlighted that sophisticated robots still struggle to achieve fast and reliable perception of task-relevant objects in complex, realistic scenarios. To improve these systems' perceptive speed and robustness, we present SegICP, a novel integrated solution to object recognition and pose estimation. SegICP couples convolutional neural networks and multi-hypothesis point cloud registration to achieve both robust pixel-wise semantic segmentation as well as accurate and real-time 6-DOF pose estimation for relevant objects. Our architecture achieves 1 cm position error and <; 5° angle error in real time without an initial seed. We evaluate and benchmark SegICP against an annotated dataset generated by motion capture.

In this work, we propose a search-based planning method to compute dynamically feasible trajectories for a quadrotor flying in an obstacle-cluttered environment. Our approach searches for smooth, minimum-time trajectories by exploring the map using a set of short-duration motion primitives. The primitives are generated by solving an optimal control problem and induce a finite lattice discretization on the state space which can be explored using a graph-search algorithm. The proposed approach is able to generate resolution-complete (i.e., optimal in the discretized space), safe, dynamically feasibility trajectories efficiently by exploiting the explicit solution of a Linear Quadratic Minimum Time problem. It does not assume a hovering initial condition and, hence, is suitable for fast online re-planning while the robot is moving. Quadrotor navigation with online re-planning is demonstrated using the proposed approach in simulation and physical experiments and comparisons with trajectory generation based on state-of-art quadratic programming are presented.

Efficient exploration of unknown terrains by extraterrestrial rovers requires the development of strategies that reduce the entropy in the geological classification of a given terrain. Without such intelligent strategies, teleoperation of the rover is reliant either on human intuition or on the exhaustive exploration of the entire terrain. This paper highlights the use of low-resolution reconnaissance using satellite imagery to generate plans for rovers that reduce the overall uncertainty in the various geological classes. This becomes pivotal when exploration to collect diverse samples is resource constrained through exploration budgets and transmission bandwidths. We put forward two major contributions - a science-aware planner that uses information gain and a novel method of estimating this information gain. We propose an exploration strategy, based on the Multi-Heuristic A*, to solve the trade-off between optimizing path lengths and geological exploration through Pareto-optimal solutions. We show that our algorithm, which explicitly uses projected entropy-reduction in planning, significantly outperforms science-agnostic approaches and other science-aware strategies like greedy best-first searches. We further propose a feature-space based entropy formulation in contrast to the frequently used differential entropy formulation and show superior results when reconstructing the unsampled data from the set of sampled points.

In this paper we introduce a novel sparsification method for efficient decision making under uncertainty and belief space planning in high dimensional state spaces. By using a sparse version of the state's information matrix, we are able to improve the high computational cost of examination of all candidate actions. We also present an in-depth analysis for the general case of approximated decision making, and use it in order to set bounds over the induced error in potential revenue. The scalability of the method allows balancing between the degree of sparsification and the tolerance for this error, in order to maximize its benefits. The approach differs from recent methods by focusing on improving the decision making process directly, and not as a byproduct of a sparsification of the state inference. Eventually, we demonstrate the superiority of the approach in a SLAM simulation, where we manage to maintain the accuracy of the solution, while demonstrating a significant improvement in run time.

In this paper, we propose a novel algorithm for learning from demonstration, which can learn a policy function robustly from a large number of demonstrations with mixed qualities. While most of the existing approaches assume that demonstrations are collected from skillful experts, the proposed method alleviates such restrictions by estimating the proficiency level of each demonstration using the proposed leverage optimization. Furthermore, a novel leveraged cost function is proposed to represent a policy function using deep neural networks by reformulating the objective function of leveraged Gaussian process regression using the representer theorem. The proposed method is successfully applied to autonomous track driving tasks, where a large number of demonstrations with mixed qualities are given as training data without labels.

Sampling-based methods have advanced the state of the art in robotic motion planning and control across complex, high-dimensional domains. With few exceptions, such approaches only admit simple constraints and objectives, such as collision-avoidance and reaching a goal state. In this work we leverage the best of two worlds: the scalability of sampling-based motion planning and the precise formal guarantees of temporal logic. We present an incremental sampling-based algorithm that synthesizes a motion control policy satisfying a bounded Signal Temporal Logic formula over properties of a given environment. Our key insight is that we can bias the selection of samples using a quantitative measure of how well the best path in the current tree of samples satisfies the specification. This allows us both to converge to a path that satisfies the specification, and to improve upon an existing path, i.e. to satisfy the specification with maximum robustness. We illustrate the performance of our method in several case studies.

Camera-based navigation within a given three-dimensional map enables heterogeneous robotic systems to share maps and use more abstract environment models like floor plans. This paper builds upon our previous work, and addresses the problem of how to combine 3D distance information from the map and the current visual image from the robot's camera in order to navigate within this map. The underlying assumption is that features which cause depth changes are also likely to create visual gradients. Based on this assumption, the similarity of visual image and depth images that are synthesized from the 3D map can be used to evaluate pose hypothesis. This paper integrates this idea into a Monte Carlo localization approach and additionally presents its application to path following. The presented approach is evaluated on a synthetic datasets that provides perfect knowledge of the ground truth, as well as two real-world datasets acquired by a heterogeneous robotic team: a proof-of-concept dataset in a scattered indoor environment, and a challenging corridor dataset.

This paper presents a novel sampling-based motion planning method for shape inspection applications with a redundant robotic system. In this paper, a 7-Degree-of-Freedom (DOF) redundant robotic system consisting of a 6-DOF manipulator and a 1-DOF turntable is used for the industrial inspection problem. A Set Covering Problem (SCP) is formulated to select suitable viewpoints that satisfy the inspection requirements, and a Generalized Travelling Salesman Problem (GTSP) is formulated to determine both the robot poses and the visiting sequences. While previous studies solve the two problems separately, we formulate the SCP and GTSP problems as a combined sequencing SC-GTSP problem. A Random-Key Genetic Algorithm (RKGA) is then used to solve the combined SC-GTSP problem in a one-step optimization process. To validate the effectiveness of our method, we applied the proposed method to several motion planning cases. The results show that the proposed method outperforms the previous approaches by requiring up to 28.1% less total inspection time.

We propose a new saliency-guided method for generating supervoxels in 3D space. Rather than using an evenly distributed spatial seeding procedure, our method uses visual saliency to guide the process of supervoxel generation. This results in densely distributed, small, and precise supervoxels in salient regions which often contain objects, and larger supervoxels in less salient regions that often correspond to background. Our approach largely improves the quality of the resulting supervoxel segmentation in terms of boundary recall and under-segmentation error on publicly available benchmarks.

This paper addresses the problem of selecting from a choice of possible grasps, so that impact forces will be minimised if a collision occurs while the robot is moving the grasped object along a post-grasp trajectory. Such considerations are important for safety in human-robot interaction, where even a certified “human-safe” (e.g. compliant) arm may become hazardous once it grasps and begins moving an object, which may have significant mass, sharp edges or other dangers. Additionally, minimising collision forces is critical to preserving longevity of robots which operate in uncertain and hazardous environments, e.g. robots deployed for nuclear decommissioning, where removing a damaged robot from a contaminated zone for repairs may be extremely difficult and costly. Also, unwanted collisions between a robot and critical infrastructure (e.g. pipework) in such high-consequence environments can be disastrous. In this paper we investigate how the safety of the post-grasp motion can be considered during the pre-grasp approach phase, so that the selected grasp is optimal in terms of applying minimum impact forces if a collision occurs during the desired post-grasp manipulation. We build on the methods of augmented robot-object dynamic model and “effective mass” and propose a method for combining these concepts with modern grasp and trajectory planners, to enable the robot to achieve a grasp which maximises the safety of the post-grasp trajectory, by minimising potential collision forces. We demonstrate the effectiveness of our approach through several experiments with both simulated and real robots.

In order to perform autonomous sequential manipulation tasks, perception in cluttered scenes remains a critical challenge for robots. In this paper, we propose a probabilistic approach for robust sequential scene estimation and manipulation - Sequential Scene Understanding and Manipulation (SUM). SUM considers uncertainty due to discriminative object detection and recognition in the generative estimation of the most likely object poses maintained over time to achieve a robust estimation of the scene under heavy occlusions and unstructured environment. Our method utilizes candidates from discriminative object detector and recognizer to guide the generative process of sampling scene hypothesis, and each scene hypotheses is evaluated against the observations. Also SUM maintains beliefs of scene hypothesis over robot physical actions for better estimation and against noisy detections. We conduct extensive experiments to show that our approach is able to perform robust estimation and manipulation.

Active depth cameras suffer from several limitations, which cause incomplete and noisy depth maps, and may consequently affect the performance of RGB-D Odometry. To address this issue, this paper presents a visual odometry method based on point and line features that leverages both measurements from a depth sensor and depth estimates from camera motion. Depth estimates are generated continuously by a probabilistic depth estimation framework for both types of features to compensate for the lack of depth measurements and inaccurate feature depth associations. The framework models explicitly the uncertainty of triangulating depth from both point and line observations to validate and obtain precise estimates. Furthermore, depth measurements are exploited by propagating them through a depth map registration module and using a frame-to-frame motion estimation method that considers 3D-to-2D and 2D-to-3D reprojection errors, independently. Results on RGB-D sequences captured on large indoor and outdoor scenes, where depth sensor limitations are critical, show that the combination of depth measurements and estimates through our approach is able to overcome the absence and inaccuracy of depth measurements.

Small 3D LIDAR and a multimodal-based localization are fundamentally important for autonomous robots. This paper describes presentation and demonstration of a sensor and a method for LIDAR-image based localization. Our small LIDAR, named SPAD LIDAR, uses a single-photon avalanche diode (SPAD). The SPAD LIDAR incorporates laser receiver and environmental light receiver in a single chip. Therefore, the sensor simultaneously outputs range data and monocular image data. By virtue of this structure, the sensor requires no external calibration between range data and monocular image data. Based on this sensor, we introduce a localization method using a deep convolutional neural network (SPAD DCNN), which fuses SPAD LIDAR outputs: range data, monocular image data, and peak intensity data. Our method regresses LIDAR's position in an environment. We also introduce improved SPAD DCNN, designated as Fast SPAD DCNN. To reduce the computational demands of SPAD DCNN, Fast SPAD DCNN integrates range data and peak intensity data. The integrated data reduces runtime without greatly increasing localization error compared to the conventional method. We evaluate our SPAD DCNN and Fast SPAD DCNN localization method in indoor environments and compare its performance. Results show that SPAD DCNN and Fast SPAD DCNN improve localization in terms of accuracy and runtime.

Interpreting the semantics and motion of objects are prerequisites for autonomous robots that enable them to reason and operate in dynamic real-world environments. Existing approaches that tackle the problem of semantic motion segmentation consist of long multistage pipelines and typically require several seconds to process each frame. In this paper, we present a novel convolutional neural network architecture that learns to predict both the object label and motion status of each pixel in an image. Given a pair of consecutive images, the network learns to fuse features from self-generated optical flow maps and semantic segmentation kernels to yield pixel-wise semantic motion labels. We also introduce the Cityscapes-Motion dataset which contains over 2,900 manually annotated semantic motion labels, which is the largest dataset of its kind so far. We demonstrate that our network outperforms existing approaches achieving state-of-the-art performance on the KITTI dataset, as well as in the more challenging Cityscapes-Motion dataset while being substantially faster than existing techniques.

Scanning electron microscope (SEM) has been widely used for characterization and manipulation in micro/nano field. However, existing SEMs can only provide images for samples from one single direction. This limitation sometimes would lead to misleading results. In order to provide multidirectional SEM imaging, a nanorobotic manipulation system with a rotation robot is designed and integrated with SEM. During the robot's rotation, samples fixed on the robot can be viewed by SEM from multi-direction, instead of one direction. The rotation robot is able to rotate 360° along one rotation axis. To facilitate the multidirectional imaging, an automatic alignment method is put forward. This automatic alignment is to align samples to the rotation robot's rotation axis, so that during rotation, samples wouldn't move out of SEM's field of view (FOV). Alignment results are given to justify the proposed alignment method. Magnetic wires have been observed from multi-direction to demonstrate the nanorobotic manipulation system's potential application in mic.ro/nano defect study.

We introduce a robust control architecture for the whole-body motion control of torque controlled robots with arms and legs. The method is based on the robust control of contact forces in order to track a planned Center of Mass trajectory. Its appeal lies in the ability to guarantee robust stability and performance despite rigid body model mismatch, actuator dynamics, delays, contact surface stiffness, and unobserved ground profiles. Furthermore, we introduce a task space decomposition approach which removes the coupling effects between contact force controller and the other non-contact controllers. Finally, we verify our control performance on a quadruped robot and compare its performance to a standard inverse dynamics approach on hardware.

Event cameras are a new technology that can enable low-latency, fast visual sensing in dynamic environments towards faster robotic vision as they respond only to changes in the scene and have a very high temporal resolution (<; 1μs). Moving targets produce dense spatio-temporal streams of events that do not suffer from information loss “between frames”, which can occur when traditional cameras are used to track fast-moving targets. Event-based tracking algorithms need to be able to follow the target position within the spatio-temporal data, while rejecting clutter events that occur as a robot moves in a typical office setting. We introduce a particle filter with the aim to be robust to temporal variation that occurs as the camera and the target move with different relative velocities, which can lead to a loss in visual information and missed detections. The proposed system provides a more persistent tracking compared to prior state-of-the-art, especially when the robot is actively following a target with its gaze. Experiments are performed on the iCub humanoid robot performing ball tracking and gaze following.