IROS 2019 论文列表

When a humanoid robot manipulates a heavy object, balance and heavy loads become problems. To solve these problems, we provide a method to generate a key pose sequence of the robot and the object which balance constraints and joint torque limits are kept. As we consider the configurations of both the robot and the object, the key poses of them are optimized in a view of kinematics and statics. Moreover, we generate a smooth key pose sequence by adding an objective function which makes adjacent poses closer. By using the proposed method, we make a humanoid robot RHP4B place a heavy suitcase on a step.

We present a learning-based method for representing grasp poses of a high-DOF hand using neural networks. Due to redundancy in such high-DOF grippers, there exists a large number of equally effective grasp poses for a given target object, making it difficult for the neural network to find consistent grasp poses. We resolve this ambiguity by generating an augmented dataset that covers many possible grasps for each target object and train our neural networks using a consistency loss function to identify a one-to-one mapping from objects to grasp poses. We further enhance the quality of neural-network-predicted grasp poses using a collision loss function to avoid penetrations. We use an object dataset that combines the BigBIRD Database, the KIT Database, the YCB Database, and the Grasp Dataset to show that our method can generate high-DOF grasp poses with higher accuracy than supervised learning baselines. The quality of the grasp poses is on par with the groundtruth poses in the dataset. In addition, our method is robust and can handle noisy object models such as those constructed from multi-view depth images, allowing our method to be implemented on a 25-DOF Shadow Hand hardware platform.

In order to operate autonomously, a robot should explore the environment and build a model of each of the surrounding objects. A common approach is to carefully scan the whole workspace. This is time-consuming. It is also often impossible to reach all the viewpoints required to acquire full knowledge about the environment. Humans can perform shape completion of occluded objects by relying on past experience. Therefore, we propose a method that generates images of an object from various viewpoints using a single input RGB image. A deep neural network is trained to imagine the object appearance from many viewpoints. We present the whole pipeline, which takes a single RGB image as input and returns a sequence of RGB and depth images of the object. The method utilizes a CNN-based object detector to extract the object from the natural scene. Then, the proposed network generates a set of RGB and depth images. We show the results both on a synthetic dataset and on real images.

Similarity distance measure between two trajectories is an essential tool to understand patterns in motion, for example, in Human-Robot Interaction or Imitation Learning. The problem has been faced in many fields, from Signal Processing, Probabilistic Theory field, Topology field or Statistics field. Anyway, up to now, none of the trajectory similarity measurement metrics are invariant to all possible linear transformation of the trajectories (rotation, scaling, reflection, shear mapping or squeeze mapping). Also not all of them are robust in front of noisy signals or fast enough for real-time trajectory classification. To overcome this limitation this paper proposes a similarity distance metric that will remain invariant in front of any possible linear transformation. Based on Pearson's Correlation Coefficient and the Coefficient of Determination, our similarity metric, the Generalized Multiple Correlation Coefficient (GMCC) is presented like the natural extension of the Multiple Correlation Coefficient. The motivation of this paper is two-fold: First, to introduce a new correlation metric that presents the best properties to compute similarities between trajectories invariant to linear transformations and compare it with some state of the art similarity distances. Second, to present a natural way of integrating the similarity metric in an Imitation Learning scenario for clustering robot trajectories.

To generate a dynamic bipedal walking with foot rolling motion for bipedal robot, hybrid trajectory optimization is capable of planning level walking with great energetic efficiency. However, the direct implementation of this optimization requires different sets of variables to express different active contact constraints, which can be complicated to implement. To simplify the optimization formulation, we propose the generalized contact constraints where the same set of variables are used through all the walking phases. By changing the variable and constraint bounds, different contact constraints for different contact conditions can be generally expressed. The proposed modifications are applied on the bipedal robot AMBER 3, where the optimization results on different terrains are compared and discussed. On the other hand, it is known that a randomized initial guess can be used to solve this optimization, yet its effect on the gaits on different terrains is unclear. As a result, we analyzed the sensitivity of the optimization to a set of randomized initial guesses. The level and downslope walking gaits are also validated via the experiments on AMBER 3.

Hand guidance emerged from the safety requirements for collaborative robots, namely possessing joint-torque sensors. Since then it has proven to be a powerful tool for easy trajectory programming, allowing lay-users to reprogram robots intuitively. Going beyond, a robot can learn tasks by user demonstrations through kinesthetic teaching, enabling robots to generalise tasks and further reducing the need for reprogramming. However, hand guidance is still mostly relegated to collaborative robots. Here we propose a method that does not require any sensors on the robot or in the robot cell, by using a Microsoft HoloLens augmented reality head mounted display. We reference the robot using a registration algorithm to match the robot model to the spatial mesh. The in-built hand tracking and localisation capabilities are then used to calculate the position of the hands relative to the robot. By decomposing the hand movements into orthogonal rotations and propagating it down through the kinematic chain, we achieve a generalised hand guidance without the need to build a dynamic model of the robot itself. We tested our approach on a commonly used industrial manipulator, the KUKA KR-5.

Anticipating a human collaborator's intention enables safe and efficient interaction between a human and an autonomous system. Specifically, in the context of semiautonomous driving, studies have revealed that correct and timely prediction of the driver's intention needs to be an essential part of Advanced Driver Assistance System (ADAS) design. To this end, we propose a framework that exploits drivers' time-series eye gaze and fixation patterns to anticipate their real-time intention over possible future manoeuvres, enabling a smart and collaborative ADAS that can aid drivers to overcome safety-critical situations. The method models human intention as the latent states of a hidden Markov model and uses probabilistic dynamic time warping distributions to capture the temporal characteristics of the observation patterns of the drivers. The method is evaluated on a data set of 124 experiments from 75 drivers collected in a safety-critical semi-autonomous driving scenario. The results illustrate the efficacy of the framework by correctly anticipating the drivers' intentions about 3 seconds beforehand with over 90% accuracy.

Imitation learning by behavioral cloning is a prevalent method that has achieved some success in vision-based autonomous driving. The basic idea behind behavioral cloning is to have the neural network learn from observing a human expert's behavior. Typically, a convolutional neural network learns to predict the steering commands from raw driver-view images by mimicking the behaviors of human drivers. However, there are other cues, such as gaze behavior, available from human drivers that have yet to be exploited. Previous researches have shown that novice human learners can benefit from observing experts' gaze patterns. We present here that deep neural networks can also profit from this. We propose a method, gaze-modulated dropout, for integrating this gaze information into a deep driving network implicitly rather than as an additional input. Our experimental results demonstrate that gaze-modulated dropout enhances the generalization capability of the network to unseen scenes. Prediction error in steering commands is reduced by 23.5% compared to uniform dropout. Running closed loop in the simulator, the gaze-modulated dropout net increased the average distance travelled between infractions by 58.5%. Consistent with these results, the gazemodulated dropout net shows lower model uncertainty.

In this paper, we have developed a Gaussian Mixture Model (GMM) based algorithm with dynamic patch estimation for real-time detection and tracking of a known object. This research work detects the object of interest, estimates its 3-D position using Extended Kalman Filter (EKF) and generates the control output to the quad-rotor to track the target. The proposed algorithm is capable of tracking the object with a high Frame Per Second (FPS). Rigorous experiments are carried out to demonstrate the efficacy of the proposed approach in outdoor environment.

Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer from overfitting during the training process and are less robust under unforeseen conditions (which makes them especially vulnerable to adversarial scenarios). In this work, we propose Generative Robust Inference and Perception (GRIP) as a two-stage object detection and pose estimation system that aims to combine the relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of sample-based generative inference is able to recover from false object detections by CNNs, and produce robust estimations in adversarial conditions. We demonstrate the efficacy of GRIP robustness through comparison with state-of-the-art learning-based pose estimators and pick-and-place manipulation in dark and cluttered environments.

Grasping is a fundamental robotic task needed for the deployment of household robots or furthering warehouse automation. However, few approaches are able to perform grasp detection in real time (frame rate). To this effect, we present Grasp Quality Spatial Transformer Network (GQ-STN), a one-shot grasp detection network. Being based on the Spatial Transformer Network (STN), it produces not only a grasp configuration, but also directly outputs a depth image centered at this configuration. By connecting our architecture to an externally-trained grasp robustness evaluation network, we can train efficiently to satisfy a robustness metric via the backpropagation of the gradient emanating from the evaluation network. This removes the difficulty of training detection networks on sparsely annotated databases, a common issue in grasping. We further propose to use this robustness classifier to compare approaches, being more reliable than the traditional rectangle metric. Our GQ-STN is able to detect robust grasps on the depth images of the Dex-Net 2.0 dataset with 92.4 % accuracy in a single pass of the network. We finally demonstrate in a physical benchmark that our method can propose robust grasps more often than previous sampling-based methods, while being more than 60 times faster.

We propose a fast and accurate 3D reconstruction system that takes a sequence of RGB-D frames and produces a globally consistent camera trajectory and a dense 3D geometry. We redesign core modules of a state-of-the-art offline reconstruction pipeline to maximally exploit the power of GPU. We introduce GPU accelerated core modules that include RGBD odometry, geometric feature extraction and matching, point cloud registration, volumetric integration, and mesh extraction. Therefore, while being able to reproduce the results of the high-fidelity offline reconstruction system, our system runs more than 10 times faster on average. Nearly 10Hz can be achieved in medium size indoor scenes, making our offline system even comparable to online Simultaneous Localization and Mapping (SLAM) systems in terms of the speed. Experimental results show that our system produces more accurate results than several state-of-the-art online systems. The system is open source at https://github.com/theNded/Open3D.

In this paper, we describe a method for global localization in a previously unvisited environment using only a schematic floor plan as a prior map. The floor plan need not be a precision map-it can be the sort of image found in buildings to guide people or aid evacuation. The core idea is to identify features that are stable across both a drawn floor plan and robot point-of-view LIDAR data, for example wall intersections, which appear as corners from overhead and as vertical lines from the ground. We introduce a factor graph-based global localization method that uses these features as landmarks. The detections of such descriptorless features are noisy and often ambiguous. We therefore propose robust data association based on a pairwise measurement consistency check and max-mixtures error model. We evaluate the resulting system in a real-world indoor environment, demonstrating performance comparable to a baseline system that uses a conventional LIDAR-based prior map.

With the underwater human activity actively developing and the rise of the autonomous unmanned underwater vehicles (AUV) industry, researchers and engineers face the task of maintenance and repair of communications. For this purpose AUVs can be used, with e.g. the task of inspecting pipelines. The article describes the processing algorithm for the signal from passive ferromagnetic (fluxgate) sensors mounted on an AUV carrier used to search for metal-containing objects at the sea bottom. A scheme for such a measurement is proposed - the installation of two sensors at the opposite ends of the carrier. This allows to measure the gradient of magnetic field between the sensors. The characteristic form of such a signal and the dependence of the signal on the motion parameters of the vehicle and external factors are determined. To eliminate false positives, filters are used based on the readings of the position, speed and orientation sensors of the navigation system. Using data on the motion parameters of the device allows to generate a reference signal, which is used to validate the detection of an object using the cross-correlation method. The use of data on orientation angles makes it possible to compensate for the influence of the orientation of the device in the Earths magnetic field.

This paper addresses a vehicle localization method that fuses aerial maps and lidar data in urban canyon environments where global positioning system (GPS) signals are inaccurate. The boundaries of buildings are extracted from the aerial map and they are matched to point cloud data provided by the lidar. However, most aerial maps contain perspective projection distortions which can be significant in urban canyons with tall buildings. In this study, a new method to correct such projection distortion is proposed and it is applied to precise localization by fusing the corrected map and lidar data. In order to achieve this, the semantic segmentation of an aerial image is performed using a convolutional neural network, and the mutual information between the lidar measurements and the building boundaries is obtained to measure their similarity. A particle filter framework is employed to localize the vehicle and match the map using the mutual information as the weight of a particle. An experimental dataset is then used to validate the feasibility of the proposed method.

In this work, we propose a novel method termed Frustum ConvNet (F-ConvNet) for amodal 3D object detection from point clouds. Given 2D region proposals in an RGB image, our method first generates a sequence of frustums for each region proposal, and uses the obtained frustums to group local points. F-ConvNet aggregates point-wise features as frustum-level feature vectors, and arrays these feature vectors as a feature map for use of its subsequent component of fully convolutional network (FCN), which spatially fuses frustum-level features and supports an end-to-end and continuous estimation of oriented boxes in the 3D space. We also propose component variants of F-ConvNet, including an FCN variant that extracts multi-resolution frustum features, and a refined use of F-ConvNet over a reduced 3D space. Careful ablation studies verify the efficacy of these component variants. F-ConvNet assumes no prior knowledge of the working 3D environment and is thus dataset-agnostic. We present experiments on both the indoor SUN-RGBD and outdoor KITTI datasets. F-ConvNet outperforms all existing methods on SUN-RGBD, and at the time of submission it outperforms all published works on the KITTI benchmark. Code has been made available at: https://github.com/zhixinwang/frustum-convnet.

We introduce TopoNets, end-to-end probabilistic deep networks for modeling semantic maps with structure reflecting the topology of large-scale environments. TopoNets build a unified deep network spanning multiple levels of abstraction and spatial scales, from pixels representing geometry of local places to high-level descriptions of semantics of buildings. To this end, TopoNets leverage complex spatial relations expressed in terms of arbitrary, dynamic graphs. We demonstrate how TopoNets can be used to perform end-to-end semantic mapping from partial sensory observations and noisy topological relations discovered by a robot exploring large-scale office spaces. Thanks to their probabilistic nature and generative properties, TopoNets extend the problem of semantic mapping beyond classification. We show that TopoNets successfully perform uncertain reasoning about yet unexplored space and detect novel and incongruent environment configurations unknown to the robot. Our implementation of TopoNets achieves real-time, tractable and exact inference, which makes these new deep models a promising, practical solution to mobile robot spatial understanding at scale.

In many applications, maintaining a consistent map of the environment is key to enabling robotic platforms to perform higher-level decision making. Detection of already visited locations is one of the primary ways in which map consistency is maintained, especially in situations where external positioning systems are unavailable or unreliable. Mapping in 2D is an important field in robotics, largely due to the fact that man-made environments such as warehouses and homes, where robots are expected to play an increasing role, can often be approximated as planar. Place recognition in this context remains challenging: 2D lidar scans contain scant information with which to characterize, and therefore recognize, a location. This paper introduces a novel approach aimed at addressing this problem. At its core, the system relies on the use of the distance function for representation of geometry. This representation allows extraction of features which describe the geometry of both surfaces and free-space in the environment. We propose a feature for this purpose. Through evaluations on public datasets, we demonstrate the utility of free-space in the description of places, and show an increase in localization performance over a state-of-the-art descriptor extracted from surface geometry.

This paper proposes a new object handling device for a robot, in which a conventional vacuum suction cup is provided with force sensing ability by coating a piezoelectric thin film polymer, i.e., PolyVinylidene DiFluoride (PVDF), on its surface. It can detect the applied force by piezoelectric effect of coated thin film. The feature of our study is that the suction cup can work not only as a grasping tool but also as a force sensor. PVDF thin film was prepared by dip-coating method which can coat thin film directly on a three dimensional (3D) surface. Although the film with sufficient piezoelectric d 33 constant, e.g., 34 pC/N, was achieved after polarization process in case of flat surface, it was not achieved, however, on 3D bellows surface. To address this problem, as an alternative method, the feasibility of lamination of already polarized film was investigated.