IROS 2017 论文列表

Dynamical Systems (DS) are often used to represent motion, with the advantage of being easy to learn from demonstrations. We present a method to modulate DS depending on an external signal, extending our previous work on Locally Modulated DS (LMDS [1]). We present two applications of our system, which would not have been possible to achieve without taking external sensing into account in the DS motion formulation. The first application is a task of localization and grasping of objects, using our previous work on compliant tactile exploration. We successfully localize and grasp objects whose position is unknown, using touch in a simulated environment. In the second application, we teach a robot how to react to collisions in order to navigate between obstacles while reaching.

Deep networks thrive when trained on large scale data collections. This has given ImageNet a central role in the development of deep architectures for visual object classification. However, ImageNet was created during a specific period in time, and as such it is prone to aging, as well as dataset bias issues. Moving beyond fixed training datasets will lead to more robust visual systems, especially when deployed on robots in new environments which must train on the objects they encounter there. To make this possible, it is important to break free from the need for manual annotators. Recent work has begun to investigate how to use the massive amount of images available on the Web in place of manual image annotations. We contribute to this research thread with two findings: (1) a study correlating a given level of noisily labels to the expected drop in accuracy, for two deep architectures, on two different types of noise, that clearly identifies GoogLeNet as a suitable architecture for learning from Web data; (2) a recipe for the creation of Web datasets with minimal noise and maximum visual variability, based on a visual and natural language processing concept expansion strategy. By combining these two results, we obtain a method for learning powerful deep object models automatically from the Web. We confirm the effectiveness of our approach through object categorization experiments using our Web-derived version of ImageNet on a popular robot vision benchmark database, and on a lifelong object discovery task on a mobile robot.

We propose a new probabilistic framework that allows mobile robots to autonomously learn deep, generative models of their environments that span multiple levels of abstraction. Unlike traditional approaches that combine engineered models for low-level features, geometry, and semantics, our approach leverages recent advances in Sum-Product Networks (SPNs) and deep learning to learn a single, universal model of the robot's spatial environment. Our model is fully probabilistic and generative, and represents a joint distribution over spatial information ranging from low-level geometry to semantic interpretations. Once learned, it is capable of solving a wide range of tasks: from semantic classification of places, uncertainty estimation, and novelty detection, to generation of place appearances based on semantic information and prediction of missing data in partial observations. Experiments on laser-range data from a mobile robot show that the proposed universal model obtains performance superior to state-of-the-art models fine-tuned to one specific task, such as Generative Adversarial Networks (GANs) or SVMs.

Being able to fall safely is a necessary motor skill for humanoids performing highly dynamic tasks, such as running and jumping. We propose a new method to learn a policy that minimizes the maximal impulse during the fall. The optimization solves for both a discrete contact planning problem and a continuous optimal control problem. Once trained, the policy can compute the optimal next contacting body part (e.g. left foot, right foot, or hands), contact location and timing, and the required joint actuation. We represent the policy as a mixture of actor-critic neural network, which consists of n control policies and the corresponding value functions. Each pair of actor-critic is associated with one of the n possible contacting body parts. During execution, the policy corresponding to the highest value function will be executed while the associated body part will be the next contact with the ground. With this mixture of actor-critic architecture, the discrete contact sequence planning is solved through the selection of the best critics while the continuous control problem is solved by the optimization of actors. We show that our policy can achieve comparable, sometimes even higher, rewards than a recursive search of the action space using dynamic programming, while enjoying 50 to 400 times of speed gain during online execution.

Interactive agents are becoming increasingly common in many application domains, such as education, healthcare and personal assistance. The success of such embodied agents relies on their ability to have sustained engagement with their human users. Such engagement requires agents to be socially intelligent, equipped with the ability to understand and reciprocate both verbal and non-verbal cues. While there has been tremendous progress in verbal communication, mostly driven by the success of speech recognition and question-answering, teaching agents to appropriately react to facial expressions has received less attention. In this paper, we focus on non-verbal facial cues for face-to-face communication between a user and an embodied agent. We propose a method that automatically learns to update the agent's facial expressions based on the user's expressions. We adopt a learning scheme and train a deep neural network on hundreds of videos, containing pairs of people engaging in a conversation, and without external human supervision. Our experimental results show the efficacy of our model in sustained long-term prediction of the agent's facial landmarks. We present comparative results showing that our model significantly outperforms baseline approaches and provide insightful human studies to better understand our model's qualitative performance. We release our dataset to further encourage research in this field.

Constrained key-frame based local bundle adjustment is at the core of many recent systems that address the problem of large-scale, georeferenced SLAM based on a monocular camera and on data from inexpensive sensors and/or databases. The majority of these methods, however, impose constraints that result from proprioceptive sensors (e.g. IMUs, GPS, Odometry) while ignoring the possibility of explicitly constraining the structure (e.g. point cloud) resulting from the reconstruction process. Moreover, research on on-line interactions between SLAM and deep learning methods remains scarce, and as a result, few SLAM systems take advantage of deep architectures. We explore both these areas in this work: we use a fast deep neural network to infer semantic and structural information about the environment, and using a Bayesian framework, inject the results into a bundle adjustment process that constrains the 3d point cloud to texture-less 3d building models.

Nowadays, self-driving cars are being applied to more complex urban scenarios including intersections, merging ramps or lane changes. It is, therefore, important for self-driving cars to behave socially with human-driven cars. In this paper, we focus on generating the lane change behavior for self-driving cars: perform a safe and effective lane change behavior once a lane-change command is received. Our method bridges the gap between higher-level behavior commands and the trajectory planner. There are two challenges in the task: 1) Analyzing the surrounding vehicles' mutual effects from their trajectories. 2) Estimating the proper lane change start point and end point according to the analysis of surrounding vehicles. We propose a learning-based approach to understand surrounding traffic and make decisions for a safe lane change. Our contributions and advantages of the approach are: 1 Considers the behavior generator as a continuous function in Reproducing Kernel Hilbert Space (RKHS) which contains a family of behavior generators; 2 Constructs the behavior generator function in RKHS by non-parametric regressions on training data; 3 Takes past trajectories of all related surrounding cars as input to capture mutual interactions and output continuous values to represent behaviors. Experimental results show that the proposed approach is able to generate feasible and human-like lane-change behavior (represented by start and end points) in multi-car environments. The experiments also verified that our suggested kernel outperforms the ones which were used in a previous method.

A landmark based heuristic is investigated for reducing query phase run-time of the probabilistic roadmap (PRM) motion planning method. The heuristic is generated by storing minimum spanning trees from a small number of vertices within the PRM graph and using these trees to approximate the cost of a shortest path between any two vertices of the graph. The intermediate step of preprocessing the graph increases the time and memory requirements of the classical motion planning technique in exchange for speeding up individual queries making the method advantageous in multi-query applications. This paper investigates these trade-offs on PRM graphs constructed in randomized environments as well as a practical manipulator simulation. We conclude that the method is preferable to Dijkstra's algorithm or the A* algorithm with conventional heuristics in multi-query applications.

We propose a new method for the propagation of semantic labels in RGB-D video of indoor scenes given a set of ground truth keyframes. Manual labeling of all pixels in every frame of a video sequence is labor intensive and costly, yet required for training and testing of semantic segmentation methods. The availability of video enables propagation of labels between the frames for obtaining a large amounts of annotated pixels. While previous methods commonly used optical flow motion cues for label propagation, we present a novel approach using the camera poses and 3D point clouds for propagating the labels in superpixels computed on the unannotated frames of the sequence. The propagation task is formulated as an energy minimization problem in a Conditional Random Field (CRF). We performed experiments on 8 video sequences from SUN3D dataset [1] and showed superior performance to an optical flow based label propagation approach. Furthermore, we demonstrated that the propagated labels can be used to learn better models using data hungry deep convolutional neural network (DCNN) based approaches for the task of semantic segmentation. The approach demonstrates an increase in performance when the ground truth keyframes are combined with the propagated labels during training.

This paper presents a novel generic formulation of Timed-Elastic-Bands for efficient online motion planning of car-like robots. The planning problem is defined in terms of a finite-dimensional and sparse optimization problem subject to the robots kinodynamic constraints and obstacle avoidance. Control actions are implicitly included in the optimized trajectory. Reliable navigation in dynamic environments is accomplished by augmenting the inner optimization loop with state feedback. The predictive control scheme is real-time capable and responds to obstacles within the robot's perceptual field. Navigation in large and complex environments is achieved in a pure pursuit fashion by requesting intermediate goals from a global planner. Requirements on the initial global path are fairly mild, compliance with the robot kinematics is not required. A comparative analysis with Reeds and Shepp curves and investigation of prototypical car maneuvers illustrate the advantages of the approach.

Combining light-weight visual and inertial modalities for motion capturing has been popular in robotics researches. There exist scale ambiguity, inaccurate pose estimation with little or no baseline, incremental drifts over time in visual-inertial fusion. Thus, in this paper, we propose a robust motion capturing method based on the multi-joint kinematic chain using monocular visual-inertial sensors. Our method is able to recover monocular visual scale through the joint geometry constraint. Additionally, we take inertial pre-integration to assist visual outlier removal using Maximum A Posteriori method. Ultimately, the kinematic chain model is leveraged to constrain the associated multiple visual-inertial estimation drifts during long time tracking. In the experiments, we conduct multi-joint capturing on a robotic arm. The quality of motion reconstruction is evaluated by comparing the estimated results with the measurements from an optical motion tracking system OptiTrack.

Complementing images with inertial measurements has become one of the most popular approaches to achieve highly accurate and robust real-time camera pose tracking. In this paper, we present a keyframe-based approach to visual-inertial simultaneous localization and mapping (SLAM) for monocular and stereo cameras. Our visual-inertial SLAM system is based on a real-time capable visual-inertial odometry method that provides locally consistent trajectory and map estimates. We achieve global consistency in the estimate through online loop-closing and non-linear optimization. Furthermore, our system supports relocalization in a map that has been previously obtained and allows for continued SLAM operation. We evaluate our approach in terms of accuracy, relocalization capability and run-time efficiency on public indoor benchmark datasets and on newly recorded outdoor sequences. We demonstrate state-of-the-art performance of our system compared to a visual-inertial odometry method and baseline visual SLAM approaches in recovering the trajectory of the camera.

The ability of robots to estimate their location is crucial for a wide variety of autonomous operations. In settings where GPS is unavailable, measurements of transmissions from fixed beacons provide an effective means of estimating a robot's location as it navigates. The accuracy of such a beacon-based localization system depends both on how beacons are distributed in the environment, and how the robot's location is inferred based on noisy and potentially ambiguous measurements. We propose an approach for making these design decisions automatically and without expert supervision, by explicitly searching for the placement and inference strategies that, together, are optimal for a given environment. Since this search is computationally expensive, our approach encodes beacon placement as a differential neural layer that interfaces with a neural network for inference. This formulation allows us to employ standard techniques for training neural networks to carry out the joint optimization. We evaluate this approach on a variety of environments and settings, and find that it is able to discover designs that enable high localization accuracy.

Understanding the 3D structure of a scene is of vital importance, when it comes to developing fully autonomous robots. To this end, we present a novel deep learning based framework that estimates depth, surface normals and surface curvature by only using a single RGB image. To the best of our knowledge this is the first work to estimate surface curvature from colour using a machine learning approach. Additionally, we demonstrate that by tuning the network to infer well designed features, such as surface curvature, we can achieve improved performance at estimating depth and normals. This indicates that network guidance is still a useful aspect of designing and training a neural network. We run extensive experiments where the network is trained to infer different tasks while the model capacity is kept constant resulting in different feature maps based on the tasks at hand. We outperform the previous state-of-the-art benchmarks which jointly estimate depths and surface normals while predicting surface curvature in parallel.

Stereo vision is commonly used for local obstacle avoidance of autonomous mobile robots: stereo images are first processed to yield a dense 3D reconstruction of the observed scene, which is then used for navigation planning. Such an approach, which we term Sequential Perception and Planning (SPP), results in significant unnecessary computations as the navigation planner only needs to explore a small part of the scene to compute the shortest obstacle-free path. In this paper, we introduce an approach to Joint Perception and Planning (JPP) using stereo vision, which performs disparity checks on demand, only as necessary while searching on a planning graph. Furthermore, obstacle checks for navigation planning do not require full 3D reconstruction: we present in this paper how obstacle queries can be decomposed into a sequence of confident positive stereo matches and confident negative stereo matches, which are significantly faster to compute than the exact depth of points. The resulting complete JPP formulation is significantly faster than SPP, while still maintaining correctness of planning. We also show how the JPP works with different planners, including search-based and sampling-based planners. We present extensive experimental results from real robot data and simulation experiments, demonstrating that the JPP requires less than 10% of the disparity computations required by SPP.

The buildup of inaccuracies from frequent and imperfect data averaging can negatively impact system behavior. One potential culprit is improper orientation averaging, such as when combining data from multiple sensors, or reconciling preferences from multiple agents. In practice, the currently prevalent methodology of averaging 2D orientations is that of adding orientation vectors, which minimizes the Euclidean (or chord) distance among the vectors, instead of the geodesic (or arc) distance, resulting in inaccurate or even entirely incorrect averages. While an arc-minimizing alternative exists, it is only defined for angle averaging, posing an issue if orientations also possess a meaningful magnitude within the domain. In this work, we present an iterative weighted 2D orientation arc-based averaging algorithm that minimizes squared arc-lengths between points, incorporates orientation magnitudes as weights, and allows for multiple equally valid averages to be produced whenever applicable. We compare a vector sum approach and the weighted arc-based approach as applied to collaborative transport with obstacle avoidance, and showcase the behavioral advantages of the arc-based weighted averaging.

This paper presents an investigation of human comfort with a small Unmanned Aerial Vehicle (sUAV) through a study offering a comparison of comfort with a sUAV versus a ground vehicle. Current research on human comfort with sUAVs has been limited to a single previous study, which did not include free flight, and while ground vehicle distancing has been studied, it has never been directly compared to a sUAV. The novelty in the approach is the use of a motion capture room to achieve smooth trajectories and precise measurements, while conducting the first free flight study to compare human comfort after interaction with aerial versus ground vehicles (within subjects, N=16). These results will contribute to understanding of social, collaborative, and assistive robots, with implications for general human-robot interactions as they evolve to include aerial vehicles. Based on the reduced stress and distance (36.5cm or 1.2ft) for ground vehicles and increased stress and distance (65.5cm or 2.15ft) for sUAVs, it is recommended that studies be conducted to understand the implications of design features on comfort in interactions with sUAVs and how they differ from those with ground robots.

Subject of this paper is an industrial gantry robot with elastic behavior. To suppress residual vibrations command shaping filters are widely used. These filters, however, change the shape and duration of the originally commanded trajectory. Therefore, a command shaping filter is proposed which is based on the inverse model of the flexible robot dynamics. To that end, a planar dynamics model of the robot is derived which is suitable for control design. It captures the relevant dynamics of vibrations including the non-minimum phase characteristics. Based on the inverse dynamics, a feedforward filter for vibration suppression is developed. The tracking error due to filter delay is reduced compared to other command shaping techniques. The non-minimum phase properties of the flexible dynamics are avoided by using a virtual reference point for plant inversion. Experimental results show the effectiveness of the proposed Inverse Model Command Shaper compared to other types of filters tuned for vibration suppression.

In this paper, the well-known problem of codependence between inverse dynamics torque and contact force in bimanual object manipulation is addressed. The common contact constraint, namely rigid grasping, is exploited to decompose the set of dynamics equations into two orthogonally decoupled sets. Subsequently, the inverse dynamics control is formulated in a sub-manifold that is independent of the contact force, leading to analytically correct solutions that do not need to resort to common approximations for the aforementioned codependence problem. The contact force is also analytically computed and, therefore, can be optimally distributed using the torque redundancy. Relying on this prediction is most significant in situations where a force sensor at the end-effector is not present or is faulty. Even in the availability of sensory data, the predicted force may be used to correct typically noisy or delayed when filtered measurements, resulting in improved robustness. Simulation experiments on a planar bimanual manipulation model are presented.