ICRA 2018 Papers

We present a self-supervised approach to ignoring “distractors” in camera images for the purposes of robustly estimating vehicle motion in cluttered urban environments. We leverage offline multi-session mapping approaches to automatically generate a per-pixel ephemerality mask and depth map for each input image, which we use to train a deep convolutional network. At run-time we use the predicted ephemerality and depth as an input to a monocular visual odometry (VO) pipeline, using either sparse features or dense photometric matching. Our approach yields metric-scale VO using only a single camera and can recover the correct egomotion even when 90% of the image is obscured by dynamic, independently moving objects. We evaluate our robust VO methods on more than 400km of driving from the Oxford RobotCar Dataset and demonstrate reduced odometry drift and significantly improved egomotion estimation in the presence of large moving vehicles in urban traffic.

Because of their recent introduction, self-driving cars and advanced driver assistance system (ADAS) equipped vehicles have had little opportunity to learn, the dangerous traffic (including near-miss incident) scenarios that provide normal drivers with strong motivation to drive safely. Accordingly, as a means of providing learning depth, this paper presents a novel traffic database that contains information on a large number of traffic near-miss incidents that were obtained by mounting driving recorders in more than 100 taxis over the course of a decade. The study makes the following two main contributions: (i) In order to assist automated systems in detecting near-miss incidents based on database instances, we created a large-scale traffic near-miss incident database (NIDB) that consists of video clip of dangerous events captured by monocular driving recorders. (ii) To illustrate the applicability of NIDB traffic near-miss incidents, we provide two primary database-related improvements: parameter fine-tuning using various near-miss scenes from NIDB, and foreground/background separation into motion representation. Then, using our new database in conjunction with a monocular driving recorder, we developed a near-miss recognition method that provides automated systems with a performance level that is comparable to a human-level understanding of near-miss incidents (64.5% vs. 68.4% at near-miss recognition, 61.3% vs. 78.7% at near-miss detection).

When a human drives a car along a road for the first time, they later recognize where they are on the return journey typically without needing to look in their rear view mirror or turn around to look back, despite significant viewpoint and appearance change. Such navigation capabilities are typically attributed to our semantic visual understanding of the environment [1] beyond geometry to recognizing the types of places we are passing through such as “passing a shop on the left” or “moving through a forested area”. Humans are in effect using place categorization [2] to perform specific place recognition even when the viewpoint is 180 degrees reversed. Recent advances in deep neural networks have enabled high performance semantic understanding of visual places and scenes, opening up the possibility of emulating what humans do. In this work, we develop a novel methodology for using the semantics-aware higher-order layers of deep neural networks for recognizing specific places from within a reference database. To further improve the robustness to appearance change, we develop a descriptor normalization scheme that builds on the success of normalization schemes for pure appearance-based techniques such as SeqSLAM [3]. Using two different datasets - one road-based, one pedestrian-based, we evaluate the performance of the system in performing place recognition on reverse traversals of a route with a limited field of view camera and no turn-back-and-Iook behaviours, and compare to existing state-of-the-art techniques and vanilla off-the-shelf features. The results demonstrate significant improvements over the existing state of the art, especially for extreme perceptual challenges that involve both great viewpoint change and environmental appearance change. We also provide experimental analyses of the contributions of the various system components: the use of spatio-temporal sequences, place categorization and place-centric characteristics as opposed to object-centric seman... Show More

Actuator modules of humanoid robots have relatively higher joint elasticity than those of industrial robots. Such joint elasticity could lead to negative effects on both the tracking performance and stability for walking. Especially, unstable contact between the foot and ground caused by joint elasticity is a critical problem, as it decreases the stability of position-controlled humanoid robots. To address this problem, this paper introduces a novel control scheme for position-controlled humanoid robots by which we can obtain not only enhance compliance capability for unknown contact but also suppress the vibration caused by joint elasticity. To estimate the disturbance caused by external forces and modeling errors between the actual system and nominal system, a disturbance observer based estimator is designed at each joint. Furthermore, a linear feedback controller for the flexible joint model and a gravity compensator is considered to reduce vibration and deflection due to the joint elasticity. The proposed control scheme was implemented on our humanoid robot, DYROS-JET, and its performance was demonstrated by improved stability during dynamic walking and stepping on objects.

We study two multi-robot assignment problems for multi-target tracking. We consider distributed approaches in order to deal with limited sensing and communication ranges. We seek to simultaneously assign trajectories and targets to the robots. Our focus is on local algorithms that achieve performance close to the optimal algorithms with limited communication. We show how to use a local algorithm that guarantees a bounded approximate solution within O(hlog1/ε) communication rounds. We compare with a greedy approach that achieves a 2-approximation in as many rounds as the number of robots. Simulation results show that the local algorithm is an effective solution to the assignment problem.

We present a complete design and implementation of a system that consists of multiple quadrotors playing capture the flag game. Our main contributions in this work include a custom built quadrotor platform, an efficient implementation of a distributed trajectory planning algorithm, and a WiFi based communication infrastructure. In our design, we equip the quadrotors with autopilot modules for low level control. Moreover, we install low power computing modules for implementing the distributed trajectory planning algorithm online. Furthermore, we develop a communication infrastructure to enable coordination among the quadrotors, which is required for computing a suboptimal control action in real time. The interactions among all the hardware and software components are managed at a higher level by Robot Operating System (ROS). To test the performance of the system, we select a motivating setup of capture the flag game, which is an adversarial game played between two teams of agents called attack and defense. The system is initially simulated in the Gazebo robot simulator with software in the loop. Finally, the complete system is tested and the flight testing results are presented.

Many recent works have proposed algorithms for information gathering that benefit from multi-robot cooperation. However, most algorithms either employ discretization of the state and action spaces, which makes them computationally intractable for robotic systems with complex dynamics; or cannot deal with inter-robot restrictions like e.g. communication constraints. This paper presents an approach for multi-robot information gathering that tackles the two aforementioned issues. To this end we propose an algorithm that combines in an innovative manner Gaussian processes (GPs) to model the physical process of interest, RRT planners to plan paths in a continuous domain, and a distributed decision-making algorithm to achieve multi-robot cooperation. Specifically, we employ the Max-sum algorithm for distributed multi-robot cooperation by defining an information-theoretic utility function together with a path clustering approach. This function maximizes information gathering, subject to inter-robot communication constraints. We validate the proposed approach in simulations, and in a field experiment where three quadcopters explore a simulated wind field. Results demonstrate the effectiveness of the approach.

Decentralized control architectures, such as those conventionally defined by central pattern generators, independently coordinate spatially distributed portions of articulated bodies to achieve system-level objectives. State of the art distributed algorithms for reinforcement learning employ a different but conceptually related idea; independent agents simultaneously coordinating their own behaviors in parallel environments while asynchronously updating the policy of a system-or, rather, meta-level agent. This work, to the best of the authors' knowledge, is the first to explicitly explore the potential relationship between the underlying concepts in homogeneous decentralized control for articulated locomotion and distributed learning. We present an approach that leverages the structure of the asynchronous advantage actor-critic (A3C) algorithm to provide a natural framework for learning decentralized control policies on a single platform. Our primary contribution shows an individual agent in the A3C algorithm can be defined by an independently controlled portion of the robot's body, thus enabling distributed learning on a single platform for efficient hardware implementation. To this end, we show how the system is trained offline using hardware experiments implementing an autonomous decentralized compliant control framework. Our experimental results show that the trained agent outperforms the compliant control baseline by more than 40% in terms of steady progression through a series of randomized, highly cluttered evaluation environments.

In this paper, we develop a distributed intermittent communication framework for teams of mobile robots that are responsible for accomplishing time-critical dynamic tasks and sharing the collected information with all other robots and possibly also with a user. Specifically, we consider situations where the robot communication capabilities are not sufficient to maintain reliable and connected networks while the robots move to accomplish their tasks. In this case, intermittent communication protocols are necessary that allow the robots to temporarily disconnect from the network in order to accomplish their tasks free of communication constraints. We assume that the robots can only communicate with each other when they meet at common locations in space. Our proposed distributed control framework determines offline schedules of communication events and integrates them online with task planning. The resulting paths ensure task accomplishment and exchange of information among robots infinitely often at locations that minimize a user-specified metric. Simulation results corroborate the proposed distributed control framework.

We present a fully realised system illustrating decentralised coordination on Micro Aerial Vehicles (MAV) or pocket drones, based on distance information. This entails the development of an ultra light hardware solution to determine the distances between the drones and also the development of a model to learn good control policies. The model we present is a combination of a recurrent neural network and a Deep Q-Learning Network (DQN). The recurrent network provides bearing information to the DQN. The DQN itself is responsible for choosing movement actions to avoid collisions and to reach a desired position. Overall we are able provide a complete system which is capable of letting multiple drones navigate in a confined space only based on UWB-distance information and velocity input. We tackle the problem of neural networks and real world sensor noise, by combining the network with a particle filter and show that the combination outperforms the traditional particle filter in terms of converge speed and robustness. A video is available at: https://youtu.be/yj6QqhOzpok.

Accurate haptic feedback is a critical challenge for surgical robots, especially for flexible endoscopic surgical robots whose transmission systems are Tendon-Sheath Mechanisms (TSMs) with highly nonlinear friction profiles and force hysteresis. For distal end haptic sensing of TSMs, this paper, for the first time, proposes to measure the compression force on the sheath at the distal end so that the tension force on the tendon, which equals the compression force on the sheath, can be obtained. A new force sensor, i.e., a nitinol tube attached with an optical Fiber Bragg Grating (FBG) fiber, is proposed to measure the compression force on the sheath. This sensor, with similar diameter and configuration (hollow) as the sheath, can be compactly integrated with TSMs and surgical end-effectors. In this paper, mechanics analysis and verification tests are presented to reveal the relationship between the tension force on the tendon and the compression force on the sheath. The proposed force sensor was calibrated in tests with a sensitivity of 24.28 pm/N and integrated with a tendon-sheath driven grasper to demonstrate the effectiveness of the proposed approach and sensor. The proposed approach and sensor can also be applied for a variety of TSMs-driven systems, such as robotic fingers/hands, wearable devices, and rehabilitation devices.

Due to their inherent crystalline properties piezoelectric actuators have a limited deformation. This intrinsic drawback deprives to exploit the potential of these actuators such as, high bandwidth and high resolution in applications that require large displacement range. To overcome this limitation, classical as well as systematic approaches were proposed to design amplification mechanisms. The classical approach leads to empirical mechanisms which are not trivial and needs much experience and intuition. In contrast, systematic approach uses topology optimization method which permits to automatically derive optimal designs that can satisfy specified performances and imposed constraints simultaneously, this with a reasonable time and cost. This paper proposes the design of a mechanism devoted to amplify the displacement of a piezoelectric actuators (PEA). Based on the SIMP topology optimization method, the approach permits to derive a design with a displacement amplification ratio of 4.5, which is higher than with the existing method of Rhombus mechanism. Both finite element (FE) simulation and experimental results confirm and demonstrate the efficiency of the approach.

Tensegrity robots, which combine both rigid and soft elements, provide exciting new locomotion capabilities but introduce significant control challenges given their high-dimensionality and non-linear nature. This work first defines an effective parameterization of a spherical tensegrity for generating rhythmic gaits based on Central Pattern Generators (cp G). This allows the definition of periodic and rhythmic control signals, while exposing only five gait parameters. Then, this work proposes a framework for optimizing such gaits by exploring the parameter space through Bayesian Optimization on an underlying Gaussian Process regression model. The objective is to provide gaits that allow the platform to move along different directions with high velocity. Additionally, kNN binary classifiers are trained to estimate whether a parameter sample will result in an effective gait. The classification biases the sampling toward subspaces likely to yield effective gaits. An asynchronous communication layer is defined between the optimization and classification processes. The proposed gait discovery process is shown to efficiently optimize the parameters of gaits defined given the novel CPG architecture and outperforms less holistic approaches and Monte Carlo sampling.

The seeds of Maple trees (Samara) use autorotation as a unique mechanism to disperse their seeds. By exploiting gyroscopic stability of a spinning wing, the Samara is able to cover large horizontal distance despite having no form of propulsion. We applied and adapted this natural ability in our novel concept, the Samara Autorotating Wings (SAW), and extended its stability and direction controllability by generalizing the mechanism to incorporate designs with more than 1 wing. By conceiving cyclic control, the translational motion of autorotation is regulated. A nonlinear model of SAW with nn wings is derived and control schemes developed to control the translational position during autorotation. Numerical simulations were performed to investigate the performance of the multi-wing SAW prototypes to track a conical spiral autorotation trajectory. Direct experiments were conducted in a vertical wind-tunnel through a special ball joint that allows z-axis translation and all three rotational degrees of freedom. Finally, free-fall drop tests are used to verify the directional controllability and performance of SAW.

This paper describes a framework for direct visual simultaneous localization and mapping (SLAM) combining a monocular camera with sparse depth information from Light Detection and Ranging (LiDAR). To ensure realtime performance while maintaining high accuracy in motion estimation, we present (i) a sliding window-based tracking method, (ii) strict pose marginalization for accurate pose-graph SLAM and (iii) depth-integrated frame matching for large-scale mapping. Unlike conventional feature-based visual and LiDAR mapping, the proposed approach is direct, eliminating the visual feature in the objective function. We evaluated results using our portable camera-LiDAR system as well as KITTI odometry benchmark datasets. The experimental results prove that the characteristics of two complementary sensors are very effective in improving real-time performance and accuracy. Via validation, we achieved low drift error of 0.98 % in the KITTI benchmark including various environments such as a highway and residential areas.

We present VI-DSO, a novel approach for visual-inertial odometry, which jointly estimates camera poses and sparse scene geometry by minimizing photometric and IMU measurement errors in a combined energy functional. The visual part of the system performs a bundle-adjustment like optimization on a sparse set of points, but unlike key-point based systems it directly minimizes a photometric error. This makes it possible for the system to track not only corners, but any pixels with large enough intensity gradients. IMU information is accumulated between several frames using measurement preintegration and is inserted into the optimization as an additional constraint between keyframes. We explicitly include scale and gravity direction into our model and jointly optimize them together with other variables such as poses. As the scale is often not immediately observable using IMU data this allows us to initialize our visual-inertial system with an arbitrary scale instead of having to delay the initialization until everything is observable. We perform partial marginalization of old variables so that updates can be computed in a reasonable time. In order to keep the system consistent we propose a novel strategy which we call “dynamic marginalization”. This technique allows us to use partial marginalization even in cases where the initial scale estimate is far from the optimum. We evaluate our method on the challenging EuRoC dataset, showing that VI-DSO outperforms the state of the art.

Modern visual odometry algorithms utilize sparse point-based features for tracking due to their low computational cost. Current state-of-the-art methods are split between indirect methods that process features extracted from the image, and indirect methods that deal directly on pixel intensities. In recent years, line-based features have been used in SLAM and have shown an increase in performance albeit with an increase in computational cost. In this paper, we propose an extension to a point-based direct monocular visual odometry method. Here we that uses lines to guide keypoint selection rather than acting as features. Points on a line are treated as stronger keypoints than those in other parts of the image, steering point-selection away from less distinctive points and thereby increasing efficiency. By combining intensity and geometry information from a set of points on a line, accuracy may also be increased.

This paper proposes a new method for detecting a road from a stereo pair of images. First, the horizon is accurately estimated by a robust, weighted-sampling RANSAC-like method in the improved v-disparity map. The vanishing point of the road region is located using both the horizon information and road flatness constraints. Then it is used as the source node of a weighted graph formed by the pixels of the left stereo-image and their adjacency relationships. The weight of each edge measures the inconsistency of adjacent pixels, and is computed using both the gray-scale and disparity information. Detecting road borders is thus reduced to finding two shortest paths from the source node to the bottom row of the image by the Dijkstra algorithm. The proposed method has been tested on 2621 image pairs of different road scenes from the KITTI dataset. Our experiments demonstrate that this training free approach detects horizon, vanishing point, and road region accurately and robustly, and compares favorably with the state of the art on the KITTI benchmark.

Micro-robots have a great application prospect in the biomedical field due to the feature of small size. To solve the issues of energy supply and bio-compatibility of micro-robots, bio-syncretic micro-robots composed of biological materials and electromechanical systems have been studied widely. The skeletal muscle is a potential material to develop bio-actuator for the bio-syncretic robots on account of the great contraction force and the controllability. However, the low differentiation quality of C2C12s and the control of the bio-syncretic robots are the two of the main challenges for the development of the bio-syncretic robots based on the skeleton muscle. In this paper, an approach based on circularly distributed multiple electrodes (CDMEs) was proposed to improve the differentiation of C2C12 myoblast cells and characterize the electro-responsive beating behavior of myotubes for the development of bio-syncretic robots. Three groups of C2C12 blasts were used to fulfill the differentiation experiments without electrical stimulation and with electrical stimulation using parallel electrodes and CDMEs respectively, for evaluating the effect of CDMEs on C2C12 differentiation. It was demonstrated that electrical field through CDMEs can improve the differentiation quality of C2C12 blasts into myotubes in terms of intensity, length, and widths. Then, the effect of electrical stimulation on the beating behaviors of myotubes was also investigated with CDMEs, and it was shown that the beating amplitudes of myotubes were significantly affected by the frequencies, amplitude and direction of electrical stimulation with respect to the myotubes, which is fundamental for the control of the micro-robot based on skeletal muscle cells. The proposed approach is useful for not only the development of the bio-syncretic robots, but also the study of muscle tissue engineering.