IROS 2019 Papers

This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the convex hull property of relative Bernstein polynomial. Our approach employs a graph-based multi-agent pathfinding algorithm to generate an initial trajectory, which is used to construct a safe flight corridor (SFC) and RSFC. We express the trajectory as a piecewise Bernstein polynomial and formulate the trajectory planning problem into one quadratic programming problem using linear constraints from SFC and RSFC. The proposed method can compute collision-free trajectory for 16 agents within a second and for 64 agents less than a minute, and it is validated both through simulation and indoor flight test.

In this work, we propose a method to efficiently compute smooth, time-optimal trajectories for micro aerial vehicles (MAVs) evading a moving obstacle. Our approach first computes an n-dimensional trajectory from the start- to an arbitrary target state including position, velocity and acceleration. It respects input- and state-constraints and is thus dynamically feasible. The trajectory is then efficiently checked for collisions, exploiting the piecewise polynomial formulation. If collisions occur, viastates are inserted into the trajectory to circumvent the obstacle and still maintain time-optimality. These viastates are described by position, velocity, and acceleration. The evaluation shows that the computational demands of the proposed method are minimal such that obstacle avoidance can begin within few milliseconds. Optimality of generated trajectories, combined with the ability for frequent online re-planning from non-hover initial conditions, make the approach well suited for evasion of suddenly perceived obstacles during fast flight.

In this paper, we present a fast run-time monitoring framework for safety assurance during autonomous system operations in uncertain environments. Modern unmanned vehicles rely on periodic sensor measurements for motion planning and control. However, a vehicle may not always be able to obtain its state information due to various reasons such as sensor failures, signal occlusions, and communication problems. To guarantee the safety of a system during these circumstances under the presence of disturbance and noise, we propose a novel fast reachability analysis approach that leverages Gaussian process regression theory to predict future states of the system at run-time. We also propose a self/event-triggered monitoring and replanning approach which leverages our fast reachability scheme to recover the system when needed and replan its trajectory to guarantee safety constraints (i.e., the system will not collide with any obstacles). Our technique is validated both with simulations and experiments on unmanned aerial vehicles case studies in cluttered environments under the effect of unknown wind disturbance at run-time.

As the aging population grows at a rapid rate, there is an ever growing need for service robot platforms that can provide daily assistance at practical speed with reliable performance. In order to assist with daily tasks such as fetching a beverage, a service robot must be able to perceive its environment and generate corresponding motion trajectories. This becomes a challenging and computationally complex problem when the environment is unknown and thus the path planner must sample numerous trajectories that often are sub-optimal, extending the execution time. To address this issue, we propose a unique strategy of integrating a 3D object detection pipeline with a kinematically optimal manipulation planner to significantly increase speed performance at run-time. In addition, we develop a new robotic butler system for a wheeled humanoid that is capable of fetching requested objects at 24% of the speed a human needs to fulfill the same task. The proposed system was evaluated and demonstrated in a real-world environment setup as well as in public exhibition.

This paper presents a novel learning-based method for fast motion planning in high-dimensional spaces. A deep neural network is designed to predict the free configuration space rapidly given the environment point cloud. With a generated roadmap as an approximate view of the free C-space, LazyPRM is applied to find and check the path with A* search. Due to the application of LazyPRM, the presented method can preserve probabilistic completeness and asymptotic optimality. The new algorithm is tested on a 3-DOF robot arm and a 6-DOF UR3 robot to plan in randomly generated obstacle environments. Results indicate that compared to planners including PRM, RRT*, RRT-connect and the original LazyPRM, our method is of the lowest time consumption and relatively short path length, showing good performance on both planning speed and path quality.

A significant challenge in manipulation motion planning is to ensure agility in the face of unpredictable changes during task execution. This requires the identification and possible modification of suitable joint-space trajectories, since the joint velocities required to achieve a specific endeffector motion vary with manipulator configuration. For a given manipulator configuration, the joint space-to-task space velocity mapping is characterized by a quantity known as the manipulability index. In contrast to previous control-based approaches, we examine the maximization of manipulability during planning as a way of achieving adaptable and safe joint space-to-task space motion mappings in various scenarios. By representing the manipulator trajectory as a continuous-time Gaussian process (GP), we are able to leverage recent advances in trajectory optimization to maximize the manipulability index during trajectory generation. Moreover, the sparsity of our chosen representation reduces the typically large computational cost associated with maximizing manipulability when additional constraints exist. Results from simulation studies and experiments with a real manipulator demonstrate increases in manipulability, while maintaining smooth trajectories with more dexterous (and therefore more agile) arm configurations.

We present a system for fast and robust handovers with a robot character, together with a user study investigating the effect of robot speed and reaction time on perceived interaction quality. The system can match and exceed human speeds and confirms that users prefer human-level timing. The system has the appearance of a robot character, with a bear-like head and a soft anthropomorphic hand and uses Bézier curves to achieve smooth minimum-jerk motions. Fast timing is enabled by low latency motion capture and real-time trajectory generation: the robot initially moves towards an expected handover location and the trajectory is updated on-the-fly to converge smoothly to the actual handover location. A hybrid automaton provides robustness to failure and unexpected human actions. In a 3x3 user study, we vary the speed of the robot and add variable sensorimotor delays. We evaluate the social perception of the robot using the Robot Social Attribute Scale (RoSAS). Inclusion of a small delay, mimicking the delay of the human sensorimotor system, leads to an improvement in perceived qualities over both no delay and long delay conditions. Specifically, with no delay the robot is perceived as more discomforting, and with a long delay it is perceived as less warm.

In this paper, we report on a parallel free-viewpoint video synthesis algorithm that can efficiently reconstruct a high-quality 3D scene representation of sports scenes. The proposed method focuses on a scene that is captured by multiple synchronized cameras featuring wide-baselines. The following strategies are introduced to accelerate the production of a free-viewpoint video taking the improvement of visual quality into account: (1) a sparse point cloud is reconstructed using a volumetric visual hull approach, and an exact 3D ROI is found for each object using an efficient connected components labeling algorithm. Next, the reconstruction of a dense point cloud is accelerated by implementing visual hull only in the ROIs; (2) an accurate polyhedral surface mesh is built by estimating the exact intersections between grid cells and the visual hull; (3) the appearance of the reconstructed presentation is reproduced in a view-dependent manner that respectively renders the non-occluded and occluded region with the nearest camera and its neighboring cameras. The production for volleyball and judo sequences demonstrates the effectiveness of our method in terms of both execution time and visual quality.

In socially assistive robotics, an important research area is the development of adaptation techniques and their effect on human-robot interaction. We present a meta-learning based policy gradient method for addressing the problem of adaptation in human-robot interaction and also investigate its role as a mechanism for trust modelling. By building an escape room scenario in mixed reality with a robot, we test our hypothesis that bi-directional trust can be influenced by different adaptation algorithms. We found that our proposed model increased the perceived trustworthiness of the robot and influenced the dynamics of gaining human's trust. Additionally, participants evaluated that the robot perceived them as more trustworthy during the interactions with the meta-learning based adaptation compared to the previously studied statistical adaptation model.

Accurate vehicle localization is arguably the most critical and fundamental task for autonomous vehicle navigation. While dense 3D point-cloud-based maps enable precise localization, they impose significant storage and transmission burdens when used in city-scale environments. In this paper, we propose a highly compressed representation for LiDAR maps, along with an efficient and robust real-time alignment algorithm for on-vehicle LiDAR scans. The proposed mapping framework, which we refer to as Feature Likelihood Acquisition Map Emulation (FLAME), requires less than 0.1% of the storage space of the original 3D point cloud map. In essence, FLAME emulates an original map through feature likelihood functions. In particular, FLAME models planar, pole and curb features. These three feature classes are long-term stable, distinct and common among vehicular roadways. Multiclass feature points are extracted from LiDAR scans through feature detection. A new multiclass-based point-to-distribution alignment method is proposed to find the association and alignment between the multiclass feature points and the FLAME map. The experimental results show that the proposed framework can achieve the same level of accuracy (less than 10cm) as the 3D point cloud based localization.

Euclidean Signed Distance Field (ESDF) is useful for online motion planning of aerial robots since it can easily query the distance and gradient information against obstacles. Fast incrementally built ESDF map is the bottleneck for conducting real-time motion planning. In this paper, we investigate this problem and propose a mapping system called FIESTA to build global ESDF map incrementally. By introducing two independent updating queues for inserting and deleting obstacles separately, and using Indexing Data Structures and Doubly Linked Lists for map maintenance, our algorithm updates as few as possible nodes using a BFS framework. Our ESDF map has high computational performance and produces near-optimal results. We show our method outperforms other up-to-date methods in term of performance and accuracy by both theory and experiments. We integrate FIESTA into a completed quadrotor system and validate it by both simulation and onboard experiments. We release our method as open-source software for the community.

High-speed trajectory planning through unknown environments requires algorithmic techniques that enable fast reaction times while maintaining safety as new information about the operating environment is obtained. The requirement of computational tractability typically leads to optimization problems that do not include the obstacle constraints (collision checks are done on the solutions) or use a convex decomposition of the free space and then impose an ad-hoc time allocation scheme for each interval of the trajectory. Moreover, safety guarantees are usually obtained by having a local planner that plans a trajectory with a final “stop” condition in the freeknown space. However, these two decisions typically lead to slow and conservative trajectories. We propose FASTER (Fast and Safe Trajectory Planner) to overcome these issues. FASTER obtains high-speed trajectories by enabling the local planner to optimize in both the free-known and unknown spaces. Safety guarantees are ensured by always having a feasible, safe back-up trajectory in the free-known space at the start of each replanning step. Furthermore, we present a Mixed Integer Quadratic Program formulation in which the solver can choose the trajectory interval allocation, and where a time allocation heuristic is computed efficiently using the result of the previous replanning iteration. This proposed algorithm is tested extensively both in simulation and in real hardware, showing agile flights in unknown cluttered environments with velocities up to 3.6 m/s.

Recently, the emerging bio-inspired event cameras have demonstrated potentials for a wide range of robotic applications in dynamic environments. In this paper, we propose a novel fast and asynchronous event-based corner detection method which is called FA-Harris. FA-Harris consists of several components, including an event filter, a Global Surface of Active Events (G-SAE) maintaining unit, a corner candidate selecting unit, and a corner candidate refining unit. The proposed G-SAE maintenance algorithm and corner candidate selection algorithm greatly enhance the real-time performance for corner detection, while the corner candidate refinement algorithm maintains the accuracy of performance by using an improved event-based Harris detector. Additionally, FA-Harris does not require artificially synthesized event-frames and can operate on asynchronous events directly. We implement the proposed method in C++ and evaluate it on public Event Camera Datasets. The results show that our method achieves approximately 8× speed-up when compared with previously reported event-based Harris detector, and with no compromise on the accuracy of performance.

This paper proposes a novel approach for extending monocular visual odometry to a stereo camera system. The proposed method uses an additional camera to accurately estimate and optimize the scale of the monocular visual odometry, rather than triangulating 3D points from stereo matching. Specifically, the 3D points generated by the monocular visual odometry are projected onto the other camera of the stereo pair, and the scale is recovered and optimized by directly minimizing the photometric error. It is computationally efficient, adding minimal overhead to the stereo vision system compared to straightforward stereo matching, and is robust to repetitive texture. Additionally, direct scale optimization enables stereo visual odometry to be purely based on the direct method. Extensive evaluation on public datasets (e.g., KITTI), and outdoor environments (both terrestrial and underwater) demonstrates the accuracy and efficiency of a stereo visual odometry approach extended by scale optimization, and its robustness in environments with challenging textures.

The most recent architectures for Visual Question Answering (VQA), such as TbD or DDRprog, have already outperformed human-level accuracy on benchmark datasets (e.g. CLEVR). We administered an advanced analysis of their performance based on novel metrics called consistency (sum of all object feature instances in the scene (e.g. shapes) equals the total number of the objects in the scene) and revealed only 56% consistency for the most accurate architecture (TbD). In respect to this finding, we propose a new method of the VQA training, which reaches 98% consistency. Furthermore, testing of the VQA model in real world brings out a problem with precise mimicking of the camera position from the original dataset. We therefore created a virtual environment along with its real-world counterpart with variable camera positions to test the accuracy and consistency from different viewports. Based on these errors, we were able to estimate optimal position of the camera. The proposed method thus allows us to find the optimal camera viewport in the real environment without knowing the geometry and the exact position of the camera in the synthetic training environment.

Facial recognition tasks like identity, age, gender, and emotion recognition received substantial attention in recent years. Their deployment in robotic platforms became necessary for the characterization of most of the non-verbal Human-Robot Interaction (HRI) scenarios. In this regard, deep convolution neural networks have shown to be effective on processing different facial representations but with a high cost: to achieve maximum generalization, they require an enormous amount of task-specific labeled data. This paper proposes a unified semi-supervised deep neural model to address this problem. Our hybrid model is composed of an unsupervised deep generative adversarial network which learns fundamental characteristics of facial representations, and a set of convolution channels that fine-tunes the high-level facial concepts for the recognition of identity, age group, gender, and facial expressions. Our network employs progressive lateral connections between the convolution channels so that they share the high-abstraction particularities of each of these tasks in order to reduce the necessity of a large amount of strongly labeled training data. We propose a series of experiments to evaluate each individual mechanism of our hybrid model, in particular, the impact of the progressive connections on learning the specific facial recognition tasks and we observe that our model achieves a better performance when compared to task-specific models.

Searching for objects and distinguishing task-relevant objects from others is a key requirement for service robots. We propose a reinforcement learning solution to the active visual object search problem. Our method successfully learns to explore the environment, to approach the target object, and to decide when to terminate the search as the target object has been found. We demonstrate the efficiency of our solution on a dataset of real-world images collected by a robot. Our approach outperforms state-space planning or other baseline search strategies, reaching a higher success rate in a shorter time. We also study individual subtasks of active visual object search. Although strong baselines exist for the subtasks, our RL solution outperforms them in the overall search task.

We investigate two major limiting factors in the design and implementation of modern dynamics solvers that interfere with their full utilization in versatile, manipulation-driven robotic software architectures. The first limitation originates from the design of those solvers which aims at computational efficiency while neglecting composability. Instead, we advocate to design the solvers in such a way that they exploit linearity in the equations of motion to fully decompose the state of a kinematic chain. This enables a versatile recomposition and more flexible applications. Secondly, we have observed that most implementations follow the programming principle of information hiding. Consequently, the internal state that is used to compute motion control commands is withheld from other parts of the software architecture. We tackle this problem by following a dataflow programming paradigm and separating the software's dataflow from the control flow. Thereafter, we demonstrate those two simple, yet effective strategies to overcome the limitations along various case studies.

In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters. Our approach does not require detailed knowledge about the appearance of the world, and our maps require orders of magnitude less storage than maps utilized by traditional geometry- and LiDAR intensity-based localizers. This is important as self-driving cars need to operate in large environments. Towards this goal, we formulate the problem in a Bayesian filtering framework, and exploit lanes, traffic signs, as well as vehicle dynamics to localize robustly with respect to a sparse semantic map. We validate the effectiveness of our method on a new highway dataset consisting of 312km of roads. Our experiments show that the proposed approach is able to achieve 0.05m lateral accuracy and 1.12m longitudinal accuracy on average while taking up only 0.3% of the storage required by previous LiDAR intensity-based approaches.