IROS 2019 论文列表

The use of embedded boards on robots, including unmanned aerial and ground vehicles, is increasing thanks to the availability of GPU equipped low-cost embedded boards in the market. Porting algorithms originally designed for desktop CPUs on those boards is not straightforward due to hardware limitations. In this paper, we present how we modified and customized the open source SLAM algorithm ORB-SLAM2 to run in real-time on the NVIDIA Jetson TX2. We adopted a data flow paradigm to process the images, obtaining an efficient CPU/GPU load distribution that results in a processing speed of about 30 frames per second. Quantitative experimental results on four different sequences of the KITTI datasets demonstrate the effectiveness of the proposed approach. The source code of our data flow ORB-SLAM2 algorithm is publicly available on GitHub.

We present an approach for localization and semantic mapping in ambiguous scenarios by incrementally maintaining a hybrid belief over continuous states and discrete classification and data association variables. Unlike existing incremental approaches, we explicitly maintain data association components over time, which allows us to deal with perceptual aliasing. Crucially, we utilize a viewpoint-dependent classifier model over rich classifier outputs and leverage the coupling between poses and semantic measurements both for disambiguating data association and in pose estimation. We demonstrate in simulation that incorporating semantic measurements with a viewpoint-dependent classifier model enhances disambiguation of both data association and localization over usage of only geometric measurements or viewpoint independent models, further contributing to the tractability of the approach in practice, and providing better estimates.

In this paper, we present an appearance based line descriptor which was developed with the help of machine learning. Our descriptor uses a ResNet which we modified in its size to improve the performance. We utilized the Unreal Engine and multiple scenes provided for it to create training data. The training was performed using a triplet loss, where the loss of the network is calculated with triplets each consisting of three lines including a matching pair and another non-matching line. During learning, the goal of the minimization function is to calculate descriptors with minimal descriptor distance to matching lines' descriptors and maximal descriptor distance to other lines' descriptors. We evaluate the performance of our descriptor on our synthetic datasets, on real-world stereo images from the Middlebury Stereo Dataset and on a benchmark for line segment matching. The results show that in comparison to state-of-the-art line descriptors our method achieves a greatly improved line matching accuracy.

Infrared imaging spectrometer (IRIS) often suffers from overlapped bands and random noises, which limit the precision of subsequent processing in robot vision sensing. To address this problem, we propose a novel Gabor transform-based infrared spectrum restoration method by successfully exploring the intrinsic structure of the clean IR spectrum from the degraded one. At first, a total variation (TV) regularized Gabor coefficients adjustment descriptor is designed and incorporated into the spectrum restoration model. Then, the proposed model is inferred via an efficient optimization approach based on split Bregman iteration method. Comprehensive experiments illustrate the significant and consistent improvements of the developed model over state-of-the-art approaches. The restored high-resolution spectrum can be utilized for detecting the different materials in the robot visual tracking systems.

We present a novel dataset for training and benchmarking semantic SLAM methods. The dataset consists of 200 long sequences, each one containing 3000-5000 data frames. We generate the sequences using realistic home layouts. For that we sample trajectories that simulate motions of a simple home robot, and then render the frames along the trajectories. Each data frame contains a) RGB images generated using physically-based rendering, b) simulated depth measurements, c) simulated IMU readings and d) ground truth occupancy grid of a house. Our dataset serves a wider range of purposes compared to existing datasets and is the first large-scale benchmark focused on the mapping component of SLAM. The dataset is split into train/validation/test parts sampled from different sets of virtual houses. We present benchmarking results for both classical geometry-based [1], [2] and recent learning-based [3] SLAM algorithms, a baseline mapping method [4], semantic segmentation [5] and panoptic segmentation [6]. The dataset and source code for reproducing our experiments will be publicly available at the time of publication.

In this paper, we present the first large-scale synthetic dataset for visual perception in disaster scenarios, and analyze state-of-the-art methods for multiple computer vision tasks with reference baselines. We simulated before and after disaster scenarios such as fire and building collapse for fifteen different locations in realistic virtual worlds. The dataset consists of more than 300K high-resolution stereo image pairs, all annotated with ground-truth data for semantic segmentation, depth, optical flow, surface normal estimation and camera pose estimation. To create realistic disaster scenes, we manually augmented the effects with 3D models using physical-based graphics tools. We use our dataset to train state-of-the-art methods and evaluate how well these methods can recognize the disaster situations and produce reliable results on virtual scenes as well as real-world images. The results obtained from each task are then used as inputs to the proposed visual odometry network for generating 3D maps of buildings on fire. Finally, we discuss challenges for future research.

Datasets are an essential component for training effective machine learning models. In particular, surgical robotic datasets have been key to many advances in semi-autonomous surgeries, skill assessment, and training. Simulated surgical environments can enhance the data collection process by making it faster, simpler and cheaper than real systems. In addition, combining data from multiple robotic domains can provide rich and diverse training data for transfer learning algorithms. In this paper, we present the DESK (DExterous Surgical SKills) dataset. It comprises a set of surgical robotic skills collected during a surgical training task using three robotic platforms: the Taurus II robot, Taurus II simulated robot, and the YuMi robot. This dataset was used to test the idea of transferring knowledge across different domains (e.g. from Taurus to YuMi robot) for a surgical gesture classification task with seven gestures/surgemes. We explored two different scenarios: 1) No transfer and 2) Domain transfer (simulated Taurus to real Taurus and YuMi robots). We conducted extensive experiments with three supervised learning models and provided baselines in each of these scenarios. Results show that using simulation data during training enhances the performance on the real robots, where limited real data is available. In particular, we obtained an accuracy of 55% on the real Taurus data using a model that is trained only on the simulator data, but that accuracy improved to 82% when the ratio of real to simulated data was increased to 0.18 in the training set.

In recent years, there has been a rapid increase in the number of service robots deployed for aiding people in their daily activities. Unfortunately, most of these robots require human input for training in order to do tasks in indoor environments. Successful domestic navigation often requires access to semantic information about the environment, which can be learned without human guidance. In this paper, we propose a set of DEDUCE1 - Diverse scEne Detection methods in Unseen Challenging Environments algorithms which incorporate deep fusion models derived from scene recognition systems and object detectors. The five methods described here have been evaluated on several popular recent image datasets, as well as real-world videos acquired through multiple mobile platforms. The final results show an improvement over the existing state-of-the-art visual place recognition systems.

Given 3D LiDAR point clouds, how can we segment them fast and accurately? Fast and accurate segmentation of 3D LiDAR points is an important issue in mobile robotics with various applications in classification, tracking, SLAM, etc. Despite its importance, existing methods do not provide both speed and accuracy; in particular, methods performing segmentation in the 3D domain are too slow, disabling its use in real-time processing. In this paper, we propose Curved-Voxel Clustering (CVC), a fast and accurate method for segmenting 3D LiDAR point clouds utilizing LiDAR-optimized curved-voxel. CVC attains fine discriminations by considering three important aspects for clustering 3D LiDAR points: distance from the sensor, directional resolutions, and rarity of points. CVC succeeds in providing real-time performance by carefully managing curved-voxels with a hash table. Especially, CVC works well on sparse 3D point clouds. Through experiments, we show that our method is up to 1.7× faster and 30% more accurate than other segmentation methods. CVC enables real-time segmentation with more than 20 runs in a second.

Highly accurate maps of the road infrastructure are a crucial cornerstone for self-driving cars to enable navigation in complex traffic scenarios. Traditional methods for creating detailed maps of road environments involve expensive survey vehicles that cannot keep up with the frequent changes in the road network. In this paper, we propose a novel method to derive detailed high-definition maps by crowd sourcing data using commodity sensors. Our system uses multi-session feature-based visual SLAM to align submaps recorded by individual vehicles on a central backend server. We reconstruct 3D boundaries of road infrastructure elements such as road markings and road boundaries from semantic object contours detected in keyframes by a neural network. The result is a concise map of semantically meaningful objects suitable both for localization and higher-level planning tasks of automated vehicles. We evaluate our method on real-world data against a globally referenced ground-truth map demonstrating a high level of detail and metric accuracy.

In robotic manipulation, finding a feasible motion plan doesn't guarantee a successful execution. The real world could bring all kinds of unexpected changes to the planned motion, the most deadly ones are usually marked by or caused by unexpected changes of contacts (object slipping away between fingers; getting stuck somewhere, etc). We notice that some actions are more likely to maintain desired contacts than others. To help finding these actions, in this work we propose a set of criteria to quantify the robustness of contacts against modeling uncertainties and disturbance forces. Under the quasi-static assumption, we analyze the causes of contact mode (sticking, sliding, disengaged) transitions and discuss how to endure larger uncertainties and disturbances. We summarize our results into several physically meaningful and easy-to-compute scores, which can be used to evaluate the quality of each individual contacts in a manipulation system. We illustrate the meaning of the scores with a simple example.

Unmanned Aerial Vehicles (UAVs) have been implemented for environmental monitoring by using their capabilities of mobile sensing, autonomous navigation, and remote operation. However, in real-world applications, the limitations of on-board resources (e.g., power supply) of UAVs will constrain the coverage of the monitored area and the number of the acquired samples, which will hinder the performance of field estimation and mapping. Therefore, the issue of constrained resources calls for an efficient sampling planner to schedule UAV-based sensing tasks in environmental monitoring. This paper presents a mission planner of coverage sampling and path planning for a UAV-enabled mobile sensor to effectively explore and map an unknown environment that is modeled as a random field. The proposed planner can generate a coverage path with an optimal coverage density for exploratory sampling, and the associated energy cost is subjected to a power supply constraint. The performance of the developed framework is evaluated and compared with the existing state-of-the-art algorithms, using a real-world dataset that is collected from an environmental monitoring program as well as physical field experiments. The experimental results illustrate the reliability and accuracy of the presented coverage sampling planner in a prior survey for environmental exploration and field mapping.

Planning the path to gather the surface information of the target objects is crucial to improve the efficiency of and reduce the overall cost, for visual inspection applications with Unmanned Aerial Vehicles (UAVs). Coverage Path Planning (CPP) problem is often formulated for these inspection applications because of the coverage requirement. Traditionally, researchers usually plan and optimize the viewpoints to capture the surface information first, and then optimize the path to visit the selected viewpoints. In this paper, we propose a novel planning method to directly sample and plan the inspection path for a camera-equipped UAV to acquire visual and geometric information of the target structures as a video stream setting in complex 3D environment. The proposed planning method first generates via-points and path primitives around the target object by using sampling methods based on voxel dilation and subtraction. A novel Primitive Coverage Graph (PCG) is then proposed to encode the topological information, flying distances, and visibility information, with the sampled via-points and path primitives. Finally graph search is performed to find the resultant path in the PCG to complete the inspection task with the coverage requirements. The effectiveness of the proposed method is demonstrated through simulation and field tests in this paper.

Delay compensation in filter based sensor fusion frameworks for multiple sensors with varying delays and different rates quickly results in large computational overhead should the delayed measurements be incorporated in a statistically meaningful way. Even more so if high rate propagation sensors (e.g. IMU) are used. This work presents an approach to implement such frameworks with significant complexity reduction compared to standard implementations. We set particular focus on the state covariance propagation as this chain of re-computations (i.e. FPFT + Q per propagation step) upon a delayed update is the dominant bottleneck. We draw our inspiration from the scattering theory and propose a method which projects the idea of wave propagation to an efficient concatenation of covariance propagation steps between filter updates. Through this approach, we reach a speed-up of more than a factor of 10 for the covariance propagation and render the computational complexity independent of the number of propagation steps between filter updates. We evaluated our method in simulation and with real data.

To solve the poor compliance and safety problems in current rehabilitation robots, a novel two-degrees-of-freedom (2-DOF) soft ankle rehabilitation robot driven by pneumatic muscles (PMs) is presented, taking advantages of the PM's inherent compliance and the parallel structure's high stiffness and payload capacity. However, the PM's nonlinear, time-varying and hysteresis characteristics, and the coupling interference from parallel structure, as well as the unpredicted disturbance caused by arbitrary human behavior all raise difficulties in achieving high-precision control of the robot. In this paper, a multi-input-multi-output disturbance compensated sliding mode controller (MIMO-DCSMC) is proposed to tackle these problems. The proposed control method can tackle the un-modeled uncertainties and the coupling interference existed in multiple PMs' synchronous movement, even with the subject's participation. Experiment results on a healthy subject confirmed that the PMs-actuated ankle rehabilitation robot controlled by the proposed MIMO-DCSMC is able to assist patients to perform high-accuracy rehabilitation tasks by tracking the desired trajectory in a compliant manner.

Harmful algal blooms occur frequently and deteriorate water quality. A reliable method is proposed in this paper to track algal blooms using a set of autonomous surface robots. A satellite image indicates the existence and initial location of the algal bloom for the deployment of the robot system. The algal bloom area is approximated by a circle with time varying location and size. This circle is estimated and circumnavigated by the robots which are able to locally sense its boundary. A multi-agent control algorithm is proposed for the continuous monitoring of the dynamic evolution of the algal bloom. Such algorithm comprises of a decentralised least squares estimation of the target and a controller for circumnavigation. We prove the convergence of the robots to the circle and in equally spaced positions around it. Simulation results with data provided by the SINMOD ocean model are used to illustrate the theoretical results.

Recent work in decentralized, schedule-driven traffic control has demonstrated the ability to improve the efficiency of traffic flow in complex urban road networks. In this approach, a scheduling agent is associated with each intersection. Each agent senses the traffic approaching its intersection and in real-time constructs a schedule that minimizes the cumulative wait time of vehicles approaching the intersection over the current look-ahead horizon. In this paper, we propose a cooperative algorithm that utilizes both connected and autonomous vehicles (CAV) and schedule-driven traffic control to create better traffic flow in the city. The algorithm enables an intersection scheduling agent to adjust the arrival time of an approaching platoon through use of wireless communication to control the velocity of vehicles. The sequence of approaching platoons is thus shifted toward a new shape that has smaller cumulative delay. We demonstrate how this algorithm outperforms the original approach in a real-time traffic signal control problem.

Most range-only simultaneous localization and mapping (RO-SLAM) algorithms have considered direct measurements between robot and its neighbor nodes only, even though other inter-node measurements can be actively used to improve the performance. In addition, most of them assume nodes are static, i.e., fixed on one place and cannot be applied to dynamic environments where some or all of nodes are moving. To address the aforementioned issues, a cooperative dynamic RO-SLAM (CDRO-SLAM) is proposed in this paper, which is a RO-SLAM algorithm based on sum of Gaussian (SoG) filter with cooperative measurements in dynamic environments. The proposed CDRO-SLAM integrates all the inter-node measurements for localization, which results in a smaller map estimation error with a faster convergence speed than the conventional ROSLAM. Moreover, the proposed CDRO-SLAM can also track movement of any nodes under the dynamic environment by resetting and updating the SoG variables. Such advantages of the proposed CDRO-SLAM algorithm are verified with the real experimental data.

Employing small size aerial robots, acting as mobile airborne sensors, to work alongside ground robots can be extremely useful in many different robotic missions. Due to the strict constraints in terms of size, weight, 3D coverage, processing power and power consumption. There are not many technological possibilities for performing independent self-localization for such tiny robots. This paper describes a cooperative audio-visual localization system to robustly estimate the position of a small aerial robot from a ground robot. Experimental results with a 40-gram quadrotor assess the performance of the system and demonstrate the reliability gained through fusion of sound measurements with visual information.