IROS 2020 论文列表

We report the feasibility study of a new optoacoustic sensor for both near-distance ranging and material thickness classification for robotic grasping. It is based on the optoacoustic effect where focused laser pulses are used to generate wideband ultrasound signals in the target. With a much smaller optical focal spot, the optoacoustic sensor achieves a lateral resolution of 93 μm, which is six times higher than ultrasound pulse-echo ranging under the same condition. A new multi-mode wideband PZT (lead zirconate titanate) transducer is built to properly receive the wideband optoacoustic signal. The ability to receive both low- and high-frequency components of the optoacoustic signal enhances the material sensing capability, which makes it promising to determine not only material type but also the sub-surface structures. For demonstration, optoacoustic spectra are collected from hard and soft materials with different thickness. A Bag-of-SFA-Symbols (BOSS) classifier is designed to perform primary material and then thickness classification based on the optoacoustic spectra. The accuracy of material / thickness classification reaches ≥ 99% and ≥ 94%, respectively, which shows the feasibility of differentiating solid materials with different thickness by the optoacoustic sensor.

Brain-Computer Interfaces (BCI) based on Electroencephalography (EEG) signals, in particular motor imagery (MI) data have received a lot of attention and show the potential towards the design of key technologies both in healthcare and other industries. MI data is generated when a subject imagines movement of limbs and can be used to aid rehabilitation as well as in autonomous driving scenarios. Thus, classification of MI signals is vital for EEG-based BCI systems. Recently, MI EEG classification techniques using deep learning have shown improved performance over conventional techniques. However, due to inter-subject variability, the scarcity of unseen subject data, and low signal-to-noise ratio, extracting robust features and improving accuracy is still challenging. In this context, we propose a novel two-way few shot network that is able to efficiently learn how to learn representative features of unseen subject categories and how to classify them with limited MI EEG data. The pipeline includes an embedding module that learns feature representations from a set of samples, an attention mechanism for key signal feature discovery, and a relation module for final classification based on relation scores between a support set and a query signal. In addition to the unified learning of feature similarity and a few shot classifier, our method leads to emphasize informative features in support data relevant to the query data, which generalizes better on unseen subjects. For evaluation, we used the BCI competition IV 2b dataset and achieved an 9.3% accuracy improvement in the 20-shot classification task with state-of-the-art performance. Experimental results demonstrate the effectiveness of employing attention and the overall generality of our method.

Haptic feedback in teleoperation of flying robots can enable safe flight in unknown and densely cluttered environments. It is typically part of the robot's control scheme and used to aid navigation and collision avoidance via artificial force fields displayed to the operator. However, to achieve fully immersive embodiment in this context, high fidelity force feedback is needed. In this paper we present a telepresence scheme that provides haptic feedback of the external forces or wind acting on the robot, leveraging the ability of a state-of-the-art flying robot to estimate these values online. As a result, we achieve true force feedback telepresence in flying robots by rendering the actual forces acting on the system. To the authors' knowledge, this is the first telepresence scheme for flying robots that is able to feedback real contact forces and does not depend on their representations. The proposed event-based teleoperation scheme is stable under varying latency conditions. Secondly, we present a haptic interface design such that any haptic interface with at least as many force-sensitive and active degrees of freedom as the flying robot can implement this telepresence architecture. The approach is validated experimentally using a Skydio R1 autonomous flying robot in combination with a ForceDimension sigma.7 and a Franka Emika Panda as haptic devices.

We present a hierarchical framework for trajectory optimization and optimal feedback whole-body control of wheeled inverted pendulum (WIP) humanoid robot. The framework extends rapidly exponentially stabilizing control Lyapunov functions (RES-CLF) to operational space for controlling WIP humanoid robots while utilizing a hierarchical framework to compute an optimal policy. The upper level of the hierarchy encodes locomotion tasks, while the lower level incorporates the full system dynamics, including manipulation tasks to be performed. The framework computes an optimal policy directly in the operational space. Thus it avoids computing inverse kinematics or inverse dynamics explicitly. The framework can handle torque and task constraints while guaranteeing exponential convergence and min-norm control from RES-CLF. The efficacy of the framework is demonstrated on 18 degrees of freedom (DoF) WIP humanoid robot, Golem Krang, and 7 DoF planar WIP humanoid robot.

In this work, we address the motion planning problem for autonomous vehicles through a new lattice planning approach, called Feedback Enhanced Lattice Planner (FELP). Existing lattice planners have two major limitations, namely the high dimensionality of the lattice and the lack of modeling of agent vehicle behaviors. We propose to apply the Intelligent Driver Model (IDM) [1] as a speed feedback policy to address both of these limitations. IDM both enables the responsive behavior of the agents, and uniquely determines the acceleration and speed profile of the ego vehicle on a given path. Therefore, only a spatial lattice is needed, while discretization of higher order dimensions is no longer required. Additionally, we propose a directed-graph map representation to support the implementation and execution of lattice planners. The map can reflect local geometric structure, embed the traffic rules adhering to the road, and is efficient to construct and update. We show that FELP is more efficient compared to other existing lattice planners through runtime complexity analysis, and we propose two variants of FELP to further reduce the complexity to polynomial time. We demonstrate the improvement by comparing FELP with an existing spatiotemporal lattice planner using simulations of a merging scenario and continuous highway traffic. We also study the performance of FELP under different traffic densities.

The recent introduction of powerful embedded graphics processing units (GPUs) has allowed for unforeseen improvements in real-time computer vision applications. It has enabled algorithms to run onboard, well above the standard video rates, yielding not only higher information processing capability, but also reduced latency. This work focuses on the applicability of efficient low-level, GPU hardware-specific instructions to improve on existing computer vision algorithms in the field of visual-inertial odometry (VIO). While most steps of a VIO pipeline work on visual features, they rely on image data for detection and tracking, of which both steps are well suited for parallelization. Especially non-maxima suppression and the subsequent feature selection are prominent contributors to the overall image processing latency. Our work first revisits the problem of non-maxima suppression for feature detection specifically on GPUs, and proposes a solution that selects local response maxima, imposes spatial feature distribution, and extracts features simultaneously. Our second contribution introduces an enhanced FAST feature detector that applies the aforementioned non-maxima suppression method. Finally, we compare our method to other state-of-the-art CPU and GPU implementations, where we always outperform all of them in feature tracking and detection, resulting in over 1000fps throughput on an embedded Jetson TX2 platform. Additionally, we demonstrate our work integrated into a VIO pipeline achieving a metric state estimation at ~200fps.Code available at: https://github.com/uzh-rpg/vilib.

Electronic Health Record (EHR) and healthcare claim data provide rich clinical information for time series analysis. In this work, we provide a different angle of solving healthcare multivariate time series classification by turning it into a computer vision problem. We propose a Convolutional Feature Engineering (CFE) methodology, that can effectively extract long sequence dependency time series features. Combined with LightGBM, it can achieve the state-of-the-art results with 35X speed acceleration compared with LSTM based approaches on MIMIC-III In Hospital Mortality benchmark task. We deploy CFE based LightGBM into our Mortality Early Warning System at Humana, and train it on 1 million member samples. The offline metrics shows that this new approach generates better-quality predictions than previous LSTM based approach, and meanwhile greatly decrease the training and inference time.

We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based on Bayesian neural networks have been proposed to handle this issue, they are computationally expensive. Thus, to speed up the processing time, our approach applies a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach. By using synthetically generated images of spacecraft, we demonstrate that the trained generative model can produce the uncertainty map 100∼700 times faster than the conventional uncertainty estimation method used for training the generative model itself. We also show that the quality of uncertainty map derived by the generative model is close to that of the original uncertainty map. By applying the proposed approach, the deep learning model operated in real-time can avoid disastrous failures by considering the uncertainty as well as achieving better performance removing uncertain portions of the prediction result.

Touch is arguably the most important sensing modality in physical interactions. However, tactile sensing has been largely under-explored in robotics applications owing to the complexity in making perceptual inferences until the recent advancements in machine learning or deep learning in particular. Touch perception is strongly influenced by both its temporal dimension similar to audition and its spatial dimension similar to vision. While spatial cues can be learned episodically, temporal cues compete against the system’s re-sponse/reaction time to provide accurate inferences. In this paper, we propose a fast tactile-based texture classification framework which makes use of the spiking neural network to learn from the neural coding of the conventional tactile sensor readings. The framework is implemented and tested on two independent tactile datasets collected in sliding motion on 20 material textures. Our results show that the framework is able to make much more accurate inferences ahead of time as compared to that by the state-of-the-art learning approaches.

In this work, we propose a system for humanoid robot fast motions. When a humanoid robot performs a motion such as a tennis forehand stroke motion, a whole-body fast motion in reaction to visual information is required. There are three problems to tackle. (1) Motion is desired to be quick. (2) Real-time visual processing considering visual noises is needed. (3) Real-time joint angle modification with balance keeping is needed. To solve the problem (1), we used an offline optimization system to enhance the motion speed. To solve the problem (2), we implement a ball trajectory prediction algorithm using the Extended Kalman Filter (EKF). To solve the trade-off between (1) and (3), we propose an offline optimization condition with an estimated balance margin. By using these methods, we achieved a non-step tennis forehand stroke motion with a humanoid robot by predicting a ball's trajectory with stereo cameras on the robot's head.

Multi-goal curvature-constrained planning such as the Dubins Traveling Salesman Problem (DTSP) combines NP-hard combinatorial routing with continuous optimization to determine the optimal vehicle heading angle for each target location. The problem can be addressed as combinatorial routing using a finite set of heading samples at target locations. In such a case, optimal heading samples can be determined for a sequence of targets in polynomial time, and the DTSP can be solved as searching for a sequence with the minimal cost. However, the examination of sequences can be computationally demanding for high numbers of heading samples and target locations. A fast rejection schema is proposed to quickly examine unfavorable sequences using lower bound estimation of Dubins tour cost based on windowing technique that evaluates short subtours of the sequences. Furthermore, the computation using small problem instances can benefit from reusing stored results and thus speed up the search. The reported results indicate that the computational burden is decreased about two orders of magnitude, and the proposed approach supports finding high-quality solutions of routing problems with Dubins vehicle.

Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain where the dynamics of the robot can be significantly different from one another. In this paper, first, we show that when meta-training situations (the prior situations) have such diverse dynamics, using a single set of meta-trained parameters as a starting point still requires a large number of observations from the real system to learn a useful model of the dynamics. Second, we propose an algorithm called FAMLE that mitigates this limitation by meta-training several initial starting points (i.e., initial parameters) for training the model and allows robots to select the most suitable starting point to adapt the model to the current situation with only a few gradient steps. We compare FAMLE to MBRL, MBRL with a meta-trained model with MAML, and model-free policy search algorithm PPO for various simulated and real robotic tasks, and show that FAMLE allows robots to adapt to novel damages in significantly fewer time-steps than the baselines.

This paper studies the problem of Image-Based Visual Servo Control (IBVS) for quadrotors. Although the control of quadrotors has been extensively studied in the last decades, combining the IBVS module with the quadrotor's dynamics is still hard, mainly due to the under-actuation issues related to the quadrotor control as opposed to the 6 DoF control outputs generated by the IBVS modules. We propose an alternative formulation to solve this problem, by particularly using linear Model Predictive Control (MPC), that allows us to relax the UAVs under-actuation issues. Stability guarantees of the proposed scheme are presented. The proposed model is validated with synthetic data and tested in a real UAV's setup.

In this paper, we propose a method for automatically generating object handling actions based on simple action definitions. The need to replace workers by robots is increasing, and, in fact, many research projects on robots have worked with simple motion definitions. Many applications are for mobile robots such as drones, however, and if such methods are applied directly to object handling, like a pick and place operation, it is necessary for humans to give detailed instructions. Hence, our contribution is to propose a model that simulates the real world with an augmented hybrid system that includes the states of objects. Then, it becomes possible to automatically generate robot motions with simple motion definitions and calculate them within a reasonable time. We demonstrate through computer simulation with a dual-arm robot that robot motions can be generated by simple definitions even if the environment changes to a certain degree.

This work presents a motion planning algorithm for legged robots capable of constructing long-horizon dynamic plans in real-time. Many existing methods use models that prohibit flight phases or even require static stability, while those that permit these dynamics often plan over short horizons or take minutes to compute. The algorithm presented here resolves these issues through a reduced-order dynamical model that handles motion primitives with stance and flight phases and supports an RRT-Connect framework for rapid exploration. Kinematic and dynamic constraint approximations are computed efficiently and validated with a whole-body trajectory optimization. The algorithm is tested over challenging terrain requiring long planning horizons and dynamic motions in seconds - an order of magnitude faster than existing methods. The speed and global nature of the planner offer a new level of autonomy for legged robot applications.

Accurate and reliable tracking of multiple moving objects in 3D space is an essential component of urban scene understanding. This is a challenging task because it requires the assignment of detections in the current frame to the predicted objects from the previous one. Existing filter-based approaches tend to struggle if this initial assignment is not correct, which can happen easily.We propose a novel optimization-based approach that does not rely on explicit and fixed assignments. Instead, we represent the result of an off-the-shelf 3D object detector as Gaussian mixture model, which is incorporated in a factor graph framework. This gives us the flexibility to assign all detections to all objects simultaneously. As a result, the assignment problem is solved implicitly and jointly with the 3D spatial multi-object state estimation using non-linear least squares optimization.Despite its simplicity, the proposed algorithm achieves robust and reliable tracking results and can be applied for offline as well as online tracking. We demonstrate its performance on the real world KITTI tracking dataset and achieve better results than many state-of-the-art algorithms. Especially the consistency of the estimated tracks is superior offline as well as online.

This paper presents F-Siamese Tracker, a novel approach for single object tracking prominently characterized by more robustly integrating 2D and 3D information to reduce redundant search space. A main challenge in 3D single object tracking is how to reduce search space for generating appropriate 3D candidates. Instead of solely relying on 3D proposals, firstly, our method leverages the Siamese network applied on RGB images to produce 2D region proposals which are then extruded into 3D viewing frustums. Besides, we perform an on-line accuracy validation on the 3D frustum to generate refined point cloud searching space, which can be embedded directly into the existing 3D tracking backbone. For efficiency, our approach gains better performance with fewer candidates by reducing search space. In addition, benefited from introducing the online accuracy validation, for occasional cases with strong occlusions or very sparse points, our approach can still achieve high precision, even when the 2D Siamese tracker loses the target. This approach allows us to set a new state-of-the-art in 3D single object tracking by a significant margin on a sparse outdoor dataset (KITTI tracking). Moreover, experiments on 2D single object tracking show that our framework boosts 2D tracking performance as well.

While automotive radars are widely used in most assisted and autonomous driving systems, only a few works were proposed to tackle the calibration problems of automotive radars with other perception sensors. One of the key calibration challenges of automotive planar radars with other sensors is the missing elevation angle in 3D space. In this paper, extrinsic calibration is accomplished based on the observation that the radar cross section (RCS) measurements have different value distributions across radar's vertical field of view. An approach to accurately and efficiently estimate the time delay between radars and LiDARs based on spatial-temporal relationships of calibration target positions is proposed. In addition, a localization method for calibration target detection and localization in pre-built maps is proposed to tackle insufficient LiDAR measurements on calibration targets. The experimental results show the feasibility and effectiveness of the proposed Radar-LiDAR extrinsic and temporal calibration approaches.

The computational difficulty of planning search paths that seek to maximize a general deterministic value function increases dramatically as desired path lengths increase. Mobile search agents with limited computational resources often utilize receding horizon methods to address the path planning problem. Unfortunately, receding horizon planners may perform poorly due to myopic planning horizons. We provide methods of incorporating terminal costs in the construction of receding horizon paths that provide a theoretical lower bound on the performance of the search paths produced. The results presented in this paper are of particular value in subsea search applications. We present results from simulated subsea search missions that use real-world data acquired by an autonomous underwater vehicle during a subsea survey of Boston Harbor.