IROS 2020 论文列表

Ground plane estimation and ground point segmentation is a crucial precursor for many applications in robotics and intelligent vehicles like navigable space detection and occupancy grid generation, 3D object detection, point cloud matching for localization and registration for mapping. In this paper, we present GndNet, a novel end-to-end approach that estimates the ground plane elevation information in a grid-based representation and segments the ground points simultaneously in real-time. GndNet uses PointNet and Pillar Feature Encoding network to extract features and regresses ground height for each cell of the grid. We augment the SemanticKITTI dataset to train our network. We demonstrate qualitative and quantitative evaluation of our results for ground elevation estimation and semantic segmentation of point cloud. GndNet establishes a new state-of-the-art, achieves a run-time of 55Hz for ground plane estimation and ground point segmentation.

Map based visual inertial localization is a crucial step to reduce the drift in state estimation of mobile robots. The underlying problem for localization is to estimate the pose from a set of 3D-2D feature correspondences, of which the main challenge is the presence of outliers, especially in changing environment. In this paper, we propose a robust solution based on efficient global optimization of the consensus maximization problem, which is insensitive to high percentage of outliers. We first introduce translation invariant measurements (TIMs) for both points and lines to decouple the consensus maximization problem into rotation and translation subproblems, allowing for a two-stage solver with reduced search space. Then we show that (i) the rotation can be estimated by minimizing TIMs using only 1-dimensional branch-and-bound (BnB), (ii) the translation can be estimated by running 1-dimensional search for each of the three axes with prioritized progressive voting. Compared with the popular randomized solver, our solver achieves deterministic global convergence without requiring an initial value. Furthermore, ours is exponentially faster compared with existing BnB based methods. Finally, our experiments on both simulation and real-world datasets demonstrate that the proposed method gives accurate pose estimation even in the presence of 90% outliers (only 2 inliers).

Many applications with mobile robots require self-localization in indoor maps. While such maps can be previously generated by SLAM strategies, there are various localization approaches that use 2D floor plans as reference input. In this paper, we present a localization strategy using floor plan as map, which is based on spatial density information computed from dense depth data of RGB-D cameras. We propose an interval-based model, called Interval Free-Space Density, that bounds the uncertainty of observations and minimizes the effects of movable objects in the environment. Our model was applied in a Monte Carlo Localization strategy and compared with traditional observation models. The results of experiments showed the robustness of the proposed method in single-camera and multi-camera experiments in home environments.

We present a simple, yet effective and flexible method for action recognition supporting multiple sensor modalities. Multivariate signal sequences are encoded in an image and are then classified using a recently proposed EfficientNet CNN architecture. Our focus was to find an approach that generalizes well across different sensor modalities without specific adaptions while still achieving good results. We apply our method to 4 action recognition datasets containing skeleton sequences, inertial and motion capturing measurements as well as Wi-Fi fingerprints that range up to 120 action classes. Our method defines the current best CNN-based approach on the NTU RGB+D 120 dataset, lifts the state of the art on the ARIL Wi-Fi dataset by +6.8%, improves the UTD-MHAD inertial baseline by +14.4%, the UTD-MHAD skeleton baseline by +0.5% and achieves 96.1% on the Simitate motion capturing data (80/20 split). We further demonstrate experiments on both, modality fusion on a signal level and signal reduction to prevent the representation from overloading.

Common experience suggests that agents who know each other well are better able to work together. In this work, we address the problem of calibrating intention and capabilities in human-robot collaboration. In particular, we focus on scenarios where the robot is attempting to assist a human who is unable to directly communicate her intent. Moreover, both agents may have differing capabilities that are unknown to one another. We adopt a decision-theoretic approach and propose the TICC-POMDP for modeling this setting, with an associated online solver. Experiments show our approach leads to better team performance both in simulation and in a real-world study with human subjects.

We present a pipeline for geomorphological analysis that uses structure from motion (SfM) and deep learning on close-range aerial imagery to estimate spatial distributions of rock traits (size, roundness, and orientation) along a tectonic fault scarp. The properties of the rocks on the fault scarp derive from the combination of initial volcanic fracturing and subsequent tectonic and geomorphic fracturing, and our pipeline allows scientists to leverage UAS-based imagery to gain a better understanding of such surface processes. We start by using SfM on aerial imagery to produce georeferenced orthomosaics and digital elevation models (DEM). A human expert then annotates rocks on a set of image tiles sampled from the orthomosaics, and these annotations are used to train a deep neural network to detect and segment individual rocks in the entire site. The extracted semantic information (rock masks) on large volumes of unlabeled, high-resolution SfM products allows subsequent structural analysis and shape descriptors to estimate rock size, roundness, and orientation. We present results of two experiments conducted along a fault scarp in the Volcanic Tablelands near Bishop, California. We conducted the first, proof-of-concept experiment with a DJI Phantom 4 Pro equipped with an RGB camera and inspected if elevation information assisted instance segmentation from RGB channels. Rock-trait histograms along and across the fault scarp were obtained with the neural network inference. In the second experiment, we deployed a hexrotor and a multispectral camera to produce a DEM and five spectral orthomosaics in red, green, blue, red edge, and near infrared. We focused on examining the effectiveness of different combinations of input channels in instance segmentation.

Modern robots act in dynamic and partially unknown environments where path replanning can be mandatory if changes in the environment are observed. Task-prioritized control strategies are well known and effective solutions to ensure local adaptation of robot behaviour. The highest priority in a stack of tasks is typically given to the management of correct robot operation or safe interaction with the environment such as obstacles or joint limits avoidance, that we can consider as constraints. If a constraint makes impossible achieving a certain task, such as tracking a Cartesian trajectory, a local control algorithm partially sacrifices the latter which is only accomplished to the best of the robot's ability to generate internal motions. In this control framework, problems may occur in some applications, like in the surgical domain, where it is not safe that some tasks are simply sacrificed without prior notice. The contribution of this work is to introduce a coordinate invariant index, that is used to provide a geometrical interpretation of task conflicts in a task-priority control framework and to develop a method for on-line detection of algorithmic singularities, with the goal of increasing safety and performances during robot operations.

There is a high cost associated to the time and expertise required to program complex robot applications with high variability. This is one of the main barriers that inhibit the entry of robotic automation in small and medium-sized enterprises. To tackle the high level of task uncertainty associated with changing conditions of the environment, we propose a framework that leverages a combination between learning from demonstration (LfD) and constraint-based task specification and control. This synergy enables our framework to use LfD to generalize reactive approach motions (RAMo) towards not only a single pose but towards an allowable manifold defined with respect to the object to interact with. As a result, the robot executes the task by following a feasible approach motion gen-eralized from the learned information. This approach motion is generated based on an initial representation of the environment, and it can be reactively adapted in function of current updates of the environment using sensor information. The proposed framework enables the system to deal with applications that involve a high level of uncertainty, increasing the flexibility and robustness, compared to traditional sense-plan-act paradigms.

We propose a method of subdividing robot tasks into new lower abstract tasks. The description of robot tasks in an abstract manner is effective for motion planning for complex tasks and teaching robot movements in various environments. However, a more efficient task description may be obtained by using a lower abstraction according to the work environment. We argue that a higher abstract task can be expressed as a new lower abstract subtasks by applying Grid-based Signal Temporal Inference (Grid-TLI). We show that a new task can be completed using the Signal Temporal Logic formula for each cluster. We demonstrated the efficiency of our method through computer simulations using a 2-D security robot task.

We propose an algorithm for generating minimum-snap trajectories for quadrotors with linear computational complexity with respect to the number of segments in the spline trajectory. Our algorithm is numerically stable for large numbers of segments and is able to generate trajectories of more than 500, 000 segments. The computational speed and numerical stability of our algorithm makes it suitable for real-time generation of very large scale trajectories. We demonstrate the performance of our algorithm and compare it to existing methods, in which it is both faster and able to calculate larger trajectories than state-of-the-art. We also show the feasibility of the trajectories experimentally with a long quadrotor flight.

In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states, and update task assignments to robots as quickly as possible. We present a forward simulation-based alert system that proactively notifies the human supervisor of possible, negatively-impactful events, which provides an opportunity for the human to retask agents to avoid undesirable scenarios. We propose methods for speeding up mission simulations and extracting alerts from simulation data in order to enable real-time alert generation suitable for time-critical missions. We present the results from a user trial and verify our hypothesis that the decision making performance of human supervisors can be improved by introducing forward simulation-based alerts.

Robots should be able to learn and perform a manipulation task across different settings. This paper presents an approach that learns an RNN-based manipulation skill model from demonstrations and then generalizes the learned skill in new settings. The manipulation skill model learned from demonstrations in an initial set of setting performs well in those settings and similar ones. However, the model may perform poorly in a novel setting that is significantly different from the learned settings. Therefore a novel approach called generalization in practice (GiP) is developed to tackle this critical problem. In this approach, the robot practices in the new setting to obtain new training data and refine the learned skill using the new data to gradually improve the learned skill model. The proposed approach has been implemented for one type of manipulation task – pouring that is the most performed manipulation in cooking applications. The presented approach enables a pouring robot to pour gracefully like a person in terms of speed and accuracy in learned setups and gradually improve the pouring performance in novel setups after several practices.

Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either only have a flat sensing surface or a compliant tip with a limited sensing area. In this paper, we propose a novel optical tactile sensor, the GelTip, that is shaped as a finger and can sense contacts on any location of its surface. The sensor captures high-resolution and color-invariant tactile image that can be exploited to extract detailed information about the end-effector’s interactions against manipulated objects. Our extensive experiments show that the GelTip sensor can effectively localise the contacts on different locations its finger-shaped body, with a small localisation error of approximately 5 mm, on average, and under 1 mm in the best cases. The obtained results show the potential of the GelTip sensor in facilitating dynamic manipulation tasks with its all-round tactile sensing capability. The sensor models and further information about the GelTip sensor can be found at http://danfergo.github.io/geltip.

We study human perception of gaze rendered by popular semi-virtual robotic heads, which use a screen to render a robot's face. It is known that when these heads are stationary, the screen may induce the Mona Lisa gaze effect, which widens the robot's apparent cone of direct gaze. But how do people perceive gaze when the head can move as well? To study this question, we conducted a laboratory experiment that investigated human perception of robot gaze when a semi-virtual platform looked in different directions. We varied the way in which the robot conveyed gaze, using several behaviors involving 2D eye and head motion. Our results suggest that the interplay between these motions can regulate how wide users perceive the robot's cone of direct gaze. Also, our findings suggest that the location of observers can affect the perception of gaze by semi-virtual robotic heads. We discuss the implications of our findings for social interaction.

The ability to recognize previously mapped locations is an essential feature for autonomous systems. Unstructured planetary-like environments pose a major challenge to these systems due to the similarity of the terrain. As a result, the ambiguity of the visual appearance makes state-of-the-art visual place recognition approaches less effective than in urban or man-made environments. This paper presents a method to solve the loop closure problem using only spatial information. The key idea is to use a novel continuous and probabilistic representations of terrain elevation maps. Given 3D point clouds of the environment, the proposed approach exploits Gaussian Process (GP) regression with linear operators to generate continuous gradient maps of the terrain elevation information. Traditional image registration techniques are then used to search for potential matches. Loop closures are verified by leveraging both the spatial characteristic of the elevation maps (SE (2) registration) and the probabilistic nature of the GP representation. A submap-based localization and mapping framework is used to demonstrate the validity of the proposed approach. The performance of this pipeline is evaluated and benchmarked using real data from a rover that is equipped with a stereo camera and navigates in challenging, unstructured planetary-like environments in Morocco and on Mt. Etna.

Modeling the stochastic behavior of interacting agents is key for safe motion planning. In this paper, we study the interaction of risk-aware agents in a game-theoretical framework. Under the entropic risk measure, we derive an iterative algorithm for approximating the intractable feedback Nash equilibria of a risk-sensitive dynamic game. We use an iteratively linearized approximation of the system dynamics and a quadratic approximation of the cost function in solving a backward recursion for finding feedback Nash equilibria. In this respect, the algorithm shares a similar structure with DDP and iLQR methods. We conduct experiments in a set of challenging scenarios such as roundabouts. Compared to ignoring the game interaction or the risk sensitivity, we show that our risk-sensitive game-theoretic framework leads to more timeefficient, intuitive, and safe behaviors when facing underlying risks and uncertainty.

We study strategies to deploy defenders/sensors to detect intruders that approach a targeted region. This scenario is formulated as a barrier coverage, which aims to minimize the number of unseen paths. The problem becomes challenging when the number of defenders is insufficient for a full coverage, requiring us to find the most effective location to deploy them. To this end, we use ideas from game theory to account for various paths that the intruders may take. Specifically, we propose an iterative algorithm to refine the set of candidate defender formations, which uses the payoff matrix to directly evaluate the utility of different formations. Given the set of candidate formations, a mixed Nash equilibrium gives a stochastic policy to deploy the defenders. The efficacy of the proposed strategy is demonstrated by a numerical analysis that compares our method with an existing graph-theoretic method.

Adaptive assistance of gait training robots has been determined to improve gait performance through motion assistance. An important control role during walking is to avoid tripping by controlling minimum toe clearance (MTC), which is an indicator of tripping risk, to avoid its decrease among gait cycles. No conventional gait training robots can adjust assistance timing based on MTC. In this paper, we propose a system that applies force intermittently based on the MTC prediction algorithm to encourage people to avoid lowering the MTC. This prediction algorithm is based on a radial basis function network, the input data of which include the angles, angular velocities, and angular accelerations of the hip, knee, and ankle joints in the sagittal and coronal planes at toe-off. The cable-driven system that can switch between assistance and non-assistance modes applies force when the predicted MTC is lower than the mean value. Nine participants were asked to walk on a treadmill, and we tested the effect of the system. The MTC data before, during, and after the assistance phase were analyzed for 120 s. The results showed that the minimum and first quartile values of MTC could be increased after the assistance phase.

In this paper, the gain scheduling technique is applied to design a balance controller for an autonomous bicycle with an inertia wheel. Previously, two different balance controllers are needed depending on whether the bicycle is stationary or dynamic. The switch between the two different controllers may cause the instability of the autonomous bicycle. Our proposed gain scheduled controller can balance the autonomous bicycle in both stationary and dynamic cases. A physical system is built and experiments are carried out to demonstrate the effectiveness of the gain scheduled controller.