ICRA 2021 论文列表

Collaborative object localization aims to collaboratively estimate locations of objects observed from multiple views or perspectives, which is a critical ability for multi-agent systems such as connected vehicles. To enable collaborative localization, several model-based state estimation and learning-based localization methods have been developed. Given their encouraging performance, model-based state estimation often lacks the ability to model the complex relationships among multiple objects, while learning-based methods are typically not able to fuse the observations from an arbitrary number of views and cannot well model uncertainty. In this paper, we introduce a novel spatiotemporal graph filter approach that integrates graph learning and model-based estimation to perform multi-view sensor fusion for collaborative object localization. Our approach models complex object relationships using a new spatiotemporal graph representation and fuses multi-view observations in a Bayesian fashion to improve location estimation under uncertainty. We evaluate our approach in the applications of connected autonomous driving and multiple pedestrian localization. Experimental results show that our approach outperforms previous techniques and achieves the state-of-the-art performance on collaborative localization.

This paper considers a multi-target coverage problem where a robot team aims to efficiently cover multi-targets while maintaining connectivity in a distributed manner. A novel knowledge-incorporated policy framework is proposed to derive a distributed, efficient, and connectivity guaranteed coverage policy. In particular, a knowledge-guided policy network (KGPnet) is designed, which consists of observation attention representation, interaction attention representation, and knowledge-guided policy learning. Giving credit to the KGPnet, the connectivity guaranteed coverage policy can be applied to different number targets. Moreover, based on the knowledge of the algebraic connectivity and coverage rate, a comprehensive reward is designed to guide the training of the behavior of multi-target coverage with connectivity maintenance. Furthermore, since the policy learned through deep reinforcement learning (DRL) can not guarantee the connectivity of the robot team, a knowledge-nested policy filtering is designed to filter dis-connectivity policies to satisfy the connectivity constraint based on the knowledge model of connectivity maintenance. Various simulations are conducted to verify the effectiveness of the proposed method. Besides, numerous real-world experiments with three-wheel omnidirectional cars and a motion capture system are presented to demonstrate the practicability of the proposed method.

Concurrent mapping necessitates data association among vehicles to overcome temporal and sensor modality differences. In this work, we focus on an underwater multi-vehicle mapping scenario in which vehicles have various sensor modalities, namely sonar and camera. This inter-session sonar-optical image matching poses two main challenges. First, ensuring covisibility for the opti-acoustic pair is complex due to their projection models and field of view (FOV) difference. Second, even with secured covisible frames, feature matching over various sensor modalities is not trivial. To overcome these challenges, we complete multi-session simultaneous localization and mapping (SLAM) by introducing an opti-acoustic pairwise factor. We alleviate the covisibility requirement by introducing inter-session measurements. We achieved opti-acoustic feature matching by applying a style-transfer and integration with SuperGlue. The proposed method is validated via simulation and real underwater tank tests.

We consider typical scenarios where an autonomous multi-robot team is used for surveying a large region. The desired output is a spatial map of the physical values of interest. Accounting for spatial correlation and uncertainty, the map is modeled using a Gaussian Process. Considering real-world constraints such as limited time budget and collision avoidance, we model team’s mission as a joint informative path planning problem which is tackled using a leader-follower architecture balancing centralized and fully distributed computation of plans. The leader first identifies a convex containment region that is to be sampled by the team. Next, through a combination of Bayesian optimization and Monte Carlo simulation, distinct sampling locations are identified and assigned to the followers. Each follower independently solves an orienteering problem to find a collision-free path maximizing information gain. A team-level adaptive replanning criterion is designed to keep redirecting sampling towards the most informative regions. The algorithm has been validated in computational experiments for map estimation. Compared to a baseline reference algorithm, it has shown a significantly higher accuracy. Moreover, the approach has shown good ability to support network connectivity, as well as good scalability in computation.

This paper presents a novel control strategy to herd a group of non-cooperative evaders by means of a team of robotic herders. In herding problems, the motion of the evaders is typically determined by strong nonlinear reactive dynamics, escaping from the herders. Many applications demand the herding of numerous and/or heterogeneous entities, making the development of flexible control solutions challenging. In this context, our main contribution is a control approach that finds suitable herding actions even when the nonlinearities in the evaders’ dynamics yield to implicit equations. We resort to numerical analysis theory to characterise the existence conditions of such actions and propose two design methods to compute them, one transforming the continuous time implicit system into an expanded explicit system, and the other applying a numerical method to find the action in discrete time. Simulations and real experiments validate the proposal in different scenarios.

Learning the uncertain dynamical environments for online learning and prediction from noisy sensory measurement streams is essential for various tasks in robotics. Recently, Gaussian process (GP) online learning such as an infinite-horizon Gaussian process (IHGP) has shown effectiveness to cope with non-stationary dynamical random processes in learning hyperparameters online by reducing the computational cost. However, the IHGP was originally proposed to deal with only a single-output. Therefore, to tackle complex real-world problems, we propose a multi-output infinite-horizon Gaussian process (MOIHGP) that generalizes the single-output IHGP to deal with multiple outputs for better prediction. Our approach allows us to consider correlations between multiple outputs for better prediction, even with occlusions in a Bayesian way. Finally, we successfully demonstrate the effectiveness of our approach by benchmark and experimental results. For simulated benchmark experiments with high noise levels, our approach reduced 16.6% of the averaged RMSE value achieved by the single-output IHGP.

Conventional multi-agent path planners typically compute an ensemble of paths while optimizing a single objective, such as path length. However, many applications may require multiple objectives, say fuel consumption and completion time, to be simultaneously optimized during planning and these criteria may not be readily compared and sometimes lie in competition with each other. Naively applying existing multi-objective search algorithms to multi-agent path finding may prove to be inefficient as the size of the space of possible solutions, i.e., the Pareto-optimal set, can grow exponentially with the number of agents (the dimension of the search space). This article presents an approach named Multi-objective Conflict-based Search (MO-CBS) that bypasses this so-called curse of dimensionality by leveraging prior Conflict-based Search (CBS), a well-known algorithm for single-objective multi-agent path finding, and principles of dominance from multi-objective optimization literature. We prove that MO-CBS is able to compute the entire Pareto-optimal set. Our results show that MO-CBS can solve problem instances with hundreds of Pareto-optimal solutions which the standard multi-objective A* algorithms could not find within a bounded time.

We present a multi-agent approach to receding horizon path planning that utilizes terminal costs. We show that the value of the receding horizon paths produced using the proposed methods have a guaranteed lower bound that can be determined using any readily-available, naive solution. We present a modified sequentially allocated optimal path planner with terminal costs that is guaranteed to satisfy the assumptions required to provide a guaranteed lower bound. We utilize a slightly modified version of the Decentralized Monte Carlo Tree Search algorithm to solve for near-optimal paths within a short planning horizon with an appended terminal cost to demonstrate the flexibility of the proposed method. We compare these receding horizon methods that incorporate a terminal cost to related receding horizon methods that do not incorporate a terminal cost. Our approach is developed specifically for multiple agents engaged in search, but can be easily adapted for other information gathering applications.

We propose a novel scheme for surveillance of a dynamic ground convoy moving along a non-linear trajectory, by aerial agents that maintain a uniformly spaced formation on a time-varying elliptical orbit encompassing the convoy. Elliptical orbits are used as they are more economical than circular orbits for circumnavigating the group of targets in the moving convoy. The proposed scheme includes an algorithm for computing feasible elliptical orbits, a vector guidance law for agent motion along the desired orbit, and a cooperative strategy to control the speeds of the aerial agents in order to quickly achieve and maintain the desired formation. It achieves mission objectives while accounting for linear and angular speed constraints on the aerial agents. The scheme is validated through simulations and actual experiments with a convoy of ground robots and a team of quadrotors as the aerial agents, in a motion capture environment.

Learning object manipulation is a critical skill for robots to interact with their environment. Even though there has been significant progress in robotic manipulation of rigid objects, interacting with non-rigid objects remains challenging for robots. In this work, we introduce velcro peeling as a new application for robotic manipulation of non-rigid objects in complex environments. We present a method of learning force-based manipulation from noisy and incomplete sensor inputs in partially observable environments by modeling long term dependencies between measurements with a multi-step deep recurrent network. We present experiments on a real robot to show the necessity of modeling these long term dependencies and validate our approach in simulation and robot experiments. Our results show that using tactile input enables the robot to overcome geometric uncertainties present in the environment with high fidelity in ∼ 90% of all cases, outperforming the baselines by a large margin.

Stereo matching is essential for robot navigation. However, the accuracy of current widely used traditional methods is low, while methods based on CNN need expensive computational cost and running time. This is because different cost volumes play a crucial role in balancing speed and accuracy. Thus we propose MSCVNet, which combines traditional methods and neural networks to improve the quality of cost volume. Concretely, our network first generates multiple 3D cost volumes with different resolutions and then uses 2D convolutions to construct a novel cascade hourglass network for cost aggregation. Meanwhile, we design an algorithm to distinguish and calculate the loss for discontinuous areas of the disparity result. According to the KITTI official website, our network is much faster than most top-performing methods (24than CSPN, 44than GANet, etc.). Meanwhile, compared to traditional methods (SPS-St, SGM) and other real-time stereo matching networks (Fast DS-CS, DispNetC, and RTSNet, etc.), our network achieves a big improvement in accuracy, demonstrating the feasibility and capability of the proposed method.

We propose a game-theoretic multi-robot task allocation framework that enables a large team of robots to optimally allocate tasks in dynamically changing environments. As our main contribution, we design a decision-making algorithm that defines how the robots select tasks to perform and how they repeatedly revise their task selections in response to changes in the environment. Our convergence analysis establishes that the algorithm enables the robots to learn and asymptotically achieve the optimal stationary task allocation. Through experiments with a multi-robot trash collection application, we assess the algorithm’s responsiveness to changing environments and resilience to failure of individual robots.

This paper studies the multi-robot motion-planning problem with unlabeled goals where n robots have to reach m goals. The proposed approach also takes into account the underlying dynamics of each robot to produce dynamically-feasible trajectories that enable the robots to reach all the goals while avoiding collisions with the obstacles and each other. The approach leverages the idea of combining sampling-based motion planning with goal assignment and multi-agent search. In fact, the goal-assignment layer seeks to effectively utilize the robots based on estimated costs to reach the remaining goals. The multi-agent search provides nonconflicting paths over roadmap graphs, which then guide the sampling-based expansion of a motion tree. The goal assignments and multi-agent paths are frequently updated based on the progress made during the motion-tree expansion. Simulation experiments using an increasing number of robots with nonlinear dynamics demonstrate the efficiency of the approach.

We study the problem of multi-robot coverage over an unknown, nonuniform sensory field. Modeling the sensory field as a realization of a Gaussian Process and using Bayesian techniques, we devise a policy which aims to balance the tradeoff between learning the sensory function and covering the environment. We propose an adaptive coverage algorithm called Deterministic Sequencing of Learning and Coverage (DSLC) that schedules learning and coverage epochs such that its emphasis gradually shifts from exploration to exploitation while never fully ceasing to learn. Using a novel definition of coverage regret which characterizes overall coverage performance of a multi-robot team over a time horizon T , we analyze DSLC to provide an upper bound on expected cumulative coverage regret. Finally, we illustrate the empirical performance of the algorithm through simulations of the coverage task over an unknown distribution of wildfires.

This paper presents an algorithmic framework for the distributed on-line source seeking, termed as DoSS, with a multi-robot system in an unknown dynamical environment. Our algorithm, building on a novel concept called dummy confidence upper bound (D-UCB), integrates both estimation of the unknown environment and task planning for the multiple robots simultaneously, and as a result, drives the team of robots to a steady state in which multiple sources of interest are located. Unlike the standard UCB algorithm in the context of multi-armed bandits, the introduction of D-UCB significantly reduces the computational complexity in solving subproblems of the multi-robot task planning. This also enables our DoSS algorithm to be implementable in a distributed on-line manner. The performance of the algorithm is theoretically guaranteed by showing a sub-linear upper bound of the cumulative regret. Numerical results on a real-world methane emission seeking problem are also provided to demonstrate the effectiveness of the proposed algorithm.

We present a solution to multi-robot distributed semantic mapping of novel and unfamiliar environments. Most state-of-the-art semantic mapping systems are based on supervised learning algorithms that cannot classify novel observations online. While unsupervised learning algorithms can invent labels for novel observations, approaches to detect when multiple robots have independently developed their own labels for the same new class are prone to erroneous or inconsistent matches. These issues worsen as the number of robots in the system increases and prevent fusing the local maps produced by each robot into a consistent global map, which is crucial for cooperative planning and joint mission summarization. Our proposed solution overcomes these obstacles by having each robot learn an unsupervised semantic scene model online and use a multiway matching algorithm to identify consistent sets of matches between learned semantic labels belonging to different robots. Compared to the state of the art, the proposed solution produces 20-60% higher quality global maps that do not degrade even as many more local maps are fused.

SLAM systems can retrieve their metric scales and depth information using RGB-D cameras. However, limited by the sensing range and objects structure, RGB-D cameras can not always work well, resulting in failures sometimes. In this work, we present initialization and localization methods based on maximum-a-posteriori estimation. Our system endows monocular keypoints with valid depth values and introduce them into bundle adjustment. Depth bias coefficient and scale factor are also optimized in the local window, obtaining robustness in large scale environments and long-running operations. The experimental results indicate that our system provides the best robustness compared with other excellent methods in the literature, being able to process the most challenging sequences in the TUM RGB-D dataset.

We present methods for co-designing rigid robots over control and morphology (including discrete topology) over multiple objectives. Previous work has addressed problems in single-objective robot co-design or multi-objective control. However, the joint multi-objective co-design problem is extremely important for generating capable, versatile, algorithmically designed robots. In this work, we present Multi-Objective Graph Heuristic Search, which extends a single-objective graph heuristic search from previous work to enable a highly efficient multi-objective search in a combinatorial design topology space. Core to this approach, we introduce a new universal, multiobjective heuristic function based on graph neural networks that is able to effectively leverage learned information between different task trade-offs. We demonstrate our approach on six combinations of seven terrestrial locomotion and design tasks, including one three-objective example. We compare the captured Pareto fronts across different methods and demonstrate that our multi-objective graph heuristic search quantitatively and qualitatively outperforms other techniques.

This work focuses on learning useful and robust deep world models using multiple, possibly unreliable, sensors. We find that current methods do not sufficiently encourage a shared representation between modalities; this can cause poor performance on downstream tasks and over-reliance on specific sensors. As a solution, we contribute a new multi-modal deep latent state-space model, trained using a mutual information lower-bound. The key innovation is a specially-designed density ratio estimator that encourages consistency between the latent codes of each modality. We tasked our method to learn policies (in a self-supervised manner) on multi-modal Natural MuJoCo benchmarks and a challenging Table Wiping task. Experiments show our method significantly outperforms state-of-the-art deep reinforcement learning methods, particularly in the presence of missing observations.