ICRA 2022 论文列表

Behavior prediction models have proliferated in recent years, especially in the popular real-world robotics application of autonomous driving, where representing the distribution over possible futures of moving agents is essential for safe and comfortable motion planning. In these models, the choice of coordinate frames to represent inputs and outputs has crucial trade offs which broadly fall into one of two categories. Agent-centric models transform inputs and perform inference in agent-centric coordinates. These models are intrinsically invari-ant to translation and rotation between scene elements, are best-performing on public leaderboards, but scale quadratically with the number of agents and scene elements. Scene-centric models use a fixed coordinate system to process all agents. This gives them the advantage of sharing representations among all agents, offering efficient amortized inference computation which scales linearly with the number of agents. However, these models have to learn invariance to translation and rotation between scene elements, and typically underperform agent-centric models. In this work, we develop knowledge distillation techniques between probabilistic motion forecasting models, and apply these techniques to close the gap in performance between agent-centric and scene-centric models. This improves scene-centric model performance by 13.2% on the public Argoverse benchmark, 7.8% on Waymo Open Dataset and up to 9.4% on a large In-House dataset. These improved scene-centric models rank highly in public leaderboards and are up to 15 times more efficient than their agent-centric teacher counterparts in busy scenes.

Monocular depth estimation has been a popu-lar area of research for several years, especially since self-supervised networks have shown increasingly good results in bridging the gap with supervised and stereo methods. However, these approaches focus their interest on dense 3D reconstruction and sometimes on tiny details that are superfluous for autonomous navigation. In this paper, we propose to address this issue by estimating the navigation map under a quad tree representation. The objective is to create an adaptive depth map prediction that only extract details that are essential for the obstacle avoidance. Other 3D space which leaves large room for navigation will be provided with approximate distance. Experiment on KITTI dataset shows that our method can significantly reduce the number of output information without major loss of accuracy.

Owing to the restrictions of live experimentation, musculoskeletal simulation models play a key role in biological motor control studies and investigations. Successful results of which are then tried on live subjects to develop treatments as well as robot aided rehabilitation procedures for addressing neuromusculoskeletal anomalies ranging from limb loss, to tendinitis, from sarcopenia to brain and spinal injuries. Despite its significance, current musculoskeletal models are computationally expensive, and provide limited support for contact-rich interactions which are essential for studying motor behaviors in activities of daily living, during rehabilitation treatments, or in assistive robotic devices. To bridge this gap, this work proposes an automatic pipeline to generate physiologically accurate musculoskeletal, as well as hybrid musculoskeletal-exoskeletal models. Leveraging this pipeline we present MyoSim - a set of computationally efficient (over 2 orders of magnitude faster than state of the art) musculoskeletal models that support fully interactive contact rich simulation. We further extend MyoSim to support additional features that help simulate various real-life changes/diseases, such as muscle fatigue, and sarcopenia. To demonstrate the potential applications, several use cases, including interactive rehabilitation movements, tendon-reaffirmation, and the cosimulation with an exoskeleton, were developed and investigated for physiological correctness. Web-page: https://sites.google.com/view/myosuite

In this paper, we propose a formation control system for deforming and transporting simultaneously a de-formable object with a team of robots, modeled with double-integrator dynamics. The goal is to reach a target configuration, defined as a combination of shape, scale, orientation and position of the formation. We augment this controller with a set of control barrier functions (CBFs). The CBFs allow us to satisfy fundamental constraints for the success of the task: avoidance of agent-to-agent, agent-to-obstacle and object-to-obstacle collisions, and of excessive stretching. We test the performance of our proposal in different simulation scenarios.

Optical Coherence Tomography (OCT) is a rapidly growing and promising imaging technique, enabling non-invasive high-resolution visualization of biological tissues. Segmentation of tissue structures from OCT scans is essen-tial for disease diagnosis but remains challenging for the blurry boundaries and large volumes. Deep learning-based OCT segmentation algorithms always require large numbers of annotations for satisfying performance, which is hard to meet since manually labeling is time-consuming and labor-intensive. Therefore, we propose a novel semi-supervised OCT segmentation framework utilizing very few labeled scans, i.e., 5 samples, and abundant unlabeled data. Specifically, our framework con-sists of one shared encoder and two different decoder branches. For the two branches, we design a strong augmentation-consistent supervision module and a scaling transformation-consistent supervision module respectively to improve their generalization ability. Besides, cross consistency supervision with feature perturbations between two branches is proposed to incorporate their advantages for further regularization. With such multiple consistency supervision, we aim to enrich the diversity of unsupervised information so as to make full use of labeled and unlabeled data. Experimental results on a public retinal OCT dataset demonstrate the effectiveness of our method, achieving an average dice score of 87.25% in the case of only 5 labeled samples used. It outperforms the supervised baseline by 3.46% and the best semi-supervised model by 1.42% in our experiments.

Wearable robots are limited by their actuators performances because they must bear the weight of their own power system and energy source. This paper explores the idea of leveraging hybrid modes to meet multiple operating points with a lightweight and efficient system by using hydraulic valves to dynamically reconfigure the connections of a hydrostatic actuator. The analyzed opportunities consist in 1) switching between a highly geared power source or a fast power source, 2) dynamically connecting an energy accumulator and 3) using a locking mechanism for holding. Based on a knee exoskeleton case study analysis, results show that switching between gearing ratio can lead to a lighter and more efficient actuator. Also, results show that using an accumulator to provide a preload continuous force has great mass-saving potential, but does not reduce mass significantly if used as a power booster for short transients. Finally, using a locking valve can slightly reduce battery mass if the work cycle includes frequent stops. The operating principles of the proposed multimodal schemes are demonstrated with a one-DOF prototype.

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture [34] by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques—common in other ML domains—and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition [10] and the Waymo Open Dataset Motion Prediction Challenge [13].

The ability to track the 6D pose distribution of an object when a mobile manipulator robot is still approaching the object can enable the robot to pre-plan grasps that combine base and arm motion. However, tracking a 6D object pose distribution from a distance can be challenging due to the limited view of the robot camera. In this work, we present a framework that fuses observations from external stationary cameras with a moving robot camera and sequentially tracks it in time to enable 6D object pose distribution tracking from a distance. We model the object pose posterior as a multi-modal distribution which results in a better performance against uncertainties introduced by large camera-object distance, occlusions and object geometry. We evaluate the proposed framework on a simulated multi-view dataset using objects from the YCB data set. Results show that our framework enables accurate tracking even when the robot camera has poor visibility of the object.

It is of great challenge, though promising, to coordinate collective robots for hunting an evader in a decentralized manner purely in light of local observations. In this paper, this challenge is addressed by a novel hybrid cooperative pursuit algorithm that combines reinforcement learning with the artificial potential field method. In the proposed algorithm, decentralized deep reinforcement learning is employed to learn cooperative pursuit policies that are adaptive to dynamic environments. The artificial potential field method is integrated into the learning process as predefined rules to improve the data efficiency and generalization ability. It is shown by numerical simulations that the proposed hybrid design outperforms the pursuit policies either learned from vanilla reinforcement learning or designed by the potential field method. Furthermore, experiments are conducted by transferring the learned pursuit policies into real-world mobile robots. Experimental results demonstrate the feasibility and potential of the proposed algorithm in learning multiple cooperative pursuit strategies.

Predicting the behaviors of other agents on the road is critical for autonomous driving to ensure safety and efficiency. However, the challenging part is how to represent the social interactions between agents and output different possible trajectories with interpretability. In this paper, we introduce a neural prediction framework based on the Transformer structure to model the relationship among the interacting agents and extract the attention of the target agent on the map waypoints. Specifically, we organize the interacting agents into a graph and utilize the multi-head attention Transformer encoder to extract the relations between them. To address the multi-modality of motion prediction, we propose a multi-modal attention Transformer encoder, which modifies the multi-head attention mechanism to multi-modal attention, and each predicted trajectory is conditioned on an independent attention mode. The proposed model is validated on the Argoverse motion forecasting dataset and shows state-of-the-art prediction accuracy while maintaining a small model size and a simple training process. We also demonstrate that the multi-modal attention module can automatically identify different modes of the target agent's attention on the map, which improves the interpretability of the model.

Disaster areas involving floods and earthquakes are commonly large, with the rescue time being quite tight, suggesting multi-Unmanned Aerial Vehicles (UAV) exploration rather than employing a single UAV. For such scenarios, current UAV exploration is modeled as a Coverage Path Planning (CPP) problem to achieve full area coverage in the presence of obstacles. However, the UAV's endurance capability is limited, and the rescue time is constrained, prohibiting even multiple UAVs from completing disaster area coverage on time. Therefore, this paper defines a multi-Agent Endurance-limited CPP (MAEl-CPP) problem that is based on an a priori known heatmap of the disaster area, which affords to explore the most valuable areas under UAV limited energy constraints. Furthermore, we propose a path planning algorithm for the MAEl-CPP problem by ranking the possible disaster areas according to their importance through satellite or remote sensing aerial images and completing path planning according to this ranking. Experimental results demonstrate that the search efficiency of the proposed algorithm is 4.2 times that of the existing algorithm.

Enabling robots to solve multiple manipulation tasks has a wide range of industrial applications. While learning-based approaches enjoy flexibility and generalizability, scaling these approaches to solve such compositional tasks remains a challenge. In this work, we aim to solve multi-task learning through the lens of sequence-conditioning and weighted sampling. First, we propose a new suite of benchmark specifically aimed at compositional tasks, MultiRavens, which allows defining custom task combinations through task modules that are inspired by industrial tasks and exemplify the difficulties in vision-based learning and planning methods. Second, we propose a vision-based end-to-end system architecture, Sequence-Conditioned Transporter Networks, which augments Goal-Conditioned Transporter Networks with sequence-conditioning and weighted sampling and can efficiently learn to solve multi-task long horizon problems. Our analysis suggests that not only the new framework significantly improves pick-and-place performance on novel 10 multi-task benchmark problems, but also the multi-task learning with weighted sampling can vastly improve learning and agent performances on individual tasks.

In this paper, we propose a novel decentralized method based on deep reinforcement learning using robot-level and target-level relational graphs, to solve the problem of multi-target encirclement with collision avoidance (MECA). Specifically, the robot-level relational graphs, composed of three heterogeneous relational graphs between each robot and other robots, targets and obstacles, are modeled and learned through using graph attention networks (GATs) for extracting different spatial relational representations. Moreover, for each target within the observation of each robot, a target-level relational graph is built with GAT to construct spatial relations from the robot. Furthermore, the movement of each target is modeled by the target-level relational graph and learned through supervised learning for predicting the trajectory of the target. In addition, a knowledge-embedded compound reward function is defined to solve the multi-objective problem in MECA, and guide the policy learning for deriving the behavior of MECA. An actor-critic training algorithm based on the centralized training and decentralized execution framework is adopted to train the policy network. Simulation and real-world experiment results demonstrate the effectiveness and generalization of our method.

This paper considers a multi-robot team tasked with monitoring an environmental field of interest over long time horizons. The approach is based on a control-theoretic measure of the information collected by the robots, namely a norm of the constructability Gramian. This measure is leveraged in order to learn a distributed multi-robot control policy using the reinforcement learning paradigm. The learned policy is then combined with energy constraints using the constraint-driven control framework in order to achieve persistent environmental monitoring. The proposed approach is tested in a simulated multi-robot persistent environmental monitoring scenario where a team of robots with limited availability of energy is to be controlled in a coordinated fashion in order to estimate the concentration of a gas diffusing in the environment.

Involving human operators to support swarms of robots can be beneficial to address increasingly complex scenarios. However, the shared control between multiple operators remains a challenge, especially where communication between the operators is not available. This paper studies the problem of forming a dynamic chain of robots connecting two operators moving within an environment. The robot chain enables operators to share information and robots among themselves. Based on supervisory control theory, we propose a distributed solution which formally guarantees that the deployed robot controllers match the modeled specifications. We validate the controllers through simulations with groups of up to 40 mobile robots in an environment with obstacles, demonstrating the feasibility of the approach.

This paper proposes 12 multi-object grasps (MOGs) types from a human and robot grasping data set. The grasp types are then analyzed and organized into a MOG taxonomy. This paper first presents three MOG data collection setups: a human finger tracking setup for multi-object grasping demonstrations, a real system with Barretthand, UR5e arm, and a MOG algorithm, a simulation system with the same settings as the real system. Then the paper describes a novel stochastic grasping routine designed based on a biased random walk to explore the robotic hand's configuration space for feasible MOGs. Based on obser-vations in both the human demonstrations and robotic MOG solutions, this paper proposes 12 MOG types in two groups: shape-based types and function-based types. The new MOG types are compared using six characteristics and then compiled into a taxonomy. This paper then introduces the observed MOG type combinations and shows examples of 16 different combinations.

Proprioception and variable stiffness are two trending topics in soft robotics research. The former could endow soft robots with the ability to perceive the environment as well as their internal states without the need of dedicated sensors, while the latter could strengthen the otherwise excessive compliance, enabling soft robots for tasks which require a higher force. Both directions have been extensively reported in existing literature, achieving both concurrently was even more challenging. The major limiting factor was the limited stiffness due to the hyper elasticity of conventional soft robots, which increases the difficulties in capturing the continues deformation. In this work, we proposed an alternative approach to tackle these two challenges, a novel “tune-down” approach, combining proprioception with stiffness regulation and implemented over-constrained soft robotic joint designs to further strengthen this spirit. As a result, the soft robotic joint could achieve multi-directional proprioception, as well as variable stiffness tuning, concurrently, using merely an on-board sensor for basic pneumatic control. The concept, design, modeling, actuation/control, and experimental validation were presented in detail, demonstrating the efficacy and potential of the proposed approach.

In this paper, we discuss the role of gripper compliance in successful grasping and manipulation of thin, flexible materials. We show, both conceptually and empirically, that each axis of compliance in a planar gripper provides unique benefits in this domain. Vertical compliance allows robust grasping of thin materials in the presence of large uncertainty in positioning. Lateral compliance increases opportunity to respond to unexpected snags by increasing the time window over which tensile forces are applied. Rotational compliance avoids damage to objects by decreasing the maximum tensile forces applied during snags. We explore these three benefits through empirical tests comparing a rigid gripper to a soft gripper, evaluating the level of vertical uncertainty each can handle for prehensile and non-prehensile manipulation, as well as the forces and displacements incurred during snags. The results show how a soft gripper's three-axis compliance provides a passive ability to prevent damage to delicate materials.

While multi-class 3D detectors are needed in many robotics applications, training them with fully labeled datasets can be expensive in labeling cost. An alternative approach is to have targeted single-class labels on disjoint data samples. In this paper, we are interested in training a multi-class 3D object detection model, while using these single-class labeled data. We begin by detailing the unique stance of our “Single-Class Supervision” (SCS) setting with respect to related concepts such as partial supervision and semi supervision. Then, based on the case study of training the multi-class version of Range Sparse Net (RSN), we adapt a spectrum of algorithms - from supervised learning to pseudo-labeling - to fully exploit the properties of our SCS setting, and perform extensive ablation studies to identify the most effective algorithm and practice. Empirical experiments on the Waymo Open Dataset show that proper training under SCS can approach or match full supervision training while saving labeling costs.