ICRA 2021 论文列表

The recent remarkable progress of deep reinforcement learning (DRL) stands on regularization of policy for stable and efficient learning. A popular method, named proximal policy optimization (PPO), has been introduced for this purpose. PPO clips density ratio of the latest and baseline policies with a threshold, while its minimization target is unclear. As another problem of PPO, the symmetric threshold is given numerically while the density ratio itself is in asymmetric domain, thereby causing unbalanced regularization of the policy. This paper therefore proposes a new variant of PPO by considering a regularization problem of relative Pearson (RPE) divergence, so-called PPO-RPE. This regularization yields the clear minimization target, which constrains the latest policy to the baseline one. Through its analysis, the intuitive threshold-based design consistent with the asymmetry of the threshold and the domain of density ratio can be derived. Through four benchmark tasks, PPO-RPE performed as well as or better than the conventional methods in terms of the task performance by the learned policy.

Learning how to perform precise and controlled gestures is difficult, especially when feedback about made errors is sparse. Therefore, some works try to facilitate learning by providing virtual "coaches". Most of them propose to automatically score task quality. But simply assessing quality through a score is not enough. Indeed, it is essential to provide explanations on assigned scores just like experts do when supervising trainees. However when quality assessment is done automatically, such explanations are rare and computing an automatic feedback is complex. In this work, we propose to address this problem by providing an automatic feedback based on neural network explanation. Contrary to previous state of the art methods, which are focused on neural networks explicability for classification tasks, we want to explain network decision on a regression problem (quality score prediction). Thus, we propose to use gradient-based approaches and adapt them to a regression task. Moreover, to address the problem of noise present in sensitivity maps, we propose a solution that leads to more robust gradients. To test our approach, since automatic quality assessment datasets do not contain ground truth about errors position and amplitude, a synthetic dataset representing a simple temporal task has been created, with its associated ground truth. Once the method has been validated on this synthetic dataset, we apply it on real data composed of robotic surgical gestures.

Being able to transfer existing skills to new situations is a key capability when training robots to operate in unpredictable real-world environments. A successful transfer algorithm should not only minimize the number of samples that the robot needs to collect in the new environment, but also prevent the robot from damaging itself or the surrounding environment during the transfer process. In this work, we introduce a policy transfer algorithm for adapting robot locomotion skills to novel scenarios while minimizing serious failures. Our algorithm trains two control policies in the training environment: a task policy that is optimized to complete the task of interest, and a protective policy that is dedicated to keep the robot from unsafe events (e.g. falling to the ground). To decide which policy to use during execution, we learn a safety estimator model in the training environment that estimates a continuous safety level of the robot. When used with a set of thresholds, the safety estimator becomes a classifier for switching between the protective policy and the task policy. We evaluate our approach on four simulated robot locomotion problems and show that our method can achieve successful transfer to notably different environments while taking the robot’s safety into consideration.

A remote team of robots may be teleoperated by multiple users to explore unstructured environments and to tackle unforeseen emergencies therein. During a large-scale environmental search, each user may visually observe a unique hazard endangering the remote robot connected to their local robot. Therefore, each user may want to tele-drive the remote robot team to a location different than the target locations of other users. This paper resolves the possible conflicts among the multiple user commands through a distributed clustering algorithm that allocates to each user a number of remote robots proportional to the urgency of their request. A pivotal design challenge in the teleoperation context is to ensure that the remote robots allocated to each user are topologically reachable from the user’s local robot within the induced communication subnetwork. The proposed design overcomes this challenge through a reachability-constrained integer linear program that modulates the interconnections of the remote robots on the fly. A comparative experiment on a platform with 2 local and 12 remote robots validates the practical efficacy of the proposed clustering algorithm.

Fused deposition modeling (FDM) using mobile robots instead of the gantry-based 3D printer enables additive manufacturing at a larger scale with higher speed. This introduces challenges including accurate localization, control of the printhead, and design of a stable mobile manipulator with low vibrations and proper degrees of freedom. We proposed and developed a low-cost non-holonomic mobile 3D printing system guided by a projector via learning-based visual servoing. It requires almost no manual calibration of the system parameters. Using a regular top-down projector without any expensive external localization device for pose feedback, this system enabled mobile robots to accurately follow pre-designed millimeter-level printing trajectories with speed control. We evaluate the system in terms of its trajectory accuracy and printing quality compared with original 3D designs. We further demonstrated the potential of this system using two such mobile robots to collaboratively print a 3D object with dimensions of 80 cm × 30 cm size, which exceeds the limitation of common desktop FDM 3D printers.

The application of robotic solutions to small-batch production is challenging: economical constraints tend to dramatically limit the time for setting up new batches. Organizing robot tasks into modular software components, called skills, and allowing the assignment of multiple concurrent tasks to a single robot is potentially game-changing. However, due to cycle time constraints, it may be necessary for a skill to take over without waiting on another to terminate, and the available literature lacks a systematic approach in this case. In the present article, we fill the gap by (a) establishing the specifications of skills that can be sequenced with partial executions, (b) proposing an implementation based on the combination of finite-state machines and behavior trees, and (c) demonstrating the benefits of such skills through extensive trials in the environment of ARIAC (Agile Robotics for Industrial Automation Competition).

For autonomous driving in urban environments it is usually assumed that the road is flat. To drive off-road, however, we need a more sophisticated model of the ground surface. While previous work is mapping the terrain along with static obstacles, we propose to separate the tasks and introduce a new approach to probabilistic terrain estimation. It combines recursive Gaussian state estimation with a subsequent maximum a posteriori estimation. This allows us to efficiently accumulate obtained measurements and at the same time get a probabilistic terrain estimate based on a geometric terrain model. This way, also (measurement) uncertainties as well as inter- and extrapolation to unobserved areas are handled stochastically correct. We demonstrate the effectiveness and real-time capability of our approach using real-world data.

Estimating position and orientation change of a mobile platform from two consecutive point clouds provided by a high-resolution sensor is a key problem in autonomous navigation. In particular, scan matching algorithms aim to find the translation and rotation of the platform such that the two point clouds coincide. The association of measurements in point cloud one with measurements in point cloud two is a problem inherent to scan matching. Existing methods perform non-probabilistic data association, i.e., they assume a single association hypothesis. This leads to overconfident pose estimates and reduced estimation accuracy in ambiguous environments. Our probabilistic scan matching approach addresses this issue by considering all association hypotheses with their respective likelihoods. We formulate a holistic Bayesian estimation problem for both data association and pose inference and present the corresponding joint factor graph. Near-optimum maximum a posteriori (MAP) estimates of the sensor pose are computed by performing iterative message passing on the factor graph. Our numerical study shows performance improvements compared to non-probabilistic scan matching methods that are based on the normal distributions transform (NDT) and implicit moving least squares (IMLS).

Autonomous vehicles face tremendous challenges while interacting with human drivers in different kinds of scenarios. Developing control methods with safety guarantees while performing interactions with uncertainty is an ongoing research goal. In this paper, we present a real-time safe control framework using bi-level optimization with Control Barrier Function (CBF) that enables an autonomous ego vehicle to interact with human-driven cars in ramp merging scenarios with a consistent safety guarantee. In order to explicitly address motion uncertainty, we propose a novel extension of control barrier functions to a probabilistic setting with provable chance-constrained safety and analyze the feasibility of our control design. The formulated bi-level optimization framework entails first choosing the ego vehicle's optimal driving style in terms of safety and primary objective, and then minimally modifying a nominal controller in the context of quadratic programming subject to the probabilistic safety constraints. This allows for adaptation to different driving strategies with a formally provable feasibility guarantee for the ego vehicle's safe controller. Experimental results are provided to demonstrate the effectiveness of our proposed approach.

Human motion prediction is an important and challenging topic that has promising prospects in efficient and safe human-robot-interaction systems. Currently, the majority of the human motion prediction algorithms are based on deterministic models, which may lead to risky decisions for robots. To solve this problem, we propose a probabilistic model for human motion prediction in this paper. The key idea of our approach is to extend the conventional deterministic motion prediction neural network to a Bayesian one. On one hand, our model could generate several future motions when given an observed motion sequence. On the other hand, by calculating the Epistemic Uncertainty and the Heteroscedastic Aleatoric Uncertainty, our model could tell the robot if the observation has been seen before and also give the optimal result among all possible predictions. We extensively validate our approach on a large scale benchmark dataset Human3.6m. The experiments show that our approach performs better than deterministic methods. We further evaluate our approach in a Human-Robot-Interaction (HRI) scenario. The experimental results show that our approach makes the interaction more efficient and safer.

In this paper, we present a novel approach that predicts spatially and temporally crowd behaviour for robotic social navigation. Integrating mobile robots into human society involves the fundamental problem of navigation in crowds. A robot should attempt to navigate in a way that is minimally invasive to the humans in its environment. However, planning in a dynamic environment is difficult as the environment must be predicted into the future. This problem has been thoroughly studied considering the behaviour of pedestrians at the level of individuals. Instead, we represent a pedestrian crowd by its macroscopic properties over space, such as density and velocity. With this spatial representation, we propose to learn a convolutional recurrent model to predict these properties into the future. The key design of a probabilistic loss function capturing the crowd's macroscopic properties empowers the spatio-temporal crowd prediction. Using a social invasiveness metric defined on these properties predicted by our convolutional recurrent model, we develop a framework that produces globally-optimal plans in expectation. Extensive results using a realistic pedestrian simulator show the validity and performance of the proposed social navigation approach.

Multi-object tracking is an important ability for an autonomous vehicle to safely navigate a traffic scene. Current state-of-the-art follows the tracking-by-detection paradigm where existing tracks are associated with detected objects through some distance metric. Key challenges to increase tracking accuracy lie in data association and track life cycle management. We propose a probabilistic, multi-modal, multiobject tracking system consisting of different trainable modules to provide robust and data-driven tracking results. First, we learn how to fuse features from 2D images and 3D LiDAR point clouds to capture the appearance and geometric information of an object. Second, we propose to learn a metric that combines the Mahalanobis and feature distances when comparing a track and a new detection in data association. And third, we propose to learn when to initialize a track from an unmatched object detection. Through extensive quantitative and qualitative results, we show that when using the same object detectors our method outperforms state-of-the-art approaches on the NuScenes and KITTI datasets.

Robotic control tasks are often first run in simulation for the purposes of verification, debugging and data augmentation. Many methods exist to specify what task a robot must complete, but few exist to specify what range of environments a user expects such tasks to be achieved in. ProbRobScene is a probabilistic specification language for describing robotic manipulation environments. Using the language, a user need only specify the relational constraints that must hold between objects in a scene. ProbRobScene then automatically generates scenes which conform to this specification. By combining aspects of probabilistic programming languages and convex geometry, we provide a method for sampling this space of possible environments efficiently. We demonstrate the usefulness of our language by using it to debug a robotic controller in a tabletop robot manipulation environment.

Proactive human-robot interaction (HRI) allows the receptionist robots to actively greet people and offer services based on vision, which has been found to improve acceptability and customer satisfaction. Existing approaches are either based on multi-stage decision processes or based on end-to-end decision models. However, the rule-based approaches require sedulous expert efforts and only handle minimal pre-defined scenarios. On the other hand, existing works with end-to-end models are limited to very general greetings or few behavior patterns (typically less than 10). To address those challenges, we propose a new end-to-end framework, the TransFormer with Visual Tokens for Human-Robot Interaction (TFVT-HRI)1. The proposed framework extracts visual tokens of relative objects from an RGB camera first. To ensure the correct interpretation of the scenario, a transformer decision model is then employed to process the visual tokens, which is augmented with the temporal and spatial information. It predicts the appropriate action to take in each scenario and identifies the right target. Our data is collected from an in-service receptionist robot in an office building, which is then annotated by experts for appropriate proactive behavior. The action set includes 1000+ diverse patterns by combining language, emoji expression, and body motions. We compare our model with other SOTA end-to-end models on both offline test sets and online user experiments in realistic office building environments to validate this framework. It is demonstrated that the decision model achieves SOTA performance in action triggering and selection, resulting in more humanness and intelligence when compared with the previous reactive reception policies.

Contact-rich manipulation tasks are commonly found in modern manufacturing settings. However, manually designing a robot controller is considered hard for traditional control methods as the controller requires an effective combination of modalities and vastly different characteristics. In this paper, we first consider incorporating operational space visual and haptic information into a reinforcement learning (RL) method to solve the target uncertainty problems in unstructured environments. Moreover, we propose a novel idea of introducing a proactive action to solve a partially observable Markov decision process (POMDP) problem. With these two ideas, our method can either adapt to reasonable variations in unstructured environments or improve the sample efficiency of policy learning. We evaluated our method on a task that involved inserting a random-access memory (RAM) using a torque-controlled robot and tested the success rates of different baselines used in the traditional methods. We proved that our method is robust and can tolerate environmental variations.

The Patrolling Problem is a crucial feature of the surveillance task in defense and other establishments. Most of the works in the literature concentrate on reducing the Idleness value at each location in the environment. However, there are often a few prioritized locations that cannot be left unvisited beyond a certain Time Period. In this paper, we study the problem of Prioritized patrolling - the task of patrolling the given environment using multiple agents while ensuring the prioritized locations are visited within the pre-specified Time Period. We present a novel algorithm, namely, Time Period Based Patrolling (TPBP) algorithm, to solve the prioritized patrolling problem. It determines a sequence of walks for each agent online that complies with the Time Period requirement of the Priority nodes while reducing the Idleness of all the other nodes. We have tested and validated the algorithm using SUMO - a realistic simulator developed for traffic management. Since the existing strategies are not designed for Prioritized Patrolling, we show through comparison that proposed algorithm is required to solve the problem.

This paper presents a framework that leverages both control theory and machine learning to obtain stable and robust bipedal locomotion without the need for manual parameter tuning. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentally — a process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization is formally combined with preference-based learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point-feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.

We propose a predictive runtime monitoring framework that forecasts the distribution of future positions of mobile robots in order to detect and avoid impending property violations such as collisions with obstacles or other agents. Our approach uses a restricted class of temporal logic formulas to represent the likely intentions of the agents along with a combination of temporal logic-based optimal cost path planning and Bayesian inference to compute the probability of these intents given the current trajectory of the robot. First, we construct a large but finite hypothesis space of possible intents represented as temporal logic formulas whose atomic propositions are derived from a detailed map of the robot’s workspace. Next, our approach uses real-time observations of the robot’s position to update a distribution over temporal logic formulae that represent its likely intent. This is performed by using a combination of optimal cost path planning and a Boltzmann noisy rationality model. In this manner, we construct a Bayesian approach to evaluating the posterior probability of various hypotheses given the observed states and actions of the robot. Finally, we predict the future position of the robot by drawing posterior predictive samples using a Monte-Carlo method. We evaluate our framework using two different trajectory datasets that contain multiple scenarios implementing various tasks. The results show that our method can predict future positions precisely and efficiently, so that the computation time for generating a prediction is a tiny fraction of the overall time horizon.

Recent work has achieved dense 3D reconstruction with wide-aperture imaging sonar using a stereo pair of orthogonally oriented sonars. This allows each sonar to observe a spatial dimension that the other is missing, without requiring any prior assumptions about scene geometry. However, this is achieved only in a small region with overlapping fields-of-view, leaving large regions of sonar image observations with an unknown elevation angle. Our work aims to achieve large-scale 3D reconstruction more efficiently using this sensor arrangement. We propose dividing the world into semantic classes to exploit the presence of repeating structures in the subsea environment. We use a Bayesian inference framework to build an understanding of each object class’s geometry when 3D information is available from the orthogonal sonar fusion system, and when the elevation angle of our returns is unknown, our framework is used to infer unknown 3D structure. We quantitatively validate our method in a simulation and use data collected from a real outdoor littoral environment to demonstrate the efficacy of our framework in the field.