ICRA 2021 论文列表

The detection of contextual anomalies is a challenging task for surveillance since an observation can be considered anomalous or normal in a specific environmental context. An unmanned aerial vehicle (UAV) can utilize its aerial monitoring capability and employ multiple sensors to gather contextual information about the environment and perform contextual anomaly detection. In this work, we introduce a deep neural network-based method (CADNet) to find point anomalies (i.e., single instance anomalous data) and contextual anomalies (i.e., context-specific abnormality) in an environment using a UAV. The method is based on a variational autoencoder (VAE) with a context sub-network. The context sub-network extracts contextual information regarding the environment using GPS and time data, then feeds it to the VAE to predict anomalies conditioned on the context. To the best of our knowledge, our method is the first contextual anomaly detection method for UAV-assisted aerial surveillance. We evaluate our method on the AU-AIR dataset in a traffic surveillance scenario. Quantitative comparisons against several baselines demonstrate the superiority of our approach in the anomaly detection tasks. The codes and data will be available at https://bozcani.github.io/cadnet.

Safety is a critical concern when deploying reinforcement learning agents for realistic tasks. Recently, safe reinforcement learning algorithms have been developed to optimize the agent’s performance while avoiding violations of safety constraints. However, few studies have addressed the nonstationary disturbances in the environments, which may cause catastrophic outcomes. In this paper, we propose the context-aware safe reinforcement learning (CASRL) method, a metal-earning framework to realize safe adaptation in non-stationary environments. We use a probabilistic latent variable model to achieve fast inference of the posterior environment transition distribution given the context data. Safety constraints are then evaluated with uncertainty-aware trajectory sampling. Prior safety constraints are formulated with domain knowledge to improve safety during exploration. The algorithm is evaluated in realistic safety-critical environments with non-stationary disturbances. Results show that the proposed algorithm significantly outperforms existing baselines in terms of safety and robustness.

We present a bilevel, contact-implicit trajectory optimization (TO) formulation that searches for robot trajectories with learned soft contact models. On the lower-level, contact forces are solved via a quadratic program (QP) with the maximum dissipation principle (MDP), based on which the dynamics constraints are formulated in the upper-level TO problem that uses direct transcription. Our method uses a contact model for granular media that is learned from physical experiments, but is general to any contact model that is stick-slip, convex, and smooth. We employ a primal interior-point method with a pre-specified duality gap to solve the lower-level problem, which provides robust gradient information to the upper-level problem. We evaluate our method by optimizing locomotion trajectories of a quadruped robot on various granular terrains offline, and show that we can obtain long-horizon walking gaits of high qualities.

Grasping unseen objects in unconstrained, cluttered environments is an essential skill for autonomous robotic manipulation. Despite recent progress in full 6-DoF grasp learning, existing approaches often consist of complex sequential pipelines that possess several potential failure points and run-times unsuitable for closed-loop grasping. Therefore, we propose an end-to-end network that efficiently generates a distribution of 6-DoF parallel-jaw grasps directly from a depth recording of a scene. Our novel grasp representation treats 3D points of the recorded point cloud as potential grasp contacts. By rooting the full 6-DoF grasp pose and width in the observed point cloud, we can reduce the dimensionality of our grasp representation to 4-DoF which greatly facilitates the learning process. Our class-agnostic approach is trained on 17 million simulated grasps and generalizes well to real world sensor data. In a robotic grasping study of unseen objects in structured clutter we achieve over 90% success rate, cutting the failure rate in half compared to a recent state-of-the-art method. Video of the real world experiments and code are available at https://research.nvidia.com/publication/2021-03_Contact-GraspNet%3A--Efficient.

The discontinuities and multi-modality introduced by contacts make manipulation planning challenging. Many previous works avoid this problem by pre-designing a set of high-level motion primitives like grasping and pushing. However, such motion primitives are often not adequate to describe dexterous manipulation motions. In this work, we propose a method for dexterous manipulation planning at a more primitive level. The key idea is to use contact modes to guide the search in a sampling-based planning framework. Our method can automatically generate contact transitions and motion trajectories under the quasistatic assumption. In the experiments, this method sometimes generates motions that are often pre-designed as motion primitives, as well as dexterous motions that are more task-specific 1.

Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax these assumptions and propose using Domain Randomization to train a neural network to localize contacts of robot arms in motion without joint torque observations. Our method uses a novel cylindrical projection encoding of the robot arm surface, which allows the network to use convolution layers to process input features and transposed convolution layers to predict contacts. The trained network achieves a contact detection accuracy of 91.5% and a mean contact localization error of 3.0cm. We further demonstrate an application of the contact localization model in an obstacle mapping task, evaluated in both simulation and the real world.

State estimation, in particular estimation of the base position, orientation and velocity, plays a big role in the efficiency of legged robot stabilization. The estimation of the base state is particularly important because of its strong correlation with the underactuated dynamics, i.e. the evolution of center of mass and angular momentum. Yet this estimation is typically done in two phases, first estimating the base state, then reconstructing the center of mass from the robot model. The underactuated dynamics is indeed not properly observed, and any bias in the model would not be corrected from the sensors. While it has already been observed that force measurements make such a bias observable, these are often only used for a binary estimation of the contact state. In this paper, we propose to simultaneously estimate the base and the underactuation state by exploiting all measurements simultaneously. To this end, we propose several contributions to implement a complete state estimator using factor graphs. Contact forces altering the underactuated dynamics are pre-integrated using a novel adaptation of the IMU pre-integration method, which constitutes the principal contribution. IMU pre-integration is also used to estimate the positional motion of the base. Encoder measurements then participate to the estimation in two ways: by providing leg odometry displacements which contributes to the observability of IMU biases; and by relating the positional and centroidal states, thus connecting the whole graph and producing a tightly-coupled whole-body estimator. The validity of the approach is demonstrated on real data captured by the Solo12 quadruped robot.

The Sequential Linear Quadratic (SLQ) algorithm is a continuous-time version of the well-known Differential Dynamic Programming (DDP) technique with a Gauss-Newton Hessian approximation. This family of methods has gained popularity in the robotics community due to its efficiency in solving complex trajectory optimization problems. However, one major drawback of DDP-based formulations is their inability to properly incorporate path constraints. In this paper, we address this issue by devising a constrained SLQ algorithm that handles a mixture of constraints with a previously implemented projection technique and a new augmented-Lagrangian approach. By providing an appropriate multiplier update law, and by solving a single inner and outer loop iteration, we are able to retrieve suboptimal solutions at rates suitable for real-time model-predictive control applications. We particularly focus on the inequality-constrained case, where three augmented-Lagrangian penalty functions are introduced, along with their corresponding multiplier update rules. These are then benchmarked against a relaxed log-barrier formulation in a cart-pole swing up example, an obstacle-avoidance task, and an object-pushing task with a quadrupedal mobile manipulator.

This paper presents an optimal planning/guidance approach for a UAV point-to-point navigation in 2D, under wind influence, with the imposition of a general desired airspeed profile. A cost function weighting the travel time and the control effort is minimized through the Pontryagin’s Minimum Principle. An analytical expression for the optimal heading to assure minimum-time flight is also derived. A general wind case in constrained path navigation is solved through an iterative procedure, using artificial potential fields.

This paper presents a novel constrained image-based visual servoing (IBVS) approach that guarantees target features to remain within the camera field of view (FOV) for the duration of the task. Barrier function (BF) is used to transform the feature dynamics on the image plane into equivalent dynamics with full state constraints. An IBVS controller is designed for the full state constrained system that regulates the original system. A stability analysis is provided to prove the local asymptotic convergence of the state error in the sense of Lyapunov. Simulation and robot implementation results show the efficiency of the proposed method to reach the desired features from any initial configuration while the visibility constraints of the feature points are being realized.

Differential Dynamic Programming (DDP) has become a well established method for unconstrained trajectory optimization. Despite its several applications in robotics and controls, however, a widely successful constrained version of the algorithm has yet to be developed. This paper builds upon penalty methods and active-set approaches towards designing a Dynamic Programming-based methodology for constrained optimal control. Regarding the former, our derivation employs a constrained version of Bellman’s principle of optimality, by introducing a set of auxiliary slack variables in the backward pass. In parallel, we show how Augmented Lagrangian methods can be naturally incorporated within DDP, by utilizing a particular set of penalty-Lagrangian functions that preserve second-order differentiability. We demonstrate experimentally that our extensions (individually and combinations thereof) enhance significantly the convergence properties of the algorithm, and outperform previous approaches on a large number of simulated scenarios.

This work presents the Manifold Invariant Extended Kalman Filter, a novel approach for better consistency and accuracy in state estimation on manifolds. The robustness of this filter allows for techniques with high noise potential like ultra-wideband localization to be used for a wider variety of applications like autonomous metal structure inspection. The filter is derived and its performance is evaluated by testing it on two different manifolds: a cylindrical one and a bivariate b-spline representation of a real vessel surface, showing its flexibility to being used on different types of surfaces. Its comparison with a standard EKF that uses virtual, noise-free measurements as manifold constraints proves that it outperforms standard approaches in consistency and accuracy. Further, an experiment using a real magnetic crawler robot on a curved metal surface with ultra-wideband localization shows that the proposed approach is viable in the real world application of autonomous metal structure inspection.

In swarm control, many robots coordinate their actions in a distributed and decentralized way. We propose a consensus-based control barrier function (CCBF) for a swarm. CCBF restricts the states of the whole distributed system, not just those of the individual robots. The barrier function is approximated by a consensus filter. We prove that CCBF constrains the control inputs for holding the forward invariance of the safety set. Moreover, we applied CCBF to a practical problem and conducted an experiment with actual robots. The results showed that CCBF restricted the states of multiple robots to the safety set. To the best of our knowledge, this is the first CBF that can restrict the state of the whole distributed system with only local communication. CCBF has various applications such as monitoring with a swarm and maintaining the network between a swarm and a base station.

The versatile nature of agile micro aerial vehicles (MAVs) poses fundamental challenges to the design of robust state estimation in various complex environments. Achieving high-quality performance in textureless scenes is one of the missing pieces in the puzzle. Previously proposed solutions either seek a remedy with visual loop closure or leverage RF localizability with inferior accuracy. None of them support accurate MAV state estimation in textureless scenes. This paper presents RFSift, a new state estimator that conquers the textureless challenge with RF-referenced monocular vision, achieving centimeter-level accuracy in textureless scenes. Our key observation is that RF and visual measurements are tied up with pose constraints. Mapping RF to feature quality and sift well-matched ones significantly improves accuracy. RFSift consists of 1) an RF-sifting algorithm that maps 3D UWB measurements to 2D visual features for sifting the best features; 2) an RF-visual-inertial sensor fusion algorithm that enables robust state estimation by leveraging multiple sensors with complementary advantages. We implement the prototype with off-the-shelf products and conduct large-scale experiments. The results demonstrate that RFSift is robust in textureless scenes, 10x more accurate than the state-of-the-art monocular vision system. The code of RFSift is available at https://github.com/weisgroup/RFSift.

This paper proposes a method to integrate the rich semantic data-set provided by Building Information Modeling (BIM) with robotics world models, taking as use case indoor semantic localization in a large university building. We convert a subset of semantic entities with associated geometry present in BIM models and represented in the Industry Foundation Classes (IFC) data format to a robot-specific world model representation. This representation is then stored in a spatial database from which the robot can query semantic objects in its immediate surroundings. The contribution of this work is that, from this query, the robot’s feature detectors are configured and used to make explicit data associations with semantic structural objects from the BIM model that are located near the robot’s current position. A graph-based approach is then used to localize the robot, incorporating the explicit map-feature associations for localization. We show that this explainable model-based approach allows a robot equipped with a 2D LiDAR and odometry to track its pose in a large indoor environment for which a BIM model is available.

Predicting agents’ future trajectories plays a crucial role in modern AI systems, yet it is challenging due to intricate interactions exhibited in multi-agent systems, especially when it comes to collision avoidance. To address this challenge, we propose to learn congestion patterns as contextual cues explicitly and devise a novel "Sense–Learn–Reason–Predict" framework by exploiting advantages of three different doctrines of thought, which yields the following desirable benefits: (i) Representing congestion as contextual cues via latent factors subsumes the concept of social force commonly used in physics- based approaches and implicitly encodes the distance as a cost, similar to the way a planning-based method models the environment. (ii) By decomposing the learning phases into two stages, a "student" can learn contextual cues from a "teacher" while generating collision-free trajectories. To make the framework computationally tractable, we formulate it as an optimization problem and derive an upper bound by leveraging the variational parametrization. In experiments, we demonstrate that the proposed model is able to generate collision- free trajectory predictions in a synthetic dataset designed for collision avoidance evaluation and remains competitive on the commonly used NGSIM US-101 highway dataset. Source code and dataset tools can be accessed via Github.

In this paper, the configuration transformation of Wheel-Legged Robot (WLR) is studied, which can enable the robot to change its multilinks configuration on Inverted Equilibrium Manifold (IEM), while keeping balance with a small location drift on the floor. First of all, the general form of dynamics equation of planar Articulated Wheeled Inverted Pendulum (AWIP) with a wheel and n − 1 rigid links, is derived. The Partial Feedback Linearization (PFL) combined with a Sliding Mode Control (SMC) is used to design the inverse dynamics controller of AWIP, while considering full dynamics terms. The well-known WLR model is used as a simple example of AWIP to accomplish the configuration transformation task. An optimization based configuration transformation algorithm is proposed to realize a comprehensive optimization of the shortest path in joint space and the minimum location drift of WLR on the floor. Finally, the effectiveness of the proposed algorithm is demonstrated through simulation to implement the configuration transformation task.

We consider a robot tasked with observing its environment and later selectively summarizing what it saw as a vivid, structured narrative. The robot interacts with an uncertain environment, modelled as a stochastic process, and must decide what events to pay attention to (substance), and how to best make its recording (style) for later compilation of its summary. If carrying a video camera, for example, it must decide where to be, what to aim the camera at, and which stylistic selections, like the focus and level of zoom, are most suitable. This paper examines planning algorithms that help the robot predict events that (1)will likely occur; (2)would be useful in telling a tale; and (3)may be hewed to cohere stylistically. The third factor, a time-extended requirement, is entirely neglected in earlier, simpler work. With formulations based on underlying Markov Decision Processes, we compare two algorithms: a monolithic planner that jointly plans over events and style pairs and a decoupled approach that prescribes style conditioned on events. The decoupled approach is seen to be effective and much faster to compute, suggesting that computational expediency justifies the separation of substance from style. Finally, we also report on our hardware implementation.

We present an approach based on conditional generative adversarial networks (GANs) to generate grasps directly and in a feed-forward manner from a raw depth image input. Building on the recently introduced StyleGAN architecture we extend results from an earlier proof-of-concept paper [1] and demonstrate successful sim2real transfer of grasp outputs for a robot arm with a Shadow Dexterous Hand. We find that the GAN model, which was only trained on a limited set of primitive objects, was able to generalize to a range of everyday real-world objects that differed significantly from the primitive objects used in simulation training. In contrast to discriminative models, the approach learns a latent representation in the set of feasible grasps that can be used for navigation in grasp space and thus allows smooth integration with other motion planning tools.