ICRA 2022 论文列表

Recent work on using natural language to specify commands to robots has grounded that language to LTL. However, mapping natural language task specifications to LTL task specifications using language models require probability distributions over finite vocabulary. Existing state-of-the-art methods have extended this finite vocabulary to include unseen terms from the input sequence to improve output generalization. However, novel out-of-vocabulary atomic propositions cannot be generated using these methods. To overcome this, we introduce an intermediate contextual query representation which can be learned from single positive task specification examples, associating a contextual query with an LTL template. We demonstrate that this intermediate representation allows for generalization over unseen object references, assuming accurate groundings are available. We compare our method of mapping natural language task specifications to intermediate contextual queries against state-of-the-art CopyNet models capable of translating natural language to LTL, by evaluating whether correct LTL for manipulation and navigation task specifications can be output, and show that our method outperforms the CopyNet model on unseen object references. We demonstrate that the grounded LTL our method outputs can be used for planning in a simulated OO-MDP environment. Finally, we discuss some common failure modes encountered when translating natural language task specifications to grounded LTL.

Reorientation (turning in plane) plays a critical role for all robots in any field application, especially those that in confined spaces. While important, reorientation remains a relatively unstudied problem for robots, including limbless mechanisms, often called snake robots. Instead of looking at snakes, we take inspiration from observations of the turning behavior of tiny nematode worms C. elegans. Our previous work presented an in-place and in-plane turning gait for limbless robots, called an omega turn, and prescribed it using a novel two-wave template [1]. In this work, we advance omega turn-inspired controllers in three aspects: 1) we use geometric methods to vary joint angle amplitudes and forward wave spatial frequency in our turning equation to establish a wide and precise amplitude modulation and frequency modulation on omega turn; 2) we use this new relationship to enable robots with fewer internal degrees of freedom (i.e., fewer joints in the body) to achieve desirable performance, and 3) we apply compliant control methods to this relationship to handle unmodelled effects in the environment. We experimentally validate our approach on a limbless robot that the omega turn can produce effective and robust turning motion in various types of environments, such as granular media and rock pile.

This paper presents recent advances to the Affordance Template (AT) task description language. Affordance Templates provide standardized, easy-to-use tools for defining robot manipulation tasks that provide a high level of augmented reality capabilities to facilitate human-in-the-loop operation, but can also be used to support robot autonomy when coupled with various planning tools. While initially defined in terms of end effector waypoint sequences for bimanual robots, such as the NASA Valkyrie and Robonaut 2, this paper extends the original specification to support integrated mobile manipulation, object-centric template definitions, autonomous grasp determination, and integration with custom and off-the-shelf collision free motion planners. ATs have proved highly adaptable to new robots and new domains by both the authors and third-party groups, and have been used in numerous contexts for NASA, DOD, and industry. As such, we believe that the AT framework provides a strong foundation for robot development in multiple real-world contexts that can be increasingly built upon and expanded to meet the challenges of many new applications.

This paper presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets(PSs) are aligned and that the anisotropic positional noise is considered. Our contributions in this paper are three folds. First, the problem of rigidly aligning two general point sets (PSs) with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm's iterations are given in closed-form or iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD.

We study the problem of generalizable task learning from human demonstration videos without extra training on the robot or pre-recorded robot motions. Given a set of human demonstration videos showing a task with different objects/tools (categorical objects), we aim to learn a representation of visual observation that generalizes to categorical objects and enables efficient controller design. We propose to introduce a geometric task structure to the representation learning problem that geometrically encodes the task specification from human demonstration videos, and that enables generalization by building task specification correspondence between categorical objects. Specifically, we propose CoVGS-IL, which uses a graph-structured task function to learn task representations under structural constraints. Our method enables task generalization by selecting geometric features from different objects whose inner connection relationships define the same task in geometric constraints. The learned task representation is then transferred to a robot controller using uncalibrated visual servoing (UVS); thus, the need for extra robot training or pre-recorded robot motions is removed.

This work presents a new version of tactile-sensing finger, GelSlim 3.0, which integrates the ability to sense high-resolution shape, force, and slip in a more compact form factor than previous implementations, designed for cluttered bin-picking scenarios. The novel design integrates real-time model-based algorithms to measure shape, estimate the 3-D contact force distribution, and detect incipient slip. The constraints imposed by the photometric stereo algorithm used for depth reconstruction and the implementation of a planar sensing surface make the miniaturization of previous designs nontrivial. To achieve a compact integration, we optimize the optical path from illumination source to camera. Using an optical simulation environment, we develop an illumination shaping lens and position the source LEDs and camera. The optimized optical configuration is integrated into a finger design composed of a robust and easily replaceable snap-to-fit fingertip module that facilitates manufacture, assembly, use, and repair. To stimulate future research in tactile-sensing and provide the robotics community access to a reliable and easily reproducible tactile finger with a diversity of sensing modalities, we open-source the design, fabrication methods, and software at https://github.com/mcubelab/gelslim.

The environments of such large industrial machines as waste cranes in waste incineration plants are often weakly observable, where little information about the environ-mental state is contained in the observations due to technical difficulty or maintenance cost (e.g., no sensors for observing the state of the garbage to be handled). Based on the findings that skilled operators in such environments choose predetermined control strategies (e.g., grasping and scattering) and their durations based on sensor values, we propose a novel non-parametric policy search algorithm: Gaussian process self-triggered policy search (GPSTPS). GPSTPS has two types of control policies: action and duration. A gating mechanism either maintains the action selected by the action policy for the duration specified by the duration policy or updates the action and duration by passing new observations to the policy; therefore, it is categorized as self-triggered. GPSTPS simultaneously learns both policies by trial and error based on sparse GP priors and variational learning to maximize the return. To verify the performance of our proposed method, we conducted experiments on garbage-grasping-scattering task for a waste crane with weak observations using a simulation and a robotic waste crane system. As experimental results, the proposed method acquired suitable policies to determine the action and duration based on the garbage's characteristics.

In this paper, we address the problem of sampling-based motion planning under motion and measurement un-certainty with probabilistic guarantees. We generalize traditional sampling-based, tree-based motion planning algorithms for deterministic systems and propose belief-A, a framework that extends any kinodynamical tree-based planner to the belief space for linear (or linearizable) systems. We introduce appropriate sampling techniques and distance metrics for the belief space that preserve the probabilistic completeness and asymptotic optimality properties of the underlying planner. We demonstrate the efficacy of our approach for finding safe low-cost paths efficiently and asymptotically optimally in simulation, for both holonomic and non-holonomic systems.

We present a novel approach for risk-aware planning with human agents in multi-agent traffic scenarios. Our approach takes into account the wide range of human driver behaviors on the road, from aggressive maneuvers like speeding and overtaking, to conservative traits like driving slowly and conforming to the right-most lane. In our approach, we learn a mapping from a data-driven human driver behavior model called the CMetric to a driver's entropic risk preference. We then use the derived risk preference within a game-theoretic risk-sensitive planner to model risk-aware interactions among human drivers and an autonomous vehicle in various traffic scenarios. We demonstrate our method in a merging scenario, where our results show that the final trajectories obtained from the risk-aware planner generate desirable emergent behaviors. Particularly, our planner recognizes aggressive human drivers and yields to them while maintaining a greater distance from them. In a user study, participants were able to distinguish between aggressive and conservative simulated drivers based on trajectories generated from our risk-sensitive planner. We also observe that aggressive human driving results in more frequent lane-changing in the planner. Finally, we compare the performance of our modified risk-aware planner with existing methods and show that modeling human driver behavior leads to safer navigation.

We present GTGraffiti, a graffiti painting system from Georgia Tech that tackles challenges in art, hardware, and human-robot collaboration. The problem of painting graffiti in a human style is particularly challenging and requires a system-level approach because the robotics and art must be designed around each other. The robot must be highly dynamic over a large workspace while the artist must work within the robot's limitations. Our approach consists of three stages: artwork capture, robot hardware, and planning & control. We use motion capture to capture collaborator painting motions which are then composed and processed into a time-varying linear feedback controller for a cable-driven parallel robot (CDPR) to execute. In this work, we will describe the capturing process, the design and construction of a purpose-built CDPR, and the software for turning an artist's vision into control commands. Our work represents an important step towards faithfully recreating human graffiti artwork by demonstrating that we can reproduce artist motions up to 2m/s and 20m/s2 within 9.3mm RMSE to paint artworks.

We introduce GRiD: a GPU-accelerated library for computing rigid body dynamics with analytical gradients. GRiD was designed to accelerate the nonlinear trajectory opti-mization subproblem used in state-of-the-art robotic planning, control, and machine learning, which requires tens to hundreds of naturally parallel computations of rigid body dynamics and their gradients at each iteration. GRiD leverages URDF parsing and code generation to deliver optimized dynamics kernels that not only expose GPU-friendly computational patterns, but also take advantage of both fine-grained parallelism within each computation and coarse-grained parallelism between computations. Through this approach, when performing multiple computations of rigid body dynamics algorithms, GRiD provides as much as a 7.2x speedup over a state-of-the-art, multi-threaded CPU implementation, and maintains as much as a 2.5x speedup when accounting for I/O overhead. We release GRiD as an open-source library for use by the wider robotics community.

The ability of Gaussian processes (GPs) to predict the behavior of dynamical systems as a more sample-efficient alternative to parametric models seems promising for real-world robotics research. However, the computational complexity of GPs has made policy search a highly time and memory consuming process that has not been able to scale to larger problems. In this work, we develop a policy optimization method by leveraging fast predictive sampling methods to process batches of trajectories in every forward pass, and compute gradient updates over policy parameters by automatic differentiation of Monte Carlo evaluations, all on GPU. We demonstrate the effectiveness of our approach in training policies on a set of reference-tracking control experiments with a heavy-duty machine. Benchmark results show a significant speedup over exact methods and showcase the scalability of our method to larger policy networks, longer horizons, and up to thousands of trajectories with a sublinear drop in speed.

Robotic systems such as unmanned ground vehicles (UGVs) often depend on GPS for navigation in outdoor environments. In GPS-denied environments, one approach to maintain a global state estimate is localizing based on preexisting georeferenced aerial or satellite imagery. However, this is inherently challenged by the significantly differing perspectives between the UGV and reference images. In this paper, we introduce a system for global localization of UGVs in remote, natural environments. We use multi-stereo visual inertial odometry (MSVIO) to provide local tracking. To overcome the challenge of differing viewpoints we use a probabilistic occupancy model to generate synthetic orthographic images from color images taken by the UGV. We then derive global information by scan matching local images to existing reference imagery and then use a pose graph to fuse the measurements to provide uninterrupted global positioning after loss of GPS signal. We show that our system generates visually accurate orthographic images of the environment, provides reliable global measurements, and maintains an accurate global state estimate in GPS-denied conditions.

High-speed motions in pick-and-place operations are critical to making robots cost-effective in many automation scenarios, from warehouses and manufacturing to hospitals and homes. However, motions can be too fast-such as when the object being transported has an open-top, is fragile, or both. One way to avoid spills or damage, is to move the arm slowly. We propose an alternative: Grasp-Optimized Motion Planning for Fast Inertial Transport (GOMP-FIT), a time-optimizing motion planner based on our prior work, that includes con-straints based on accelerations at the robot end-effector. With GOMP-FIT, a robot can perform high-speed motions that avoid obstacles and use inertial forces to its advantage. In experiments transporting open-top containers with varying tilt tolerances, whereas GOMP computes sub-second motions that spill up to 90 % of the contents during transport, GOMP-FIT generates motions that spill 0 % of contents while being slowed by as little as 0 % when there are few obstacles, 30 % when there are high obstacles and 45-degree tolerances, and 50 % when there 15-degree tolerances and few obstacles. Videos and more at: https://berkeleyautomation.github.io/gomp-fit/.

In this paper, we propose GOHOME, a method leveraging graph representations of the High Definition Map and sparse projections to generate a heatmap output representing the future position probability distribution for a given agent in a traffic scene. This heatmap output yields an unconstrained 2D grid representation of agent future possible locations, allowing inherent multimodality and a measure of the uncertainty of the prediction. Our graph-oriented model avoids the high computation burden of representing the surrounding context as squared images and processing it with classical CNNs, but focuses instead only on the most probable lanes where the agent could end up in the immediate future. GOHOME reaches 2nd on Argoverse Motion Forecasting Benchmark on the Misskate6metric while achieving significant speed-up and memory burden diminution compared to Argoverse 1st place method HOME. We also highlight that heatmap output enables multimodal ensembling and improve 1st place MissRate6by more than 15% with our best ensemble on Argoverse. Finally, we evaluate and reach state-of-the-art performance on the other trajectory prediction datasets nuScenes and Interaction, demonstrating the generalizability of our method.

LiDAR is widely adopted in Simultaneous Localization And Mapping (SLAM) and High Definition (HD) map production. The accuracy of LiDAR Odometry (LO) is of great importance, especially in GPS-denied environments. However, we found typical LO results are prone to drift upwards along the vertical direction in underground parking lots, leading to poor mapping results. This paper proposes a Ground Constrained LO method named GCLO, which exploits planar grounds in these specific environments to compress the vertical pose drifts. GCLO is divided into three parts. First, a sensor-centric sliding map is maintained, and the point-to-plane ICP method is implemented to perform the scan-to-map registration. Then, at each key-frame, the sliding map is recorded as a local map. Ground points nearby are segmented and modeled as a planar landmark in the form of Closest Point (CP) parameterization. Finally, planar ground landmarks observed at different key-frames are associated. The ground landmark observation constraints are fused into the pose graph optimization framework to improve the LO performance. Experimental results in HIK and KITTI datasets demonstrate GCLO's superior performances in terms of accuracy in indoor multi-floor parking lots and flat outdoor sites. The limitation of GCLO in adaptability for other environments is also discussed.

In this paper, we propose a novel network, Fusion-Net, which can estimate the extrinsic calibration matrix between LiDAR and a monocular RGB camera with high accuracy and robustness. FusionNet is a coarse-to-fine method, providing an online and end-to-end solution that can automatically detect and correct the decalibration without any specially designed targets or environments. First, the network applies deep-learning-based technologies to extract the features of LiDAR point clouds and RGB images. Then a novel method is adopted to fuse the features got from different sensors by projecting LiDAR features onto RGB feature maps, searching for the RGB features with the projected points as centers and concatenating the extracted RGB features with LiDAR features. To increase the accuracy, we apply a coarse-to-fine method in the network, by transforming LiDAR points and estimating the extrinsic calibration matrices from the coarse scale to the fine scale. The network is trained on random artificial decalibration matrices. Compared to existing approaches, our method doesn't need to train additional iterative networks, but it can also adapt to different ranges of decalibration.

Standard frame-based cameras that sample light intensity frames are heavily impacted by motion blur for high-speed motion and fail to perceive scene accurately in high-dynamic range environments. Event-based cameras, on the other hand, overcome these limitations by asynchronously detecting the variation in individual pixel intensities. However, event cameras only capture pixels in motion, leading to sparse information. Hence, estimating the overall dense behavior of pixels is difficult. To address aforementioned issues associated with both sensors, we present Fusion-FlowNet, a sensor fusion framework for energy -efficient optical flow estimation. Fusion-FlowNet utilizes both frame- and event-based sensors, leveraging their complementary characteristics. Our proposed network architecture is also a fusion of Spiking Neural Net-works (SNNs) and Analog Neural Networks (ANNs) where each network is designed to simultaneously process asynchronous event streams and regular frame-based images, respectively. We perform end-to-end training using unsupervised learning to avoid expensive video annotations. Our method generalizes well across distinct environments (rapid motion and challenging lighting conditions) and demonstrates state-of-the-art optical flow prediction on the Multi-Vehicle Stereo Event Camera (MVSEC) dataset. Furthermore, the usage of SNNs in our architecture offers substantial savings in terms of the number of network parameters and computational energy cost.

The ability to detect objects, under image corruptions and different weather conditions is vital for deep learning models especially when applied to real-world applications such as autonomous driving. Traditional RGB-based detection fails under these conditions and it is thus important to design a sensor suite that is redundant to failures of the primary frame-based detection. Event-based cameras can complement frame-based cameras in low-light conditions and high dynamic range scenarios that an autonomous vehicle can encounter during navigation. Accordingly, we propose a redundant sensor fusion model of event-based and frame-based cameras that is robust to common image corruptions. The method utilizes a voxel grid representation for events as input and proposes a two-parallel feature extractor network for frames and events. Our sensor fusion approach is more robust to corruptions by over 30% compared to only frame-based detections and outperforms the only event-based detection. The model is trained and evaluated on the publicly released DSEC dataset.