ICRA 2023 论文列表

In this paper, we address the problem of online quadrotor whole-body motion planning (SE(3) planning) in unknown and unstructured environments. We propose a novel multi-resolution search method, which discovers narrow areas requiring full pose planning and normal areas requiring only position planning. As a consequence, a quadrotor planning problem is decomposed into several SE(3) (if necessary) and R3 sub-problems. To fly through the discovered narrow areas, a carefully designed corridor generation strategy for narrow areas is proposed, which significantly increases the planning success rate. The overall problem decomposition and hierarchical planning framework substantially accelerate the planning process, making it possible to work online with fully onboard sensing and computation in unknown environments. Extensive simulation benchmark comparisons show that the proposed method is one to several orders of magnitude faster than the state-of-the-art methods in computation time while maintaining high planning success rate. The proposed method is finally integrated into a LiDAR-based autonomous quadrotor, and various real-world experiments in unknown and unstructured environments are conducted to demonstrate the outstanding performance of the proposed method.

Visual SLAM (Simultaneous Localisation and Mapping) aims to simultaneously estimate camera poses and depth maps from navigation videos captured. While recent deep learning based methods have achieved great success on this task, they tend to work well on source domain data and suffer from performance degradation on the unseen data of target domain. Hence, we propose an online adaptation approach to continuously adapt a pre-trained visual SLAM model to changing environments in a self-supervised manner. To preserve pre-learned knowledge against catastrophic forgetting, we perform updating on a novel adapter proposed rather than fine-tuning the whole model for adaptation. The adapter includes a cross-domain feature translation module that translates pre-learned features into translated features suitable for adaptation. Ideally, the translated new features should not only contain pre-learned knowledge but also substantially distinct from pre-learned features since these two features represent different domains. We thus introduce cycle-consistent contrastive learning to maximize the dissimilarity between these two features by enlarging the distance between them in the feature space. Besides, our contrastive learning method exploiting cycle-consistency contraint enables the translated features to be transferred back to the pre-learned ones, which helps the translated features better preserve pre-learned knowledge. Comprehensive experiments on both synthetic and real-world datasets demonstrate superior adaptation performance of our proposed method over several state-of-the-art baselines.

Robots such as autonomous vehicles and assistive manipulators are increasingly operating in dynamic environ-ments and close physical proximity to people. In such scenarios, the robot can leverage a human motion predictor to predict their future states and plan safe and efficient trajectories. However, no model is ever perfect - when the observed human behavior deviates from the model predictions, the robot might plan unsafe maneuvers. Recent works have explored maintaining a confidence parameter in the human model to overcome this challenge, wherein the predicted human actions are tempered online based on the likelihood of the observed human action under the prediction model. This has opened up a new research challenge, i.e., how to compute the future human states online as the confidence parameter changes? In this work, we propose a Hamilton-Jacobi (HJ) reachability-based approach to overcome this challenge. Treating the confidence parameter as a virtual state in the system, we compute a parameter-conditioned forward reachable tube (FRT) that provides the future human states as a function of the confidence parameter. Online, as the confidence parameter changes, we can simply query the corresponding FRT, and use it to update the robot plan. Computing parameter-conditioned FRT corre-sponds to an (offline) high-dimensional reachability problem, which we solve by leveraging recent advances in data-driven reachability analysis. Overall, our framework enables online maintenance and updates of safety assurances in human-robot interaction scenarios, even when the human prediction model is incorrect. We demonstrate our approach in several safety-critical autonomous driving scenarios, involving a state-of-the-art deep learning-based prediction model.

Deep learning-based grasp prediction models have become an industry standard for robotic bin-picking systems. To maximize pick success, production environments are often equipped with several end-effector tools that can be swapped on-the-fly, based on the target object. Tool-change, however, takes time. Choosing the order of grasps to perform, and corresponding tool-change actions, can improve system throughput; this is the topic of our work. The main challenge in planning tool change is uncertainty - we typically cannot see objects in the bin that are currently occluded. Inspired by queuing and admission control problems, we model the problem as a Markov Decision Process (MDP), where the goal is to maximize expected throughput, and we pursue an approximate solution based on model predictive control, where at each time step we plan based only on the currently visible objects. Special to our method is the idea of void zones, which are geometrical boundaries in which an unknown object will be present, and therefore cannot be accounted for during planning. Our planning problem can be solved using integer linear programming (ILP). However, we find that an approximate solution based on sparse tree search yields near optimal performance at a fraction of the time. Another question that we explore is how to measure the performance of tool-change planning: we find that throughput alone can fail to capture delicate and smooth behavior, and propose a principled alternative. Finally, we demonstrate our algorithms on both synthetic and real world bin picking tasks.

New robotics applications require robots to complete tasks in social spaces (i.e. environments shared with people), thus arising the necessity of enabling robots to operate in a socially acceptable manner. Some social spaces tend to be large and crowded (e.g. museums, shopping malls), which require robots to move around while showing appropriate social behaviors (e.g. not interfering with human's comfortable areas). Moving under such conditions is generally called social robot navigation, and there are different approaches to do so. Nonetheless, current approaches are mostly limited to navigate large and outdoor spaces, where both robots and people can easily avoid each other. Other approaches have been tested in indoor environments, however, the test environments tend to be small and largely empty. In this paper, we present an online social robot navigation framework, which allow robots to navigate indoor, large and crowded environments, while showing social behaviors. Our framework consists of 3 modules: 1) world modeling that incorporates a novel Social Heatmap (SH) to represent crowded areas, 2) multilayered path planning that uses sampling-based approaches, and 3) path following control. We extensively benchmark our approach against state-of-the-art approaches in challenging simulated scenarios, and ww e also demonstrate its feasibility with the Pepper robot in real-world trials.

Safety is essential for deploying Deep Reinforcement Learning (DRL) algorithms in real-world scenarios. Recently, verification approaches have been proposed to allow quantifying the number of violations of a DRL policy over input-output relationships, called properties. However, such properties are hard-coded and require task-level knowledge, making their application intractable in challenging safety-critical tasks. To this end, we introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time. CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties. Hence, we propose a refinement strategy to combine properties that model similar unsafe interactions. Our evaluation compares the benefits of computing the number of violations using standard hard-coded properties and the ones generated with CROP. We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.

In this paper we consider the problem of allowing a humanoid robot that is subject to a persistent disturbance, in the form of a payload-carrying task, to follow given planned footsteps. To solve this problem, we combine an online nonlinear centroidal Model Predictive Controller - MPC with a contact stable force parametrization. The cost function of the MPC is augmented with terms handling the disturbance and regularizing the parameter. The performance of the resulting controller is validated both in simulations and on the humanoid robot iCub. Finally, the effect of using the parametrization on the computational time of the controller is briefly studied.

Vibration suppression is an important skill for future robots that will collaborate with humans in industrial settings. The vibration through physical interaction is a common problem in such settings, especially in operations involving hand-held vibrating tools. The existing human-robot collaboration (HRC) works addressing this problem mostly focus on the oscillations caused by the human operator, and suppress them by adapting the admittance parameters. This, however, usually results in stiffer robot behavior and contributes to reducing the overall performance of the task, in particular when impedance planning is a requirement. In this work, we focus on the vibration coming from external sources such as power tools and suppress it actively. We learn the vibration using the bandlimited multiple Fourier linear combiner (BMFLC) algorithm and apply it as a feedforward Cartesian force to cancel the vibration. We combine the feedforward force control with variable impedance learning and show that it improves the vibration suppression performance in simulation and real-world experiments. The feedforward approach can suppress the vibration better while keeping a more compliant set of impedance parameters, which is crucial in HRC.

Hand-eye calibration estimates the pose of a camera relative to a robot, which is a fundamental problem for visually guided robots, especially for dynamic object grasping. Most methods use 2D fiducial markers with distinctive visual features and require pre-calibration for accurate calibration, which can not work online. In this paper, we propose a novel hand-eye calibration method based on the natural 3D object, which can work online and automatically even if the object is textureless or weakly textured. We first propose a Pose Refinement Network (PR-Net) to improve the accuracy of 3D object tracking. Then we build a 3D convergence point constraint based on the multi-view information with the accurate object pose to adjust the object position. Finally, we optimize the hand-eye pose by the closed-loop constraint with the optimized object position, solving the problem that is easy to fall into a local minimum. The experiments show that the average error of our hand-eye calibration method is 1.20 degrees and 23.18 mm. The results achieve state-of-the-art by using the working object to realize the online hand-eye calibration.

Coverage Path Planning (CPP) is an essential feature of robots deployed for applications such as lawn mowing, cleaning, painting, and exploration. However, most of the state-of-the-art CPP methods are proposed for fixed-morphology robots, and the coverage performance is limited by physical constraints such as the inaccessibility of narrow spaces. Apart from area coverage, productivity depends on coverage time and energy usage. A robot capable of varying its footprint size could be a solution for improving productivity in these aspects. In addition to that, the environments, where robots are deployed for coverage, are often subjected to changes causing uncertainties. Therefore, this paper proposes an online CPP scheme for a size-variable robot to improve coverage productivity. The navigation planning of the proposed Size-Variable CPP (VSCPP) scheme has been implemented by adapting a Glasius bio-inspired neural network that guides a robot in an efficient path for coverage while coping with dynamic changes. The size variation required for a situation is determined by analyzing a set of occupancy grid maps corresponding to the size steps of the robot. According to the results, the proposed VSCPP can ascertain coverage while coping with dynamic changes in an environment. The reduction of the coverage time due to the size variability is significant compared to a robot with no VSCPP scheme.

We demonstrate how off-the-shelf single-image depth estimation methods can be augmented with guidance from optical flow to achieve consistent and accurate online depth estimation using video sequences of static scenes. While previous work has successfully leveraged the complementary nature of optical flow and depth estimation, these techniques use computationally expensive test time optimization strategies that do not generalize beyond a single video sequence and also require knowledge of the future. In contrast, we present a computationally efficient feed-forward design that runs in an online fashion by utilizing learned data priors from previously seen video sequences. To accomplish this, we propose a continuous geometric scene representation that parametrically and compositionally represents the scene as a Gaussian Mixture Model (GMM). Based on this representation, our pipeline learns to estimate consistent depths and associated camera poses from video sequences of static scenes without direct supervision. Our online method achieves state-of-the-art results compared against offline methods that require all sequence frames.

In this paper, we analyze the behavior of existing techniques and design new solutions for the problem of one-shot visual imitation. In this setting, an agent must solve a novel instance of a novel task given just a single visual demonstration. Our analysis reveals that current methods fall short because of three errors: the DAgger problem arising from purely offline training, last centimeter errors in interacting with objects, and mis-fitting to the task context rather than to the actual task. This motivates the design of our modular approach where we a) separate out task inference (what to do) from task execution (how to do it), and b) develop data augmentation and generation techniques to mitigate mis-fitting. The former allows us to leverage hand-crafted motor primitives for task execution which side-steps the DAgger problem and last centimeter errors, while the latter gets the model to focus on the task rather than the task context. Our model gets 100% and 48% success rates on two recent benchmarks, improving upon the current state-of-the-art by absolute 90% and 20% respectively.

The arising application of neural networks (NN) in robotic systems has driven the development of safety verification methods for neural network dynamical systems (NNDS). Recursive techniques for reachability analysis of dynamical systems in closed-loop with a NN controller, planner or perception can over-approximate the reachable sets of the NNDS by bounding the outputs of the NN and propagating these NN output bounds forward. However, this recursive reachability analysis may suffer from compounding errors, rapidly becoming overly conservative over a longer horizon. In this work, we prove that an alternative one-shot reachability analysis framework which directly verifies the unrolled NNDS can significantly mitigate the compounding errors, enabling the use of the rolling horizon as a design parameter for verification purposes. We characterize the performance gap between the recursive and one-shot frameworks for NNDS with general computational graphs. The applicability of one-shot analysis is demonstrated through numerical examples on a cart-pole system.

Current on-board chips usually have different computing power, which means multiple training processes are needed for adapting the same learning-based algorithm to different chips, costing huge computing resources. The situation becomes even worse for 3D perception methods with large models. Previous vision-centric 3D perception approaches are trained with regular grid-represented feature maps of fixed resolutions, which is not applicable to adapt to other grid scales, limiting wider deployment. In this paper, we leverage the Polar representation when constructing the BEV feature map from images in order to achieve the goal of training once for multiple deployments. Specifically, the feature along rays in Polar space can be easily adaptively sampled and projected to the feature in Cartesian space with arbitrary resolutions. To further improve the adaptation capability, we make multi-scale contextual information interact with each other to enhance the feature representation. Experiments on a large-scale autonomous driving dataset show that our method outperforms others as for the good property of one training for multiple deployments.

In this paper, we present a swarm of Crazyflie nano-drones. The swarm can show various collective behaviors: Flocking, gradient following, going to a chosen point, formation, and scattered search of the environment. The methodology behind the behaviors is executed entirely on-board. Crazyflies use a common radio channel to share positions with each other. If desired, an operator can use the same channel and start, end, change or guide the collective behaviors online during the flight. We use the virtual force vectors and modify the way they are combined to achieve different behaviors instead of developing unique algorithms for each. This allows us to develop more collective behavior types with less effort. In the results, we show a detailed analysis of the behaviors and assess the coordination and the safety of the agents in addition to the performance as a collective. We conclude that our swarm of 6 Crazyflies was successful in the desired behaviors.

Imagine placing an online order on your way to the grocery store, then being able to pick the collected basket upon arrival or shortly after. Likewise, imagine placing any online retail order, made ready for pickup in minutes instead of days. In order to realize such a low-latency automatic warehouse logistics system, solvers must be made to be basket-aware. That is, it is more important that the full order (the basket) is picked timely and fast, than that any single item in the order is picked quickly. Current state-of-the-art methods are not basket-aware. Nor are they optimized for a positive customer experience, that is; to prioritize customers based on queue place and the difficulty associated with picking their order. An example of the latter is that it is preferable to prioritize a customer ordering a pack of diapers over a customer shopping a larger order, but only as long as the second customer has not already been waiting for too long. In this work we formalize the problem outlined, propose a new method that significantly outperforms the state-of-the-art, and present a new realistic simulated benchmark. The proposed method is demonstrated to work in an on-line and real-time setting, and to solve the on-demand multi-agent basket picking problem for automated shopping stores under realistic conditions.

In this paper we provide a practical demonstration of how the modularity in a Behavior Tree (BT) decreases the effort in programming a robot task when compared to a Finite State Machine (FSM). In recent years the way to represent a task plan to control an autonomous agent has been shifting from the standard FSM towards BTs. Many works in the literature have highlighted and proven the benefits of such design compared to standard approaches, especially in terms of modularity, reactivity and human readability. However, these works have often failed in providing a tangible comparison in the implementation of those policies and the programming effort required to modify them. This is a relevant aspect in many robotic applications, where the design choice is dictated both by the robustness of the policy and by the time required to program it. In this work, we compare backward chained BTs with a fault-tolerant design of FSMs by evaluating the cost to modify them. We validate the analysis with a set of experiments in a simulation environment where a mobile manipulator solves an item fetching task.

We investigate the utility of employing multiple buffers in solving a class of rearrangement problems with pick- n-swap manipulation primitives. In this problem, objects stored randomly in a lattice are to be sorted using a robot arm with k 1 swap spaces or buffers, capable of holding up to kk objects on its end-effector simultaneously. On the structural side, we show that the addition of each new buffer brings diminishing returns in saving the end-effector travel distance while holding the total number of pick-n-swap operations at a minimum. This is due to an interesting recursive cycle structure in random m-permutation, where the largest cycle covers over 60% of objects. On the algorithmic side, we propose fast algorithms for 1D and 2D lattice rearrangement problems that can effectively use multiple buffers to boost solution optimality. Numerical experiments demonstrate the efficiency and scalability of our methods, as well as confirm the diminishing return structure as more buffers are employed. Introduction video: https://youtu.be/KtBxoARGaVQ

State-of-the-art legged robots are either capable of measuring torque at the output of their drive systems, or have transparent drive systems which enable the computation of joint torques from motor currents. In either case, this sensor modality is seldom used in state estimation. In this paper, we propose to use joint torque measurements to estimate the centroidal states of legged robots. To do so, we project the whole-body dynamics of a legged robot into the nullspace of the contact constraints, allowing expression of the dynamics independent of the contact forces. Using the constrained dynamics and the centroidal momentum matrix, we are able to directly relate joint torques and centroidal states dynamics. Using the resulting model as the process model of an Extended Kalman Filter (EKF), we fuse the torque measurement in the centroidal state estimation problem. Through real-world experiments on a quadruped robot executing different gaits, we demonstrate that the estimated centroidal states from our torque-based EKF drastically improve the recovery of these quantities compared to direct computation.