IROS 2024 论文列表

This work explores an innovative algorithm designed to enhance the mobility of underactuated bipedal robots across challenging terrains, especially when navigating through spaces with constrained opportunities for foot support, like steps or stairs. By combining ankle torque with a refined angular momentum-based linear inverted pendulum model (ALIP), our method allows variability in the robot’s center of mass height. We employ a dual-strategy controller that merges virtual constraints for precise motion regulation across essential degrees of freedom with an ALIP-centric model predictive control (MPC) framework, aimed at enforcing gait stability. The effectiveness of our feedback design is demonstrated through its application on the Cassie bipedal robot, which features 20 degrees of freedom. Key to our implementation is the development of tailored nominal trajectories and an optimized MPC that reduces the execution time to under 500 microseconds—and, hence, is compatible with Cassie’s controller update frequency. This paper not only showcases the successful hardware deployment but also demonstrates a new capability, a bipedal robot using a moving walkway.

Deformable Objects (DOs), e.g. clothes, garments, cables, wires, and ropes, are pervasive in our everyday environment. Despite their importance and widespread presence, many limitations exist when deploying robotic systems to interact with DOs. One source of challenges arises from their complex perception. Deep learning algorithms can address these issues; however, extensive training data is usually required. This paper introduces a method for efficiently labeling DOs in images at the pixel level, starting from sparse annotations of key points. The method allows for the generation of a real-world dataset of DO images for segmentation purposes with minimal human effort. The approach comprises three main steps. First, a set of images is collected by a camera-equipped robotic arm. Second, a user performs sparse annotation via key points on just one image from the collected set. Third, the initial sparse annotations are converted into dense labels ready for segmentation tasks by leveraging a foundation model in zero-shot settings. Validation of the method on three different sets of DOs, comprising cloth and rope-like objects, showcases its practicality and efficiency. Consequently, the proposed method lays the groundwork for easy DO labeling and the seamless integration of deep learning perception of DOs into robotic agents.

Large scale active exploration has recently revealed limitations of visual SLAM’s tracking ability. Active view planning methods based on reinforcement learning have been proposed to improve visual tracking robustness.In this work, we expose the limitations of deep reinforcement learning-based visual SLAM over longer routes. We demonstrate that additional modalities (depth, scene layout) offer little improvement. Furthermore, reward shaping is not the main reason behind the shortsightedness of the state-of-the-art visual SLAM tracker. We propose a novel video vision transformer-based architecture that improves the farsightedness of the visual tracker, which results in the completion of longer routes with efficient paths.Out of 60 challenging routes, our approach manages to complete 56 routes, which is a three-fold improvement over the state-of-the-art active view mapping (DI-SLAM) baseline. Interestingly, ORB-SLAM3 was unable to complete a single route without tracking failure. Our code is available at https://tinyurl.com/w935spuz.

Recently, significant progress has been achieved in sensing real large-scale outdoor 3D environments, particularly by using modern acquisition equipment such as LiDAR sensors. Unfortunately, they are fundamentally limited in their ability to produce dense, complete 3D scenes. To address this issue, recent learning-based methods integrate neural implicit representations and optimizable feature grids to approximate surfaces of 3D scenes. However, naively fitting samples along raw LiDAR rays leads to noisy 3D mapping results due to the nature of sparse, conflicting LiDAR measurements. Instead, in this work we depart from fitting LiDAR data exactly, instead letting the network optimize a non-metric monotonic implicit field defined in 3D space. To fit our field, we design a learning system integrating a monotonicity loss that enables optimizing neural monotonic fields and leverages recent progress in large-scale 3D mapping. Our algorithm achieves high-quality dense 3D mapping performance as cap¬tured by multiple quantitative and perceptual measures and visual results obtained for Mai City, Newer College, and KITTI benchmarks. The code of our approach is publicly available at https://github.com/artonson/deepmif.

Visual Odometry (VO) is crucial for autonomous robotic navigation, especially in GPS-denied environments like planetary terrains. To improve robustness, recent model-based VO systems have begun combining standard and event-based cameras. While event cameras excel in low-light and high-speed motion, standard cameras provide dense and easier-to-track features. However, the field of image- and event-based VO still predominantly relies on model-based methods and is yet to fully integrate recent image-only advancements leveraging end-to-end learning-based architectures. Seamlessly integrating the two modalities remains challenging due to their different nature, one asynchronous, the other not, limiting the potential for a more effective image- and event-based VO. We introduce RAMP-VO, the first end-to-end learned image- and event-based VO system. It leverages novel Recurrent, Asynchronous, and Massively Parallel (RAMP) encoders capable of fusing asynchronous events with image data, providing 8× faster inference and 33% more accurate predictions than existing solutions. Despite being trained only in simulation, RAMP-VO outperforms previous methods on the newly introduced Apollo and Malapert datasets, and on existing benchmarks, where it improves image- and event-based methods by 58.8% and 30.6%, paving the way for robust and asynchronous VO in space.Multimedial Material: For code and datasets visit https://github.com/uzh-rpg/rampvo.

In trajectory forecasting tasks for traffic, future output trajectories can be computed by advancing the ego vehicle’s state with predicted actions according to a kinematics model. By unrolling predicted trajectories via time integration and models of kinematic dynamics, predicted trajectories should not only be kinematically feasible but also relate uncertainty from one timestep to the next. While current works in probabilistic prediction do incorporate kinematic priors for mean trajectory prediction, variance is often left as a learnable parameter, despite uncertainty in one time step being inextricably tied to uncertainty in the previous time step. In this paper, we show simple and differentiable analytical approximations describing the relationship between variance at one timestep and that at the next with the kinematic bicycle model. In our results, we find that encoding the relationship between variance across timesteps works especially well in unoptimal settings, such as with small or noisy datasets. We observe up to a 50% performance boost in partial dataset settings and up to an 8% performance boost in large-scale learning compared to previous kinematic prediction methods on SOTA trajectory forecasting architectures out-of-the-box, with no fine-tuning.

In mobile robotics, particularly in autonomous driving, localization is one of the key challenges for navigation and planning. For safe operation in the open world where vulnerable participants are present, precise and guaranteed safe localization is required. While current classical fusion approaches are safe due to provably bounded closed-form formulation, their situation-adaptivity is limited. In contrast, data-driven approaches are situation-adaptive based on the underlying training data but unbounded and unsafe. In our work, we propose a novel data-driven but provably bounded sensor fusion and apply it to mobile robotic localization. In extensive experiments using an autonomous driving test vehicle, we show that our fusion method outperforms other safe fusion approaches.

The difficulties posed by terrain-induced slippage for wheeled rovers traversing soft terrains are critical to ensuring safe and precise mobility. While bilateral teleoperation systems offer a promising solution to this issue, the inherent network-induced delays hinder the fidelity of the closed-loop integration, potentially compromising teleoperator system controls, and resulting in poor command-tracking performance. This work introduces a new model-free predictor framework based on deep learning designed to improve prediction performance and effectively compensate for large network delays in teleoperated wheeled rovers. Our approach employs the Recurrent Neural Network (RNN) to achieve a significant improvement in modeling complexity and prediction accuracy. Particularly, our framework consists of two distinct predictors, each tailored to the forward and backward coupling variables of the teleoperated wheeled rover. Human-in-the-loop experiments were conducted to validate the effectiveness of the developed framework in compensating for the delays encountered by teleoperated wheeled rovers coupled with terrain-induced slippage. The results confirm the improved prediction accuracy of the framework. This improvement is evidenced by improved performance and transparency metrics, which lead to better command-tracking performance. A supplementary video is available at https://youtu.be/-06UGumQ0tA.

In this paper, we introduce a novel approach for designing invariant control laws through potential functions for fully actuated dynamical systems evolving on manifolds by leveraging the power of neural networks. The geometry and non-linearity inherent to manifold-based dynamical systems pose challenges for traditional control law design, necessitating techniques with the interplay of differential geometry and dynamical systems for ensuring stability. Apart from stability, performance and optimality are other challenging areas to address for dynamical systems evolving on manifolds. On top of these, the concept of invariance helps us improve learning transferability skills from one scene to another scene. We propose invariant potential functions on manifolds defined by neural networks that can be used to generate elastic forces for asymptotic tracking of trajectories. The weights of the potential function can be tuned to shape the potential functions according to the performance requirements through minimizing a loss function.

In the future, unmanned autonomous air combat will encounter large-scale confrontation scenarios, where agents must consider complex time-varying relationships among aircraft when making decisions. Previous works have already introduced Multi-Agent Reinforcement Learning (MARL) into air combat and succeeded in surpassing the human expert level. However, they mainly focus on small-scale air combat with low relationship complexity, e.g., 1-vs-1 or 2-vs-2. As more agents join the confrontation, existing algorithms tend to suffer significant performance degradation due to the increase in problem dimensions. In view of this, this paper proposes Deep Ad-hoc Sub-Team Partition Learning(DASPL) to address large-scale air combat problems. DASPL models multi-agent air combat as a graph to handle the complex relations and introduces an automatic partitioning mechanism to generate dynamic sub-teams, which converts the existing large-scale multi-agent air combat cooperation problem into multiple small-scale equivalence problems. Additionally, DASPL incorporates an efficient message passing method among the participating sub-teams.

For aerial swarms, formation flight has been applied in various scenes. However, most existing works do not consider balancing the conflicting requirements among keeping formation, keeping the smoothness of trajectories, and obstacle avoidance within the limited time. To address this issue, we propose a decentralized trajectory planning framework for formation flight in unknown and dense environments. To ensure that feasible trajectories can be found within the limited time, the formation optimization problem is decoupled into formation affine transformation and iterative trajectory generation. Firstly, the optimization problem based on affine transformation is designed to obtain the optimal affine transformation sequence, which provides the formation reference of trajectory optimization. Secondly, the iterative optimization framework of trajectory planning is designed, which balances the conflicting requirements of formation, smooth flight, and obstacle avoidance. Besides, to escape the local minima caused by non-convex dense environments, the method of topological path planning is designed to provide distinctive initial solutions for trajectory optimization. Finally, the proposed methods are proven to be effective through the simulations and real-world experiments.

Navigating robots through dynamic multi-robot environments, avoiding collisions with both other robots and obstacles, has emerged as a central challenge in robotics. The existing approaches fall short in allowing the policy network to effectively capture spatial-temporal reciprocal collision avoidance in multi-robot environments, comprising both static and dynamic obstacles, resulting in inadequate safety and efficiency in directing robot movement. In this study, we introduce a novel policy neural network called Spatial-Temporal RetNet (STR), designed to encode reciprocal collision avoidance states between robots in spatial and temporal dimensions. The goal is to improve the safety and efficacy of the policy neural network in directing robots to complete assigned tasks. The spatial state encoder module is built upon a parallel RetNet structure, which strengthens the neural network's capacity in extracting reciprocal collision avoidance states between robots in spatial dimensions. This module addresses the limitations of position encoding in transformer-based multi-robot navigation policy neural networks. We design a temporal state encoder utilizing a recurrent RetNet structure. This innovation bolsters the multi-robot navigation policy neural network's capability to capture features in the temporal dimension of multi-robot movements. It addresses the limitations of transformer-based multi-robot navigation policy neural networks, particularly in recurrently inferring information across time dimensions. Simulation experiments were conducted to showcase the superior safety and effectiveness of our proposed method compared to previous state-of-the-art approaches in guiding robots to accomplish tasks.

This work introduces a novel decentralized algorithm and control law for stable linear convoying using a layered control approach. This algorithm was implemented in MOOS-IvP, using an abstraction layer that models a virtual system decoupled from the system’s actual dynamics. This makes the algorithm platform-agnostic and able to be combined with other behaviors such as collision avoidance and operating region behaviors. A trajectory defined by a lead agent is discretized and embedded with the leader’s dynamics and propagated to all following agents. We first demonstrate that this approach when paired with a simple PD controller prevents accumulated errors and improves trajectory tracking for follower agents. Thereafter we demonstrate how virtually coupling a subset of agent states improves the overall cohesiveness of the convoy. Improvements are demonstrated in both simulations and field trials using five autonomous surface vehicles.

The realm of multi-robot autonomous exploration tasks underscores the critical role of communication in coordinating group activities. This paper introduces an innovative decentralized multi-robot exploration algorithm, meticulously crafted to ensure unbroken communication within robotic groups, a crucial element for effective coordination. The motivation for our work is two-fold: Firstly, seamless communication is vital for coordinating multi-robot autonomous exploration tasks. Secondly, in applications such as disaster rescue operations or military maneuvers, there are numerous scenarios where spatial congregation of multiple robots is imperative for joint task accomplishment. Our approach addresses these challenges through a stringent communication constraint, ensuring that each robot remains in constant communicative contact with the rest of the group. This is realized by employing a decentralized policy that integrates Graph Neural Network (GNN) layers with self-attention mechanism. Such policy network design allows adaptation to different numbers of robots and varied environments. After an initial imitation learning phase, the policy is refined through learning from experiences generated via a tree-search-based lookahead technique. Our experimental analysis validates that the algorithm not only maintains consistent communication links among all group members but also improve the exploration efficiency under the communication constraints. These results highlight the potential of our method in enhancing the effectiveness of robotic group explorations while ensuring robust communication connection.

In urban environments where GNSS performance is degraded, localization can be performed using stable and geo-referenced map features detected by on-board sensors. Prior maps are prone to errors which have a direct impact on localization accuracy. By exchanging observed features and sharing their maps, vehicles can simultaneously improve their localization and update the map. This paper deals with indirect collaboration, where vehicles do not observe each other directly. The features are obtained from building facades using 3D lidar sensors. The paper emphasizes real-time decentralized collaboration with direct communication between vehicles, without the need for a central server. The collaboration takes place when vehicles perceive the same geo-referenced facades. Vehicle poses and maps are collaboratively updated using a Schmidt Kalman filter that carefully manages the cross-covariance terms. To maintain consistent estimates, the Kullback-Leibler Average is used. We also present a lidar data processing pipeline to obtain reliable observations from building facades. Real tests carried out with experimental vehicles on the university campus are reported. The results show that indirect collaboration makes a significant contribution to localization and map update when compared to a standalone method.

In the following, we analyze and discuss the implementation of a novel approach for distributed flocking behavior applied to a group of Uncrewed Aerial Vehicle (UAV)s, also referred to as drones. Inspired by natural flocking phenomena observed in birds, which demonstrate coordinated movement in response to internal and external stimuli, we tackle the problem of robust and dynamic aerial motion for robots and design a control law based on a novel physical model. In contrast to previous works that rely on velocity or position-based references, this approach leverages an acceleration-based law to describe the collective dynamics of many interacting particles. As observed in the following, a third-order control possesses several advantages compared to first or second-order control, such as smoother transitions, better force balancing, and more responsive and dynamic behaviors. These advantages are thoroughly analyzed in the following, thanks to physics-based realistic simulations and field experiments with medium-sized UAVs in an unstructured outdoor environment.

Optimal Control for legged robots has gone through a paradigm shift from position-based to torque-based control, owing to the latter’s compliant and robust nature. In parallel to this shift, the community has also turned to Deep Reinforcement Learning (DRL) as a promising approach to directly learn locomotion policies for complex real-life tasks. However, most end-to-end DRL approaches still operate in position space, mainly because learning in torque space is often sample-inefficient and does not consistently converge to natural gaits. To address these challenges, we propose a two-stage framework. In the first stage, we generate our own imitation data by training a position-based policy, eliminating the need for expert knowledge to design optimal controllers. The second stage incorporates decaying action priors, a novel method to enhance the exploration of torque-based policies aided by imitation rewards. We show that our approach consistently outperforms imitation learning alone and is robust to scaling these rewards from 0.1x to 10x. We further validate the benefits of torque control by comparing the robustness of a position-based policy to a position-assisted torque-based policy on a quadruped (Unitree Go1) without any domain randomization in the form of external disturbances during training.3

In this paper, we propose a radar odometry structure that directly utilizes radar velocity measurements for dead reckoning while maintaining its ability to update estimations within the Kalman filter framework. Specifically, we employ the Doppler velocity obtained by a 4D Frequency Modulated Continuous Wave (FMCW) radar in conjunction with gyroscope data to calculate poses. This approach helps mitigate high drift resulting from accelerometer biases and double integration. Instead, tilt angles measured by gravitational force are utilized alongside relative distance measurements from radar scan matching for the filter’s measurement update. Additionally, to further enhance the system’s accuracy, we estimate and compensate for the radar velocity scale factor. The performance of the proposed method is verified through five real-world open-source datasets. The results demonstrate that our approach reduces position error by 62% and rotation error by 66% on average compared to the state-of-the-art radar-inertial fusion method in terms of absolute trajectory error.

Ensuring safety and adapting to the user’s behavior are of paramount importance in physical human-robot interaction. Thus, incorporating elastic actuators in the robot’s mechanical design has become popular, since it offers intrinsic compliance and additionally provide a coarse estimate for the interaction force by measuring the deformation of the elastic components. While observer-based methods have been shown to improve these estimates, they rely on accurate models of the system, which are challenging to obtain in complex operating environments. In this work, we overcome this issue by learning the unknown dynamics components using Gaussian process (GP) regression. By employing the learned model in a Bayesian filtering framework, we improve the estimation accuracy and additionally obtain an observer that explicitly considers local model uncertainty in the confidence measure of the state estimate. Furthermore, we derive guaranteed estimation error bounds, thus, facilitating the use in safety-critical applications. We demonstrate the effectiveness of the proposed approach experimentally in a human-exoskeleton interaction scenario.