IROS 2024 Papers

Scientists often search for phenomenon of interest while exploring new environments. Autonomous vehicles are deployed to explore such areas where human-operated vehicles would be costly or dangerous. Online control of autonomous vehicles for information-gathering is called adaptive sampling and can be framed as a Partially Observable Markov Decision Process (POMDPs) that uses information gain as its principal objective. While prior work focuses largely on single-agent scenarios, this paper confronts challenges unique to multi-agent adaptive sampling, such as avoiding redundant observations, preventing vehicle collision, and facilitating path planning under limited communication. We start with Multi-Agent Path Finding (MAPF) methods, which address collision avoidance by decomposing the multi-agent path planning problem into a series of single-agent path planning problems. We present an extension to these methods called information-driven MAPF which addresses multi-agent information gain under limited communication. First, we introduce an admissible heuristic that relaxes mutual information gain to an additive function that can be evaluated as a set of independent single agent path planning problems. Second, we extend our approach to a distributed system that is robust to limited communication. When all agents are in range, the group plans jointly to maximize information. When some agents move out of range, communicating subgroups are formed and the subgroups plan independently. Since redundant observations are less likely when vehicles are far apart, this approach only incurs a small loss in information gain, resulting in an approach that gracefully transitions from full to partial communication. We evaluate our method against other adaptive sampling strategies across various scenarios, including real-world robotic applications. Our method was able to locate up to 200% more unique phenomena in certain scenarios, and each agent located its first unique phenomenon fast... Show More

When coordinating the motion of connected autonomous vehicles at a signal-free intersection, the vehicles from each direction naturally forms a team and each team seeks to minimize their own traversal time through the intersection, without concerning the traversal times of other teams. Since the intersection is shared by all teams and agent-agent collision must be avoided, the coordination has to trade the traversal time of one team for the other. This paper thus investigates a problem called Multi-Agent Teamwise Cooperative Path Finding (TCPF), which seeks a set of collision-free paths for the agents from their respective start to goal locations, and agents are grouped into multiple teams with each team having its own objective function to optimize. In general, there are more than one teams and hence multiple objectives. TCPF thus seeks the Pareto-optimal front that represents possible trade-offs among the teams. We develop a centralized planner for TCPF by leveraging the Multi-Agent Path Finding techniques to resolve agent-agent collision, and Multi-Objective Optimization to find Pareto-optimal solutions. We analyze the completeness and optimality of the planner, which is then tested in various settings with up to 40 agents to verify the runtime efficiency and showcase the usage in intersection coordination.

Due to the complex interactions between agents, learning multi-agent control policy often requires a prohibitive amount of data. This paper aims to enable multi-agent systems to effectively utilize past memories to adapt to novel collaborative tasks in a data-efficient fashion. We propose the Multi-Agent Coordination Skill Database, a repository for storing a collection of coordinated behaviors associated with key vectors distinctive to them. Our Transformer-based skill encoder effectively captures spatio-temporal interactions that contribute to coordination and provides a unique skill representation for each coordinated behavior. By leveraging only a small number of demonstrations of the target task, the database enables us to train the policy using a dataset augmented with the retrieved demonstrations. Experimental evaluations demonstrate that our method achieves a significantly higher success rate in push manipulation tasks compared with baseline methods like few-shot imitation learning. Furthermore, we validate the effectiveness of our retrieve-and-learn framework in a real environment using a team of wheeled robots.

High-level robot skills represent an increasingly popular paradigm in robot programming. However, configuring the skills’ parameters for a specific task remains a manual and time-consuming endeavor. Existing approaches for learning or optimizing these parameters often require numerous real-world executions or do not work in dynamic environments. To address these challenges, we propose Multimodal Trajectory Transformer (MuTT), a novel encoder-decoder transformer architecture designed to predict environment-aware executions of robot skills by integrating vision, trajectory, and robot skill parameters. Notably, we pioneer the fusion of vision and trajectory, introducing a novel trajectory projection. Furthermore, we illustrate MuTT’s efficacy as a predictor when combined with a model-based robot skill optimizer. This approach facilitates the optimization of robot skill parameters for the current environment, without the need for real-world executions during optimization. Designed for compatibility with any representation of robot skills, MuTT demonstrates its versatility across three comprehensive experiments, showcasing superior performance across two different skill representations.

Autonomous navigation across unstructured terrains, including forests and construction areas, faces unique challenges due to intricate obstacles and the element of the unknown. Lacking pre-existing maps, these scenarios necessitate a motion planning approach that combines agility with efficiency. Critically, it must also incorporate the robot’s kinematic constraints to navigate more effectively through complex environments. This work introduces a novel planning method for center-articulated vehicles (CAV), leveraging motion primitives within a receding horizon planning framework using onboard sensing. The approach commences with the offline creation of motion primitives, generated through forward simulations that reflect the distinct kinematic model of center-articulated vehicles. These primitives undergo evaluation through a heuristic-based scoring function, facilitating the selection of the most suitable path for real-time navigation. To account for disturbances, we develop a pose-stabilizing controller, tailored to the kinematic specifications of center-articulated vehicles. During experiments, our method demonstrates a 67% improvement in SPL (Success Rate weighted by Path Length) performance over existing strategies. Furthermore, its efficacy was validated through real-world experiments conducted with a tree harvester vehicle - SAHA.

Event cameras have garnered considerable attention due to their advantages over traditional cameras in low power consumption, high dynamic range, and no motion blur. This paper proposes a monocular event-inertial odometry incorporating an adaptive decay kernel-based time surface with polarity-aware tracking. We utilize an adaptive decay-based Time Surface to extract texture information from asynchronous events, which adapts to the dynamic characteristics of the event stream and enhances the representation of environmental textures. However, polarity-weighted time surfaces suffer from event polarity shifts during changes in motion direction. To mitigate its adverse effects on feature tracking, we optimize the feature tracking by incorporating an additional polarityinverted time surface to enhance the robustness. Comparative analysis with visual-inertial and event-inertial odometry methods shows that our approach outperforms state-of-the-art techniques, with competitive results across various datasets.

Autonomous nano-quadcopters possess large potential for indoor use. Existing works on autonomous flight however rely on large amounts of compute, therefore resulting in heavy and bulky platforms that can only be safely deployed outdoors. We present a monocular depth estimation method for autonomous indoor obstacle avoidance and waypoint navigation of nano-quadcopters demonstrated on the Bitcraze Crazyflie 2.1 which weighs a mere 33g. Our depth estimation model has 1.56 million parameters and is 4 MB, which after quantization becomes 1 MB. We transmit the images via WiFi from the onboard grayscale camera on the Bitcraze to a laptop, which then runs the 1 MB quantized model to generate small-size depth maps. Subsequently, we run our navigation algorithms on a laptop and transmit high-level motion commands back to the drone. We demonstrate obstacle avoidance capability of this end-to-end system through real-world flights in a variety of indoor environments.

This paper introduces a framework for monocular 3D reconstruction of cheetah movements, leveraging a combination of data-driven and physics-based modeling as well as trajectory optimization. Unlike traditional methods that rely solely on kinematics, our approach integrates dynamic motion principles, enhancing the plausibility and generalization of motion estimates. Validated on the cheetah running dataset, AcinoSet, we achieve mean per-joint position errors of 78.8 mm and 72.5 mm, showcasing significant advancements over the existing model used in AcinoSet. By addressing the challenge of absent ground truth data, this work not only advances animal motion capture techniques but also informs the development of bio-inspired robotic systems, offering a robust solution for accurately capturing complex animal locomotion in natural settings.

This paper presents a generalizable 3D plane detection and reconstruction framework named MonoPlane. Unlike previous robust estimator-based works (which require multiple images or RGB-D input) and learning-based works (which suffer from domain shift), MonoPlane combines the best of two worlds and establishes a plane reconstruction pipeline based on monocular geometric cues, resulting in accurate, robust and scalable 3D plane detection and reconstruction in the wild. Specifically, we first leverage large-scale pre-trained neural networks to obtain the depth and surface normals from a single image. These monocular geometric cues are then incorporated into a proximity-guided RANSAC framework to sequentially fit each plane instance. We exploit effective 3D point proximity and model such proximity via a graph within RANSAC to guide the plane fitting from noisy monocular depths, followed by image-level multi-plane joint optimization to improve the consistency among all plane instances. We further design a simple but effective pipeline to extend this single-view solution to sparse-view 3D plane reconstruction. Extensive experiments on a list of datasets demonstrate our superior zero-shot generalizability over baselines, achieving state-of-the-art plane reconstruction performance in a transferring setting. Our code is available at https://github.com/thuzhaowang/MonoPlane.

While autonomous navigation of mobile robots on rigid terrain is a well-explored problem, navigating on deformable terrain such as tall grass or bushes remains a challenge. To address it, we introduce an explainable, physics-aware and end-to-end differentiable model which predicts the outcome of robot-terrain interaction from camera images, both on rigid and non-rigid terrain. The proposed MonoForce model consists of a black-box module which predicts robot-terrain interaction forces from onboard cameras, followed by a white-box module, which transforms these forces and a control signals into predicted trajectories, using only the laws of classical mechanics. The differentiable white-box module allows backpropagating the predicted trajectory errors into the black-box module, serving as a self-supervised loss that measures consistency between the predicted forces and ground-truth trajectories of the robot. Experimental evaluation on a public dataset and our data has shown that while the prediction capabilities are comparable to state-of-the-art algorithms on rigid terrain, MonoForce shows superior accuracy on nonrigid terrain such as tall grass or bushes. To facilitate the reproducibility of our results, we release both the code and datasets.

The current state-of-the-art gradient-based optimisation frameworks are able to produce impressive dynamic manoeuvres such as linear and rotational jumps. However, these methods, which optimise over the full rigid-body dynamics of the robot, often require precise foothold locations apriori, while real-time performance is not guaranteed without elaborate regularisation and tuning of the cost function. In contrast, we investigate the advantages of a task-space optimisation framework, with special focus on acrobatic motions. Our proposed formulation exploits the system's high-order nonlinearities, such as the nonholonomy of the angular momentum, in order to produce feasible, high-acceleration manoeuvres. By leveraging the full-centroidal dynamics of the quadruped ANYmal C and directly optimising its footholds and contact forces, the framework is capable of producing efficient motion plans with low computational overhead. Finally, we deploy our proposed framework on the ANYmal C platform, and demonstrate its true capabilities through real-world experiments, with the successful execution of high-acceleration motions, such as linear and rotational jumps. Extensive analysis of these shows that the robot's dynamics can be exploited to surpass its hardware limitations of having a high mass and low-torque limits.

As people age, walking can become difficult. This difficulty in walking can lead to further physical decline and eventually result in the need for caregiving. To prevent this, wearable robots supporting walking rehabilitation have been developed. In this study, we aim to develop a modular robot wear that can be customized according to the user’s needs. The modular robot wear consists of motor modules, sensor modules, and a processor. Motor modules can be attached to any part of the body that requires assistance, enabling support not only in the sagittal plane but also in the frontal plane. By calculating the relative movement of the center of gravity based on the information from the acceleration sensors in the sensor modules, commands to the motor modules are generated. This allows for assistance tailored to the user’s walking pattern. Through verification experiments of the robot wear’s operation, we confirmed its ability to provide support according to changes in walking patterns. However, while the robot wear can induce changes in walking patterns, there are challenges regarding the output it provides to the wearer.

This paper introduces a generic filter-based state estimation framework that supports two state-decoupling strategies based on cross-covariance factorization. These strategies reduce the computational complexity and inherently support true modularity – a perquisite for handling and processing meshed range measurements among a time-varying set of devices. In order to utilize these measurements in the estimation framework, positions of newly detected stationary devices (anchors) and the pairwise biases between the ranging devices are required. In this work an autonomous calibration procedure for new anchors is presented, that utilizes range measurements from multiple tags as well as already known anchors. To improve the robustness, an outlier rejection method is introduced. After the calibration is performed, the sensor fusion framework obtains initial beliefs of the anchor positions and dictionaries of pairwise biases, in order to fuse range measurements obtained from new anchors tightly-coupled. The effectiveness of the filter and calibration framework has been validated through evaluations on a recorded dataset and real-world experiments.

Fixed-wing Uncrewed Aerial Vehicles (UAVs) can be used for remote package delivery missions by connecting a package to a UAV via a long tether. With a circular flight path at a calculated loiter radius, tether length and orbiting velocity, a package can be lowered to the ground in a quasi-stationary manner. This method achieves this with significantly less energy than hybrid hovering aircraft, which are currently used. UAV operating limitations pose a challenge for this method as ensuring the package stabilises such that it can be safely deployed without damage or posing a risk to people or property is challenging in most environmental conditions. To improve tether and package stabilisation, we introduce a novel Mid-Tether Drag Device (MTDD) which enables compliant delivery missions within regulatory frameworks. We present a mathematical model of the delivery with a UAV and MTDD. We verify the accuracy of our model with real-world flight tests in low wind conditions without the MTDD, which have not been previously conducted at this scale in literature. Further validation is presented with flight tests at a flight range using a UAV instrumented with a package deployment system with both UAV and package tracking data acquisition. Our work enhances the abilities of UAVs to conduct aerial package delivery.

To ensure operational accuracy and flexibility in confined environments, the cable-driven hyper-redundant manipulator needs to take into account both compliance and stiffness. Although the cable-driven method enables the manipulator to have adjustable stiffness, the theoretical analyses and studies on the effects of various factors on the stiffness are insufficient, leading to the possibility that the existing variable stiffness strategies may disrupt the equilibrium state of the manipulator. Accordingly, this paper presents the modeling and analysis of joint-to-end variable stiffness for cable-driven hyper-redundant manipulator. First, the multi-layered static models are constructed to decouple and characterise the complex robotic arm system by combining the manipulator kinematics with the virtual work principle. Then, the joint-to-end analytical stiffness models are developed to explore the influencing factors of stiffness, and relevant stiffness indicators are designed to evaluate the stiffness level. Finally, with platform validation and numerical method, the connections between cable tension, joint angle, joint stiffness and end stiffness are analysed, thereby summarising the variable stiffness characteristics of the cable-driven super-redundant manipulator.

In situations where robots work alongside humans, they must be capable of responding flexibly to unexpected external forces. To address this challenge, researchers have conducted numerous studies on soft robotics. However, most of the soft hands studied so far are powered by pneumatic pressure and can only exert pressure up to a several hundred kPa, resulting in low output. To solve this problem, we have developed a hydraulic soft hand in our previous research. In this paper, we derive the relationship between driving pressure, bending angle, and grasping force of a soft hand with a reservoir to evaluate the effect of a rubber sheet reservoir. Additionally, we experimentally show that the soft hand provides grasping flexibility when angle control is applied using the model proposed in this paper.

The movement of the legs involves the interaction between the feet and the ground. Consequently, most animals possess a wide variety of foot morphologies and multifunctional capabilities. The selection and switching of these foot functions are passive and environment-dependent, ensuring environmental compliance. Despite this, current research on compliant feet lacks mathematical models that simultaneously encompass locomotion and foot compliance. Therefore, conducting in-depth studies on locomotion properties under current conditions is challenging. In this study, we present novel passive compliant feet applicable to the passive walking of a rimless wheel. We first introduce a dynamic model, achieve passive walking through numerical simulations, and subsequently analyze the gait patterns for compliance and multi-period gait. This study bridges a gap in understanding the interaction between motion and compliance in foot design, providing insights into the dynamics of compliant motion.

The quadrupeds have wider active territory than humans. Their bodies can adapt various environments through evolution, enabling the efficient, elegant gait for their legged locomotion. Previous researches have indicated lots of examples of utilizing the advantages of body to achieve environment adaptive and stable gait, and for legged locomotion, especially with quadruped robot determining how to generate environment-adaptive mobile locomotion remains a significant challenge. In this study, we discussed the adaptability to environments of quadruped robots, specific walking stability and gait convergence in low-friction environments with compliant torso. The numerically simulations are proposed for observing the trend of walking performance with various friction coefficient. By analyzing the typical walking gait, the adaptability of quadruped walkers with compliant torso are found. These conclusions contribute to the design and development of compliant torso for quadruped walkers.

The application of learning-based control methods in robotics presents significant challenges. One is that model-free reinforcement learning algorithms use observation data with low sample efficiency. To address this challenge, a prevalent approach is model-based reinforcement learning, which involves employing an environment dynamics model. We suggest approximating transition dynamics with symbolic expressions, which are generated via symbolic regression. Approximation of a mechanical system with a symbolic model has fewer parameters than approximation with neural networks, which can potentially lead to higher accuracy and quality of extrapolation. We use a symbolic dynamics model to generate trajectories in model-based policy optimization to improve the sample efficiency of the learning algorithm. We evaluate our approach across various tasks within simulated environments. Our method demonstrates superior sample efficiency in these tasks compared to model-free and model-based baseline methods.