IROS 2024 论文列表

The physical coupling between robots has the potential to improve the capabilities of multi-robot systems in challenging manufacturing processes. However, the path tracking accuracy of physically coupled robots is not studied adequately, especially considering the uncertain kinematic parameters, the mechanical elasticity, and the built-in controllers of off-the-shelf robots. This paper addresses these issues with a novel differential-algebraic system model which is verified against measurement data from real execution. The uncertain kinematic parameters are estimated online to adapt the model. Consequently, an adaptive model predictive controller is designed as a coordinator between the robots. The controller achieves a path tracking error reduction of 88.6% compared to the state-of-the-art benchmark in the simulation.

In this paper, we propose a novel adaptive feedforward super-twisting sliding mode control algorithm to resolve the tracking control problem of parallel manipulators. The proposed control scheme includes three main terms, (i) the standard super-twisting algorithm, (ii) an adaptive feedforward dynamic model, and (iii) a feedback term to ensure stability. The proposed controller provides robustness towards uncertainties and disturbances, less sensitive to measurement noise, and allows dynamic parameters adaptation of the manipulator while executing a certain task. Real-time experiments are conducted on a 3-DOF non-redundant Delta parallel robot, including two main scenarios, (i) nominal case, and (ii) robustness towards operating acceleration changes. The relevance of the proposed controller is verified experimentally in both scenarios and compared with two other controllers from the literature, including the standard and the feedforward super-twisting sliding mode control algorithms.

This paper introduces a novel control design focused on enhancing the operational safety and efficiency of inspection, maintenance, and repair (IMR) operations conducted by autonomous remotely operated vehicles. Specifically, we propose using a safeguarding controller based on an adaptive control barrier function (CBF) incorporating safety properties previously identified in the literature. This approach permits temporary safe set violations, using an integrative penalty term to strengthen safety measures when necessary. A nominal nonlinear controller inside the safety bounds is proposed to ensure good reference tracking. The proposed controller is demonstrated in a simulation case study where an ROV (remotely operated vehicle) is set to clean an offshore monopile. The simulation includes unknown time-varying and step-like disturbances caused by water waves, the tether, ocean currents, and the high-pressure water jet. The proposed control law can react to the disturbances, and the ROV never leaves the defined safe set.

In this paper, we study the problem of adapting manipulation trajectories involving grasped objects (e.g. tools) defined for a single grasp pose to novel grasp poses. A common approach to address this is to define a new trajectory for each possible grasp explicitly, but this is highly inefficient. Instead, we propose a method to adapt such trajectories directly while only requiring a period of self-supervised data collection, during which a camera observes the robot’s end-effector moving with the object rigidly grasped. Importantly, our method requires no prior knowledge of the grasped object (such as a 3D CAD model), it can work with RGB images, depth images, or both, and it requires no camera calibration. Through a series of real-world experiments involving 1360 evaluations, we find that self-supervised RGB data consistently outperforms alternatives that rely on depth images including several state-of-the-art pose estimation methods. Compared to the best-performing baseline, our method results in an average of 28.5% higher success rate when adapting manipulation trajectories to novel grasps on several everyday tasks. The appendix accompanying the paper and videos of the experiments are available on our webpage at www.robot-learning.uk/adapting-skills.

Multi-rotor aerial autonomous vehicles (MAVs) primarily rely on vision for navigation purposes. However, visual localization and odometry techniques suffer from poor performance in low or direct sunlight, a limited field of view, and vulnerability to occlusions. Acoustic sensing can serve as a complementary or even alternative modality for vision in many situations, and it also has the added benefits of lower system cost and energy footprint, which is especially important for micro aircraft. This paper proposes actively controlling and shaping the aircraft propulsion noise generated by the rotors to benefit localization tasks, rather than considering it a harmful nuisance. We present a neural network architecture for self-noise-based localization in a known environment. We show that training it simultaneously with learning time-varying rotor phase modulation achieves accurate and robust localization. The proposed methods are evaluated using a computationally affordable simulation of MAV rotor noise in 2D acoustic environments that is fitted to real recordings of rotor pressure fields. Code3 and data4 are accompanied.

The installation of a multi-layer 3D LiDAR atop the vehicle is a widely adopted hardware configuration for map-matching-based localization in intelligent driving. By offering a comprehensive 360° horizontal Field of View (FoV), this setup aims to achieve precise matching outcomes through the imposition of substantial geometric constraints against dynamic interferences and structural degradation. However, several factors limit its environmental adaptability, such as sparse point cloud density at distances, insufficient maximum sensing range, and notably, the restricted beam elevation angle, limiting the perception of the environment beyond obstacles. The rapid advancement of non-repetitive scanning LiDARs shows promise in mitigating such limitations. Nevertheless, their narrow FoV remains a challenge to overcome. In this study, we propose a solution by mounting such one single LiDAR on a two-axis rotating gimbal, enabling the vehicle to surpass the ranges and vertical FoV limitations of traditional setups actively. The corresponding gimbal motion strategy has been designed to automatically focus on the environment component with the most robust geometric constraints. Experimental results validate that the proposed method achieves superior robustness under high dynamic interference while delivering sufficient performance under standard conditions.

We study pursuit-evasion games in highly occluded urban environments, e.g. tall buildings in a city, where a scout (quadrotor) tracks multiple dynamic targets on the ground. We show that we can build a neural radiance field (NeRF) representation of the city—online—using RGB and depth images from different vantage points. This representation is used to calculate the information gain to both explore unknown parts of the city and track the targets—thereby giving a completely first-principles approach to actively tracking dynamic targets. We demonstrate, using a custom-built simulator using Open Street Maps data of Philadelphia and New York City, that we can explore and locate 20 stationary targets within 300 steps. This is slower than a greedy baseline, which does not use active perception. But for dynamic targets that actively hide behind occlusions, we show that our approach maintains, at worst, a tracking error of 200m; the greedy baseline can have a tracking error as large as 600m. We observe a number of interesting properties in the scout’s policies, e.g., it switches its attention to track a different target periodically, as the quality of the NeRF representation improves over time, the scout also becomes better in terms of target tracking.

6D pose estimation of textureless shiny objects has become an essential problem in many robotic applications. Many pose estimators require high-quality depth data, often measured by structured light cameras. However, when objects have shiny surfaces (e.g., metal parts), these cameras fail to sense complete depths from a single viewpoint due to the specular reflection, resulting in a significant drop in the final pose accuracy. To mitigate this issue, we present a complete active vision framework for 6D object pose refinement and next-best-view prediction. Specifically, we first develop an optimization-based pose refinement module for the structured light camera. Our system then selects the next best camera viewpoint to collect depth measurements by minimizing the predicted uncertainty of the object pose. Compared to previous approaches, we additionally predict measurement uncertainties of future viewpoints by online rendering, which significantly improves the next-best-view prediction performance. We test our method on the real-world ROBI dataset. The results show that our pose refinement module outperforms the traditional ICP-based approach when given the same input depth data, and our next-best-view strategy can achieve high object pose accuracy with significantly fewer viewpoints than the heuristic-based policies.

We introduce a NeRF-based active mapping system that enables efficient and robust exploration of large-scale indoor environments. The key to our approach is the extraction of a generalized Voronoi graph (GVG) from the continually updated neural map, leading to the synergistic integration of scene geometry, appearance, topology, and uncertainty. Anchoring uncertain areas induced by the neural map to the vertices of GVG allows the exploration to undergo adaptive granularity along a safe path that traverses unknown areas efficiently. Harnessing a modern hybrid NeRF representation, the proposed system achieves competitive results in terms of reconstruction accuracy, coverage completeness, and exploration efficiency even when scaling up to large indoor environments. Extensive results at different scales validate the efficacy of the proposed system.

The autonomous mapping of large-scale urban scenes presents significant challenges for autonomous robots. To mitigate the challenges, global planning, such as utilizing prior GPS trajectories from OpenStreetMap (OSM), is often used to guide the autonomous navigation of robots for mapping. However, due to factors like complex terrain, unexpected body movement, and sensor noise, the uncertainty of the robot’s pose estimates inevitably increases over time, ultimately leading to the failure of robotic mapping. To address this issue, we propose a novel active loop closure procedure, enabling the robot to actively re-plan the previously planned GPS trajectory. The method can guide the robot to re-visit the previous places where the loop-closure detection can be performed to trigger the back-end optimization, effectively reducing errors and uncertainties in pose estimation. The proposed active loop closure mechanism is implemented and embedded into a real-time OSM-guided robot mapping framework. Empirical results on several large-scale outdoor scenarios demonstrate its effectiveness and promising performance.

This paper enhances the obstacle avoidance of Autonomous Surface Vehicles (ASVs) for safe navigation in high-traffic waters with an active state estimation of obstacle’s passing intention and reducing its uncertainty. We introduce a topological modeling of passing intention of obstacles, which can be applied to varying encounter situations based on the inherent embedding of topological concepts in COLREGs. With a Long Short-Term Memory (LSTM) neural network, we classify the passing intention of obstacles. Then, for determining the ASV maneuver, we propose a multi-objective optimization framework including information gain about the passing obstacle intention and safety. We validate the proposed approach under extensive Monte Carlo simulations (2,400 runs) with a varying number of obstacles, dynamic properties, encounter situations, and different behavioral patterns of obstacles (cooperative, non-cooperative). We also present the results from a real marine accident case study as well as real-world experiments of a real ASV with environmental disturbances, showing successful collision avoidance with our strategy in real-time.

In flexible redundantly-driven multi-DOF systems, like living beings, the representation of redundant kinematics including the diversity of solutions, is crucial for leveraging its distinctive characteristics. This paper proposes an active learning framework for forward and inverse modeling of complex kinematics that improves expressions of control space, task space, and null space. It consists of a Variational Auto Encoder (VAE)-type network that internally holds expressions of control space, task space, and null space, and an algorithm for selecting new data using the cross-entropy method. The validity of the proposed system was verified using a tensegrity manipulator driven by 40 pneumatic cylinders. As a result, it was confirmed that active learning contributed to achieving the entire range of motion covered and a well-organized representation of the null space.

We address the task of long-horizon navigation in partially mapped environments for which active gathering of information about faraway unseen space is essential for good behavior. We present a novel planning strategy that, at training time, affords tractable computation of the value of information associated with revealing potentially informative regions of unseen space, data used to train a graph neural network to predict the goodness of temporally-extended exploratory actions. Our learning-augmented model-based planning approach predicts the expected value of information of revealing unseen space and is capable of using these predictions to actively seek information and so improve long-horizon navigation. Across two simulated office-like environments, our planner outperforms competitive learned and non-learned baseline navigation strategies, achieving improvements of up to 63.76% and 36.68%, demonstrating its capacity to actively seek performance-critical information.

One of the core activities of an active observer involves moving to secure a "better" view of the scene, where the definition of "better" is task-dependent. This paper focuses on the task of human pose estimation from videos capturing a person’s activity. Self-occlusions within the scene can complicate or even prevent accurate human pose estimation. To address this, relocating the camera to a new vantage point is necessary to clarify the view, thereby improving 2D human pose estimation. This paper formalizes the process of achieving an improved viewpoint. Our proposed solution to this challenge comprises three main components: a NeRF-based Drone-View Data Generation Framework, an On-Drone Network for Camera View Error Estimation, and a Combined Planner for devising a feasible motion plan to reposition the camera based on the predicted errors for camera views. The Data Generation Framework utilizes NeRF-based methods to generate a comprehensive dataset of human poses and activities, enhancing the drone’s adaptability in various scenarios. The Camera View Error Estimation Network is designed to evaluate the current human pose and identify the most promising next viewing angles for the drone, ensuring a reliable and precise pose estimation from those angles. Finally, the combined planner incorporates these angles while considering the drone’s physical and environmental limitations, employing efficient algorithms to navigate safe and effective flight paths. This system represents a significant advancement in active 2D human pose estimation for an autonomous UAV agent, offering substantial potential for applications in aerial cinematography by improving the performance of autonomous human pose estimation and maintaining the operational safety and efficiency of UAVs.

We present a tracking system for relative positioning that can operate on entirely moving tracking nodes without the need for stationary anchors. Each node embeds a 9-DOF magnetic and inertial measurement unit and a single-antenna ultra-wideband radio. We introduce a multi-stage filtering pipeline through which our system estimates the relative layout of all tracking nodes within the group. The key novelty of our method is the integration of a custom Extended Kalman filter (EKF) with a refinement step via multidimensional scaling (MDS). Our method integrates the MDS output back into the EKF, thereby creating a dynamic feedback loop for more robust estimates. We complement our method with UWB ranging protocol that we designed to allow tracking nodes to opportunistically join and leave the group.In our evaluation with constantly moving nodes, our system estimated relative positions with an error of 10.2 cm (in 2D) and 21.7 cm (in 3D), including obstacles that occluded the line of sight between tracking nodes. Our approach requires no external infrastructure, making it particularly suitable for operation in environments where stationary setups are impractical.

Loop Closure Detection (LCD) is an essential component of Simultaneous Localization and Mapping (SLAM), helping to correct drift errors, facilitate map merging, or both by identifying previously observed scenes. Despite its importance, traditional LCD algorithms based on single sensor such as camera or LiDAR exhibit degraded performance in challenging scenarios due to their inherent limitations. To address this issue, we propose a novel LCD method based on camera-LiDAR fusion, exploiting the rich textural information from cameras and the accurate geometric data from LiDAR to ensure robustness and speed in challenging environments. Specifically, we first employ deep hashing learning to encode deep image features into binary image descriptors for extremely fast loop candidate (LC) retrieval. Then, LiDAR points are augmented with image color for accurate geometric verification. Finally, we incorporate a spatial-temporal consistency check that mandates an LC to have consistently matched neighbors to be accepted as true. Our method is extensively verified and compared with the state-of-the-art methods on various datasets encompassing both indoor and outdoor environments. Experimental results demonstrate that our method obtains the best performance, increasing the maximum recall rate at 100% precision by a significant margin of 20% while operating in real-time at an average speed of 30 fps.

This paper presents a novel approach to enhance Model Predictive Control (MPC) for legged robots through Distributed Optimization. Our method focuses on decomposing the robot dynamics into smaller, parallelizable subsystems, and utilizing the Alternating Direction Method of Multipliers (ADMM) to ensure consensus among them. Each subsystem is managed by its own Optimal Control Problem, with ADMM facilitating consistency between their optimizations. This approach not only decreases the computational time but also allows for effective scaling with more complex robot configurations, facilitating the integration of additional subsystems such as articulated arms on a quadruped robot. We demonstrate, through numerical evaluations, the convergence of our approach on two systems with increasing complexity. In addition, we showcase that our approach converges towards the same solution when compared to a state-of-the-art centralized whole-body MPC implementation. Moreover, we quantitatively compare the computational efficiency of our method to the centralized approach, revealing up to a 75% reduction in computational time. Overall, our approach offers a promising avenue for accelerating MPC solutions for legged robots, paving the way for more effective utilization of the computational performance of modern hardware. Accompanying video at https://youtu.be/Yar4W-Vlh2A. The related code can be found at https://github.com/iit-DLSLab/DWMPC

To give the single robot system the ability to realize many behaviors and to realize tasks with shifting environments and objectives, it is necessary to abstract the robot’s body ability to the extent that they can be detected by sensors in the body so that we can plan as the problem of state transition.In this paper, to abstract the transformer robot system that performs heavy lifting and environmental attachment tasks in an aquatic and terrestrial environment, we extend the graphical representation of the robot’s body to manage joint capability and the body adaptability for the environment. To abstract the body ability, we divide the body into elements and define Connection between them at three different granularities. And using Connection, we propose the Connection Modification Feature(CMF) as the representation for changing body ability. To implement the Connection Modification Feature, we perform the abstract description and extract Connection to construct Body Ability Graph, a graph for the robot to manage its body ability. We show that it is possible to plan to manipulate and use its own Connection Modification Feature through multiple experiments by defining Normal Action that does not change body abilities and Body Ability Modifying Action that manipulate body abilities.

Accurate localization in challenging garage environments—marked by poor lighting, sparse textures, repetitive structures, dynamic scenes, and the absence of GPS—is crucial for automated valet parking (AVP) tasks. Addressing these challenges, our research introduces AVM-SLAM, a cutting-edge semantic visual SLAM architecture with multi-sensor fusion in a bird’s eye view (BEV). This novel framework synergizes the capabilities of four fisheye cameras, wheel encoders, and an inertial measurement unit (IMU) to construct a robust SLAM system. Unique to our approach is the implementation of a flare removal technique within the BEV imagery, significantly enhancing road marking detection and semantic feature extraction by convolutional neural networks for superior mapping and localization. Our work also pioneers a semantic prequalification (SPQ) module, designed to adeptly handle the challenges posed by environments with repetitive textures, thereby enhancing loop detection and system robustness. To demonstrate the effectiveness and resilience of AVM-SLAM, we have released a specialized multi-sensor and high-resolution dataset of an underground garage, accessible at https://yale-cv.github.io/avm-slamdataset, encouraging further exploration and validation of our approach within similar settings.