IROS 2024 论文列表

When multiple tablewares are closely packed on a table, rearranging obstacles to make space is necessary to grasp the target, often called target singulation. Due to the nature of handling fragile tablewares (i.e. plates, bowls), we make a few assumptions for the target singulation problem. First, tableware is grasped with a multi-fingered gripper; second, rearrangement is based on prehensile motions like pick-and-place. Under these assumptions, we aim to generate a relocation plan that guarantees global optimality. Furthermore, if any relocation plan cannot singulate the target, we aim to determine it quickly. Therefore, we propose a search method that utilizes the relationship between the object and its nearby obstacles expressed in propositional logic. We define the problem as determining logical entailment (i.e., whether the target can be singulated) and expand the search tree from the target while generating an optimal relocation plan. We demonstrate the performance of our algorithm by increasing the number of objects and validate the plan in a simulation environment.

Imitation learning for robot dexterous manipulation, especially with a real robot setup, typically requires a large number of demonstrations. In this paper, we present a data-efficient learning from demonstration framework which exploits the use of rich tactile sensing data and achieves fine bimanual pinch grasping. Specifically, we employ a convolutional autoencoder network that can effectively extract and encode high-dimensional tactile information. Further, we develop a framework that achieves efficient multi-sensor fusion for imitation learning, allowing the robot to learn contact-aware sensorimotor skills from demonstrations. The ablation studies on encoded tactile features highlighted the effectiveness of incorporating rich contact information, which enabled dexterous bimanual grasping with active contact searching. Extensive experiments demonstrated the robustness of the fine pinch grasp policy directly learned from few-shot demonstration, including grasping of the same object with different initial poses, generalizing to ten unseen new objects, robust and firm grasping against external pushes, as well as contact-aware and reactive re-grasping in case of dropping objects under very large perturbations. Furthermore, the saliency map analysis method is used to describe weight distribution across various modalities during pinch grasping, confirming the effectiveness of our framework at leveraging multimodal information. The video is available online at: https://youtu.be/BlzxGgiKfck.

Industrial robots are essential for modern production but often struggle to adapt to new tasks. Modular (reconfigurable) robots can overcome this challenge by eliminating the need to replace the whole robot. However, finding the optimal assembly for a task remains difficult because a valid path has to be computed for each generated assembly – consuming a significant fraction of the computation time. Similar to online path planning, where previous approaches adapt known paths to a changing environment, we show that transferring paths from previously considered module assemblies accelerates path planning for the next assemblies. On average, our method reduces the planning time for single-goal tasks by 50%. The usefulness of our method is evaluated by integrating it in a genetic algorithm (GA) for optimizing assemblies and evaluating it on our benchmark suite CoBRA. Within the optimization loop for modular robots, the time used to check a single assembly is shortened by up to 50%.

For efficient and safe autonomous driving, it is essential that autonomous vehicles can predict the motion of other traffic agents. While highly accurate, current motion prediction models often impose significant challenges in terms of training resource requirements and deployment on embedded hardware. We propose a new efficient motion prediction model, which achieves highly competitive benchmark results while training only a few hours on a single GPU. Due to our lightweight architectural choices and the focus on reducing the required training resources, our model can easily be applied to custom datasets. Furthermore, its low inference latency makes it particularly suitable for deployment in autonomous applications with limited computing resources.

Penetration depth (PD) is essential for robotics due to its extensive applications in dynamic simulation, motion planning, haptic rendering, etc. The Expanding Polytope Algorithm (EPA) is the de facto standard for this problem, which estimates PD by expanding an inner polyhedral approximation of an implicit set. In this paper, we propose a novel optimization-based algorithm that incrementally estimates minimum penetration depth and its direction. One major advantage of our method is the capability to be warm-started by leveraging the spatial and temporal coherence. This coherence emerges naturally in many robotic applications (e.g., the temporal coherence between adjacent simulation time knots). As a result, our algorithm achieves substantial speedup — we demonstrate it is 5-30x faster than EPA on several benchmarks. Moreover, our approach is built upon the same implicit geometry representation as EPA, which enables easy integration into existing software stacks. The code and supplemental document are available on: https://github.com/weigao95/mind-fcl.

We present the results of experiments performed using a team of small autonomous underwater vehicles (AUVs) to determine the location of an isobath. The primary contributions of this work are (1) the development of a novel objective function for level set estimation that utilizes a rigorous assessment of uncertainty, and (2) a description of the practical challenges and corresponding solutions needed to implement our approach in the field using a team of AUVs. We combine path planning techniques and an approach to decentralization from prior work that yields theoretical performance guarantees. Experimentation with a team of AUVs provides empirical evidence that the desirable performance guarantees can be preserved in practice even in the presence of limitations that commonly arise in underwater robotics, including slow and intermittent acoustic communications and limited computational resources.

This paper addresses the problem of choosing a sparse subset of measurements for quick calibration parameter estimation. A standard solution to this is selecting a measurement only if its utility—the difference between posterior (with the measurement) and prior information (without the measurement)—exceeds some threshold. Theoretically, utility, a function of the parameter estimate, should be evaluated at the estimate obtained with all measurements selected so far, hence necessitating a recalibration with each new measurement. However, we hypothesize that utility is insensitive to changes in the parameter estimate for many systems of interest, suggesting that evaluating utility at some initial parameter guess would yield equivalent results in practice. We provide evidence supporting this hypothesis for extrinsic calibration of multiple inertial measurement units (IMUs), showing the reduction in calibration time by two orders of magnitude by forgoing recalibration for each measurement.

Non-parametric estimates of frequency response functions (FRFs) are often suitable for describing the dynamics of a mechanical system. If treating these estimates as measurements, they can be used for parametric identification of, e.g., a gray-box model. This paper shows that a more accurate parametric model can be identified based on local parametric FRF estimates, giving a shorter total experiment time, compared to classical methods. Classical methods for non-parametric FRF estimation of MIMO (Multiple Input Multiple Output) systems require at least as many experiments as the system has inputs. Local parametric FRF estimation methods have been developed for avoiding multiple experiments. In this paper, these local methods are adapted and applied for estimating the FRFs of a 6-axes robotic manipulator, which is a nonlinear MIMO system operating in closed loop. The aim is to reduce the experiment time and amount of data needed for identification. The resulting FRFs are analyzed in an experimental study and compared to estimates obtained by classical MIMO techniques.

We propose a real-time dynamic LiDAR odometry pipeline for mobile robots in Urban Search and Rescue (USAR) scenarios. Existing approaches to dynamic object detection often rely on pretrained learned networks or computationally expensive volumetric maps. To enhance efficiency on computationally limited robots, we reuse data between the odometry and detection module. Utilizing a range image segmentation technique and a novel residual-based heuristic, our method distinguishes dynamic from static objects before integrating them into the point cloud map. The approach demonstrates robust object tracking and improved map accuracy in environments with numerous dynamic objects. Even highly non-rigid objects, such as running humans, are accurately detected at point level without prior downsampling of the point cloud and hence, without loss of information. Evaluation on simulated and real-world data validates its computational efficiency. Compared to a stateof-the-art volumetric method, our approach shows comparable detection performance at a fraction of the processing time, adding only 14 ms to the odometry module for dynamic object detection and tracking. The implementation and a new realworld dataset are available as open-source for further research.

Fiber jamming modules (FJMs) offer flexibility and quick stiffness variation, making them suitable for followthe-leader (FTL) motions in continuum robots, which is ideal for minimally invasive surgery (MIS). However, their potential has not been fully exploited, particularly in designing and manufacturing small-sized FJMs with high stiffness variation. Although existing research has focused on factors like fiber materials and geometry to maximize stiffness variation, the results often do not apply to FJMs for MIS due to size constraints. Meanwhile, other factors such as fiber number and packing density, less significant to large FJMs but critical to smallsized FJMs, have received insufficient investigation regarding their impact on the stiffness variation for FTL deployment. In this paper, we design and fabricate FJMs with a diameter of 4mm. Through theoretical and experimental analysis, we find that fiber number and packing density significantly affect both absolute stiffness and stiffness variation. Our experiments confirm the feasibility of using FJMs in a medical FTL robot design. The optimal configuration is a 4mm FJM with 0.4mm fibers at a 56% packing density, achieving up to 3400% stiffness variation. A video demonstration of a prototype robot using the suggested parameters for achieving FTL motions can be found at https://youtu.be/7pI5U0z7kcE.

Hand-eye calibration plays a fundamental role in robotics by directly influencing the efficiency of critical operations such as manipulation and grasping. In this work, we present a novel framework, EasyHeC++, designed for fully automatic hand-eye calibration. In contrast to previous methods that necessitate manual calibration, specialized markers, or the training of arm-specific neural networks, our approach is the first system that enables accurate calibration of any robot arm in a marker-free, training-free, and fully automatic manner. Our approach employs a two-step process. First, we initialize the camera pose using a sampling or feature-matching-based method with the aid of pretrained image models. Subsequently, we perform pose optimization through differentiable rendering. Extensive experiments demonstrate the system’s superior accuracy in both synthetic and real-world datasets across various robot arms and camera settings. Project page: https://ootts.github.io/easyhec_plus/.

Despite significant progress in perception tasks such as 3D scene mapping and semantic information extraction using SLAM and deep learning, applying these techniques within computationally constrained embedded systems remains a challenge. In this work, we introduce a novel end-to-end framework for efficient and real-time volumetric-semantic mapping. We have developed a lightweight and robust RGB-D segmentation network for extracting semantic information. Through the introduction of three distinct modules—CFIM, DAPPF, and LAD—our network significantly enhances real-time performance while achieving Mean Intersection over Union (MIoU) scores comparable to state-of-the-art (SOTA) models. Our model reduces the parameters by 8 to 26 times compared to similar networks and improves inference speed by 2 to 3 times. Additionally, we improved a multi-class bayesian updating strategy by refining penalty function to reduce the memory size of the semantic map and enhance the mapping speed. Compared with other volumetric-semantic mapping approaches, our work maintains the same level of detail in semantic information representation, while increasing mapping speed by 1.3 to 9.6 times and reducing memory size of the map by up to 2.6 times. Finally, we applied our work to real-world mobile robot exploration scenarios, demonstrating the efficiency of the proposed framework.

Event cameras are an interesting visual exteroceptive sensor that reacts to brightness changes rather than integrating absolute image intensities. Owing to this design, the sensor exhibits strong performance in situations of challenging dynamics and illumination conditions. While event-based simultaneous tracking and mapping remains a challenging problem, a number of recent works have pointed out the sensor’s suitability for prior map-based tracking. By making use of cross-modal registration paradigms, the camera’s ego-motion can be tracked across a large spectrum of illumination and dynamics conditions on top of accurate maps that have been created a priori by more traditional sensors. The present paper follows up on a recently introduced event-based geometric semi-dense tracking paradigm, and proposes the addition of inertial signals in order to robustify the estimation. More specifically, the added signals provide strong cues for pose initialization as well as regularization during windowed, multi-frame tracking. As a result, the proposed framework achieves increased performance under challenging illumination conditions as well as a reduction of the rate at which intermediate event representations need to be registered in order to maintain stable tracking across highly dynamic sequences. Our evaluation focuses on a diverse set of real world sequences and comprises a comparison of our proposed method against a purely event-based alternative running at different rates.

Simultaneous Localization and Mapping (SLAM) is widely used in applications such as robotics and autonomous driving, with methods involving multi-sensor fusion demonstrating excellent performance. However, they simply reject dynamic features and ignore the mutual benefits of self and dynamic objects, which greatly limits their application in actual high-dynamic scenes. To address this issue, we propose ESO-SLAM, a tightly-coupled system for simultaneous self and multi-object pose estimation achieved through sensor fusion. This system employs a multi-probability fusion tracker based on filter to establish more robust object-level data association. Building upon this, we introduce a method that combines 3D Kalman filter velocity priors and camera optical flow decoupling for dynamic point cloud removal, aiming at improving the accuracy of self-pose estimation in odometry. Finally, we jointly refine the poses of the robot and objects using multiple constraint factors within our proposed framework. Experimental results on the KITTI raw dataset demonstrate that our approach achieves better pose accuracy for both self and tracked objects compared to baseline and state-of-the-art techniques. Furthermore, the proposed method exhibits feasibility in real-time performance to ensure its practical application value.

Task planning for robots in real-life settings presents significant challenges. These challenges stem from three primary issues: the difficulty in identifying grounded sequences of steps to achieve a goal; the lack of a standardized mapping between high-level actions and low-level commands; and the challenge of maintaining low computational overhead given the limited resources of robotic hardware. We introduce EMPOWER, a framework designed for open-vocabulary online grounding and planning for embodied agents aimed at addressing these issues. By leveraging efficient pre-trained foundation models and a multi-role mechanism, EMPOWER demonstrates notable improvements in grounded planning and execution. Quantitative results highlight the effectiveness of our approach, achieving an average success rate of 0.73 across six different real-life scenarios using a TIAGo robot.

3D Embodied Spatial Reasoning, emphasizing an agent’s interaction with its surroundings for spatial information inference, is adeptly facilitated by the process of Situated Question Answering in 3D Scenes (SQA3D). SQA3D requires an agent to comprehend its position and orientation within a 3D scene based on a textual situation and then utilize this understanding to answer questions about the surrounding environment in that context. Previous methods in this field face substantial challenges, including a dependency on constant retraining on limited datasets, which leads to poor performance in unseen scenarios, limited expandability, and inadequate generalization. To address these challenges, we present a new embodied spatial reasoning paradigm for enhanced SQA3D, fusing the capabilities of foundation models with the chain of thought methodology. This approach is designed to elevate adaptability and scalability in a wide array of 3D environments. A new aspect of our model is the integration of a chain of thought reasoning process, which significantly augments the model’s capability for spatial reasoning and complex query handling in diverse 3D environments. In our structured experiments, we compare our approach against other methods with varying architectures, demonstrating its efficacy in multiple tasks including SQA3D and 3D captioning. We also assess the informativeness contained in the generated answers for complex queries. Ablation studies further delineate the individual contributions of our method to its overall performance. The results consistently affirm our proposed method’s effectiveness and efficiency.

This paper presents Ego-Centric Intersection-over-Union (EC-IoU), addressing the limitation of the standard IoU measure in characterizing safety-related performance for object detectors in navigating contexts. Concretely, we propose a weighting mechanism to refine IoU, allowing it to assign a higher score to a prediction that covers closer points of a ground-truth object from the ego agent’s perspective. The proposed EC-IoU measure can be used in typical evaluation processes to select object detectors with better safety-related performance for downstream tasks. It can also be integrated into common loss functions for model fine-tuning. While geared towards safety, our experiment with the KITTI dataset demonstrates the performance of a model trained on EC-IoU can be better than that of a variant trained on IoU in terms of mean Average Precision as well.

We address the unique challenge of vibration suppression for a flexible long-reach robotic manipulator system, namely, the through-wall deployment (TWD) system that is used in nuclear environments. This paper proposes a novel dynamics-based trajectory optimization approach, which minimizes both the acceleration and the jerk at the manipulator’s joints, as well as the vibrations of the flexible long-reach boom where the manipulator’s base is mounted. Firstly, we create an integrated model for the system dynamics based on the knowledge of the robotic manipulator and the acceleration data from the vibration tests. We then develop an original procedure for generating the high-order polynomial trajectory that guarantees the zero-boundary condition for a flexible number of optimization parameters and waypoints. Following the simulation of a multi-objective optimization scheme, the optimized trajectory is experimentally validated on the practical TWD system with around 28% vibration reduction on average compared to the benchmark. Importantly, this reduction is achieved without compromising on the average speed of motion. The methodology is transferable to a wider range of flexible robotic manipulator systems with similar characteristics.

Mobile robots and autonomous vehicles are often required to function in environments where critical position estimates from sensors such as GPS become uncertain or unreliable. Single image visual place recognition (VPR) provides an alternative for localization but often requires techniques such as sequence matching to improve robustness, which incurs additional computation and latency costs. Even then, the sequence length required to localize at an acceptable performance level varies widely; and simply setting overly long fixed sequence lengths creates unnecessary latency, computational overhead, and can even degrade performance. In these scenarios it is often more desirable to meet or exceed a set target performance at minimal expense. In this paper we present an approach which uses a calibration set of data to fit a model that modulates sequence length for VPR as needed to exceed a target localization performance. We make use of a coarse position prior, which could be provided by any other localization system, and capture the variation in appearance across this region. We use the correlation between appearance variation and sequence length to curate VPR features and fit a Multi-Layer Perceptron (MLP) for selecting the optimal length. We demonstrate that this method is effective at modulating sequence length to maximize the number of sections in a dataset which meet or exceed a target performance whilst minimizing the median length used. We show applicability across several datasets and reveal key phenomena like generalization capabilities, the benefits of curating features and the utility of non-state-of-the-art feature extractors with nuanced properties.