IROS 2024 论文列表

Wire-harnessing tasks pose great challenges to be automated by the robot due to the complex dynamics and unpredictable behavior of the deformable wire. Traditional methods, often reliant on dual-robot arms or tactile sensing, face limitations in adaptability, cost, and scalability. This paper introduces a novel single-robot wire-harnessing pipeline that leverages a robot’s twisting motion to generate necessary wire tension for precise insertion into clamps, using only one robot arm with an integrated force/torque (F/T) sensor. Benefiting from this design, the single robot arm can efficiently apply tension for wire routing and insertion into clamps in a narrow space. Our approach is structured around four principal components: a Model Predictive Control (MPC) based on the Koopman operator for tension tracking and wire following, a motion planner for sequencing harnessing waypoints, a suite of insertion primitives for clamp engagement, and a fix-point switching mechanism for wire constraint updating. Evaluated on an industrial-level wire harnessing task, our method demonstrated superior performance and reliability over conventional approaches, efficiently handling both single and multiple wire configurations with high success rates.

Soft robotics, an emerging domain in modern robotics, introduces innovative possibilities alongside challenges in controllability, particularly with multi-degree inflatable actuators. We present a novel manipulation method using underactuated soft fingers that addresses these challenges by harnessing symmetry breaking. Central to our approach is the mechanism of self-organization within a ring actuator equipped with five fingers. Typically considered a drawback, we exploit the actuator’s buckling behavior to facilitate in-hand manipulation. This strategic utilization enables object motion in both clockwise and counterclockwise directions via system perturbations and adjustments in frequency and duty cycle parameters. Employing the self-organizing properties of our actuator, our method is empirically validated through simulations and real actuator experiments, demonstrating the system’s ability in manipulating objects by leveraging the inherent flexibility and morphological advantages. The design enables two degrees of freedom with minimal input, allowing objects to rotate due to the actuator’s self-organizing actions. This simplification of control mechanisms is essential for soft robotics manipulation. Our findings indicate that control systems in soft robotics can be significantly simplified, harnessing the adaptable behavior inherent in its morphology.

This study explores the dynamics of asymmetrical bounding gaits in quadrupedal robots, focusing on the integration of torso pitching and hip motion to enhance speed and stability. Traditional control strategies often enforce a fixed posture, minimizing natural body movements to simplify the control problem. However, this approach may overlook the inherent dynamical advantages found in natural locomotion. By considering the robot as two interconnected segments, we concentrate on stance leg motion while allowing passive torso oscillation, drawing inspiration from natural dynamics and underactuated robotics principles. Our control scheme employs Linear Inverted Pendulum (LIP) and Spring-Loaded Inverted Pendulum (SLIP) models to govern front and rear leg movements independently. This approach has been validated through extensive simulations and hardware experiments, demonstrating successful high-speed locomotion with top speeds nearing 4 m/s and reduced ground reaction forces, indicating a more efficient gait. Furthermore, unlike conventional methods, our strategy leverages natural torso oscillations to aid leg circulation and stride length, aligning robot dynamics more closely with biological counterparts. Our findings suggest that embracing the natural dynamics of quadrupedal movement, particularly in asymmetrical gaits like bounding, can lead to more stable, efficient, and high-speed robotic locomotion. This investigation lays the groundwork for future studies on versatile and dynamic quadrupedal gaits and their potential applications in scenarios demanding rapid and effective locomotion.

Recent advancements in robotics have enabled robots to navigate complex scenes or manipulate diverse objects independently. However, robots are still impotent in many household tasks requiring coordinated behaviors such as opening doors. The factorization of navigation and manipulation, while effective for some tasks, fails in scenarios requiring coordinated actions. To address this challenge, we introduce, HarmonicMM, an end-to-end learning method that optimizes both navigation and manipulation, showing notable improvement over existing techniques in everyday tasks. This approach is validated in simulated and real-world environments and adapts to novel unseen settings without additional tuning. Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation in a real unseen apartment, demonstrating the potential for practical indoor robot deployment in daily life.

This work consists of two main components: designing a hardware-software co-design, MT+, for adapting the Mikami-Tabuchi algorithm for on-board path planning by drones in a 3D environment; and development of a specialized custom hardware accelerator CDU, as a part of MT+, for parallel collision detection. Collision detection is a performance bottleneck in path planning. MT+ reduces the delay in path planning without using any heuristic. A comparative analysis between the state-of-the-art path planning algorithm A* and Mikami-Tabuchi is performed to show that Mikami-Tabuchi is faster than A* in typical real-world environments. In custom-generated environments, path planning using Mikami-Tabuchi shows a latency improvement of 1.7× across varying average sizes of obstacles and 2.7× across varying obstacle density over state-of-the-art path planning algorithm, A*. Further, the experiments show that the co-design achieves speedups over a full software implementation on CPU, averaging between 10% to 60% across different densities and sizes of obstacles. CDU area and power overheads are negligible against a conventional single-core processor.

Accurate time synchronization is crucial for multisensor fusion, which is widely used in mobile robotics, autonomous driving, and virtual reality. Despite many advancements, precise multi-sensor synchronization is still challenging due to the sensors’ internal characteristics, data filtering, disjointed clock reference, and transmission delay caused by operation system scheduling. This paper proposes a novel hardware-based synchronization solution to achieve synchronization in microsecond-level precision. By introducing a Sensor Adaptor board that provides a unified clock reference, the proposed hardware architecture enables high-precision synchronization across multiple sensors. Furthermore, we develop a method for Visual-Inertial time synchronization that actively controls the exposure duration using an ambient light sensor. By managing the IMU clock signal and exposure trigger, we align the camera’s sampling moment with the authentic IMU sampling time and significantly reduce the time discrepancy in the Visual-Inertial system. Experiments are conducted to evaluate the efficiency of the proposed method and system, including comparisons with previous work. The results indicate that our method can achieve precise time synchronization and be successfully implemented in multi-sensor systems.

The Smart Suction Cup is a tactile sensing and gripping system designed to enhance pick-and-place operations in industrial settings. While previous research has primarily focused on utilizing this technology for haptic search in cases of initial grasp failure, this study introduces a novel application: following contours. This function is already established as an important function for object recognition and grasp planning – substantiated by numerous works using other tactile sensors. Here, we explore contour following for a flow-based tactile sensor because it is not susceptible to visual occlusions nor tactile sensor wear. Experimental validation demonstrates the Smart Suction Cup’s ability to track edges at different speeds and navigate various planar contours, showcasing rapid and robust tracking of edges. Notably, the Smart Suction Cup can reliably operate at a speed of 3 cm/s. This is one step towards the adoption of the Smart Suction Cup for real-world applications.

Visual navigation, a foundational aspect of Embodied AI (E-AI) and robotics has been extensively studied in the past few years. While many 3D simulators have been introduced for the visual navigation tasks, scarcely works have combined human dynamics, creating the gap between simulation and real-world applications. Furthermore, current 3D simulators incorporating human dynamics have several limitations, particularly in terms of computational efficiency, which is a promise of modern simulators. To overcome these issues, we introduce HabiCrowd, the new standard benchmark for crowd-aware visual navigation that includes a crowd dynamics model with diverse human settings into photorealistic environments. Empirical evaluations demonstrate that our proposed human dynamics model achieves state-of-the-art performance in collision avoidance while exhibiting superior computational efficiency compared to its counterparts. We leverage HabiCrowd to conduct several comprehensive studies on crowd-aware visual navigation tasks and human-robot interactions. The source code and data can be found at https://habicrowd.github.io/.

The advancement of object detection algorithms has catalyzed the development of object-level semantic SLAM. However, due to missed and false detections, object-level semantic SLAM fails to represent the objects within the scene adequately. Therefore, this paper proposes a novel object-level semantic SLAM termed HSS-SLAM. We incorporate human-in-the-loop into our method, establishing an interaction module to facilitate human editing and rectifying semantic information. Additionally, to minimize the manual correction workload, a lightweight and intuitive method for semantic extension is proposed, augmenting the semantic richness of the global map with a few operations. Furthermore, our method adopts superquadrics for object representation, enabling detailed descriptions of various object shapes. This mitigates the limitation of conventional semantic mapping, where objects are difficult to distinguish due to the reliance on a single-shape representation. Subsequently, precise estimation of superquadric parameters and camera poses is achieved through joint optimization. Extensive experiments conducted on TUM RGB-D and Scenes V2 datasets demonstrate that the proposed approach exhibits competitive performance, surpassing current methods in both object representation and camera localization accuracy.

Semantic segmentation is an essential step for many vision applications in order to understand a scene and the objects within. Recent progress in hyperspectral imaging technology enables the application in driving scenarios and the hope is that the devices perceptive abilities provide an advantage over RGB-cameras. Even though some datasets exist, there is no standard benchmark available to systematically measure progress on this task and evaluate the benefit of hyperspectral data. In this paper, we work towards closing this gap by providing the HyperSpectral Semantic Segmentation benchmark (HS3-Bench). It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols. We use the benchmark to derive two strong baseline models that surpass the previous state-of-the-art performances with and without pre-training on the individual datasets. Further, our results indicate that the existing learning-based methods benefit more from leveraging additional RGB training data than from leveraging the additional hyperspectral channels. This poses important questions for future research on hyperspectral imaging for semantic segmentation in driving scenarios. Code to run the benchmark and the strong baseline approaches are available under https://github.com/nickstheisen/hyperseg.

Rapid sampling from the environment to acquire available frontier points and timely incorporating them into subsequent planning to reduce fragmented regions are critical to improve the efficiency of autonomous exploration. We propose HPHS, a fast and effective method for the autonomous exploration of unknown environments. In this work, we efficiently sample frontier points directly from the LiDAR data and the local map around the robot, while exploiting a hierarchical planning strategy to provide the robot with a global perspective. The hierarchical planning framework divides the updated environment into multiple subregions and arranges the order of access to them by considering the overall revenue of the global path. The combination of the hybrid frontier sampling method and hierarchical planning strategy reduces the complexity of the planning problem and mitigates the issue of region remnants during the exploration process. Detailed simulation and real-world experiments demonstrate the effectiveness and efficiency of our approach in various aspects. The source code will be released to benefit the further research1.

Trajectory planning for unprotected left turns poses a significant challenge in autonomous driving. Reinforcement learning (RL) offers potential, but existing methods often rely on scenario-specific state representations, limiting their adaptability. This paper introduces Hierarchical Prediction-Pretrained Planning (HP3), a generalizable hierarchical RL framework designed for unprotected left turns. HP3 leverages historical trajectories of all vehicles and complete map information to achieve versatile state representation and generalizable scene understanding. Its two-layer architecture predicts semantic behavior (upper layer) and generates corresponding trajectories (lower layer). A scene encoder comprehends trajectories and roads, while a trajectory decoder outputs sequential points. To accelerate convergence, we pretrain the main network on a modified trajectory prediction dataset. Evaluation on a CARLA-based map with complex, unprotected left-turn intersections demonstrates HP3’s superiority over rule-based and simple RL-based methods, highlighting the effectiveness of our pretraining approach for this critical autonomous driving task.

We present the Habitat-Matterport 3D Open Vocabulary Object Goal Navigation dataset (HM3D-OVON), a large-scale benchmark that broadens the scope and semantic range of prior Object Goal Navigation (ObjectNav) benchmarks. Leveraging the HM3DSem dataset, HM3D-OVON incorporates over 15k annotated instances of household objects across 379 distinct categories, derived from photo-realistic 3D scans of real-world environments. In contrast to earlier ObjectNav datasets, which limit goal objects to a predefined set of 6-21 categories, HM3D-OVON facilitates the training and evaluation of models with an open-set of goals defined through free-form language at test-time. Through this open-vocabulary formulation, HM3D-OVON encourages progress towards learning visuo-semantic navigation behaviors that are capable of searching for any object specified by text in an open-vocabulary manner. Additionally, we systematically evaluate and compare several different types of approaches on HM3D-OVON. We find that HM3D-OVON can be used to train an open-vocabulary ObjectNav agent that achieves both higher performance and is more robust to localization and actuation noise than the state-of-the-art ObjectNav approach. We hope that our benchmark and baseline results will drive interest in developing embodied agents that can navigate real-world spaces to find household objects specified through free-form language, taking a step towards more flexible and human-like semantic visual navigation. Code and videos available at: naoki.io/ovon.

Relative localization is crucial for multi-robot systems to perform cooperative tasks, especially in GPS-denied environments. Current techniques for multi-robot relative localization rely on expensive or short-range sensors such as cameras and LIDARs. As a result, these algorithms face challenges such as high computational complexity (e.g., map merging), dependencies on well-structured environments, etc. To remedy this gap, we propose a new distributed approach to perform relative localization (RL) using a common Access Point (AP). To achieve this efficiently, we propose a novel Hierarchical Gaussian Processes (HGP) mapping of the Radio Signal Strength Indicator (RSSI) values from a Wi-Fi AP to which the robots are connected. We termed this approach as HGP-RL (Hierarchical Gaussian Process for Relative Localization). Each robot performs hierarchical inference using the HGP map to locate the AP in its reference frame, and the robots obtain relative locations of the neighboring robots leveraging AP-oriented algebraic transformations. The approach readily applies to resource-constrained devices and relies only on the ubiquitously-available WiFi RSSI measurement. We extensively validate the performance of the proposed HGP-RL in Robotarium simulations against several state-of-the-art methods. The results indicate superior performance of HGP-RL regarding localization accuracy, computation, and communication over-heads. Finally, we showcase the utility of HGP-RL through a multi-robot cooperative experiment to achieve a rendezvous task in a team of three mobile robots.

Model-free reinforcement learning (RL) is inherently a reactive method, operating under the assumption that it starts with no prior knowledge of the system and entirely depends on trial-and-error for learning. This approach faces several challenges, such as poor sample efficiency, generalization, and the need for well-designed reward functions to guide learning effectively. On the other hand, controllers based on complete system dynamics do not require data. This paper addresses the intermediate situation where there is not enough model information for complete controller design, but there is enough to suggest that a model-free approach is not the best approach either. By carefully decoupling known and unknown information about the system dynamics, we obtain an embedded controller guided by our partial model and thus improve the learning efficiency of an RL-enhanced approach. A modular design allows us to deploy mainstream RL algorithms to refine the policy. Simulation results show that our method significantly improves sample efficiency compared with standard RL methods on continuous control tasks, and also offers enhanced performance over traditional control approaches. Experiments on a real ground vehicle also validate the performance of our method, including generalization and robustness.

In this paper, we propose an adaptive eye-to-hand vision-based control methodology, which enables a closed-loop grow-to-shape capability for variable length continuum manipulators in 2D. Our method utilizes shape features of the continuum robot, i.e. module curvature and length, which are obtained from the image. Our adaptive control algorithm servos the robot to converge and track the desired values of these features in the image space without the need of a robot model. As a result the robot starts from a minimum length configuration and grows into a given desired shape, always staying on the course of the desired shape. We believe that this approach unlocks capabilities for variable length continuum robots by leveraging their actuation redundancy and avoiding obstacles while carrying out object manipulation or inspection tasks in cluttered and constrained environments. We perform experiments in simulations and on a real robot to assess the performance of our visual servoing algorithm. Our experimental results demonstrate the controllers ability to accurately converge the current features to their references, for a variety of desired shapes in the image, while ensuring a smooth tracking response. We also present some proof of concept results demonstrating the effectiveness of this technique for controlling the robot in constrained environments. Markedly, this is the first successful demonstration for automatic grow-to-shape control using visual feedback for variable length continuum manipulators.

The main challenge of Multi-Object Tracking (MOT) lies in maintaining a distinctive identity for each target in dense crowds or occluded scenarios. Although the existing methods have achieved significantly progress by using robust object detectors or complex association strategies, they cannot effectively solve long-term tracking due to individually motion or appearance modeling for each single target. In this paper, we propose a novel 2D MOT tracker GroupTrack, to learn reliable motion state for each target using group motion patterns. Specifically, for each tracklet, we first choose its neighboring ones to form a group of motion patterns, which can provide informative clues for the motion estimation of the current tracklet. Then, we apply the group motion patterns to perform tracklet prediction and data association. By integrating prior from neighboring motion patterns into the data association process, GroupTrack provides a new paradigm for target motion modeling in extremely crowded and occluded scenarios. Through extensive experiments on the public MOT17 and MOT20 datasets, we demonstrate the effectiveness of our approach in challenging scenarios and show state-of-the-art performance at various MOT metrics.

Instance and semantic segmentation form the backbone of robotic perception and are crucial to many tasks. While most research in the area focuses on improving segmentation quality metrics, there are plenty of applications where approximate methods are adequate as long as they are fast, especially in applications with large amounts of data like precision agriculture. In order to apply the recent successes of machine learning and computer vision on a large scale using robotics, efficient and general algorithms must be designed to intelligently split point clouds into small, yet actionable, portions that can then be processed by more complex algorithms. In this paper, we capitalize on a similarity between the current state-of-the-art for roughly segmenting corn plants and a commonly used density-based clustering algorithm, Quickshift. Exploiting this similarity we propose a novel algorithm, Ground-Density Quickshift++, with the goal of producing a general and scalable field segmentation algorithm that segments individual plants and their stems. This algorithm produces quantitatively better results than the current state-of-the-art on both plant separation and stem segmentation while being less sensitive to input parameters and maintaining the same algorithmic time complexity. When incorporated into field-scale phenotyping systems, the proposed algorithm should work as a drop-in replacement that can greatly improve the accuracy of results while ensuring that performance and scalability remain undiminished.

In recent years, the role of robots across industries has become increasingly diverse, and they are now required to perform complex missions beyond simple repetitive tasks. However, robots used in confined spaces that humans cannot reach or in disaster field missions have challenges in performing various tasks due to their small size. In this study, we developed a compact hybrid gripper that fuses a multi-finger gripper and a jamming gripper to perform various tasks in a confined environment. Such a hybrid gripper can have both the strengths of a multi finger gripper that can perform various tasks and a jamming gripper that can effectively handle irregular small objects. In this study, we developed a hybrid gripper "GripFlexer" based on theoretical analysis and confirmed its performance through experiments by taking the task of turning a circular doorknob, which is one of the most difficult tasks in disaster sites, as the final target task. We also confirmed that the two grippers of GripFlexer can interact by showing performance improvement effects when two grippers are operated simultaneously.