IROS 2024 论文列表

Navigating toward specific objects in unknown environments without additional training, known as Zero-Shot object navigation, poses a significant challenge in the field of robotics, which demands high levels of auxiliary information and strategic planning. Traditional works have focused on holistic solutions, overlooking the specific challenges agents encounter during navigation such as collision, low exploration efficiency, and misidentification of targets. To address these challenges, our work proposes TriHelper, a novel framework designed to assist agents dynamically through three primary navigation challenges: collision, exploration, and detection. Specifically, our framework consists of three innovative components: (i) Collision Helper, (ii) Exploration Helper, and (iii) Detection Helper. These components work collaboratively to solve these challenges throughout the navigation process. Experiments on the Habitat-Matterport 3D (HM3D) and Gibson datasets demonstrate that TriHelper significantly outperforms all existing baseline methods in Zero-Shot object navigation, showcasing superior success rates and exploration efficiency. Our ablation studies further underscore the effectiveness of each helper in addressing their respective challenges, notably enhancing the agent’s navigation capabilities. By proposing TriHelper, we offer a fresh perspective on advancing the object navigation task, paving the way for future research in the domain of Embodied AI and visual-based navigation.

Metamorphic robots have gained the attention of many researchers due to their ability to change shape and adapt to various tasks. In order to utilize the versatility of metamorphic systems, we need to be able to find a shape-shifting (reconfiguration) plan efficiently; however, finding these plans is challenging due to the high degree of freedom of modular systems. Reconfiguration algorithms proposed so far either scale poorly with a growing number of modules, impose specific restrictions on modules, or produce plans that are unrealistic outside of zero-gravity environments. This paper presents a new approach to the reconfiguration problem of chain-type metamorphic robots. Our algorithm relies on forming tentacles and using them to transport modules, which allows us to search through a reduced state space by computing many smaller planning instances. As a result, we obtain a heuristic solution that is more scalable than optimal planners, while producing realistic plans that impose no specific module requirements.

Lower Limb Exoskeletons (LLEs) are wearable robotic systems that provide mechanical power to the user. Human-exoskeleton (HE) connections must guarantee the subsistence of the user’s natural behavior during the interaction, avoiding the exertion of undesired forces, i.e., the robot must be transparent. Since transparency is an essential feature of exoskeletons’ design, numerous works focus on its maximization, e.g., employing passive joints at the HE interfaces. Given the inherent complications of repeatedly prototyping and experimentally testing a device, modeling the exoskeleton and its physical interaction with the user emerges as an extremely valuable approach for assessing the design effects. This paper proposes a novel method to compare different exoskeleton configurations with a flexible simulation tool. This approach contemplates simulating the dynamics of the device, including its interaction with the wearer, to evaluate multiple connection mechanism designs along with the kinematics and actuation of the LLE. This evaluation is based on the minimization of the interaction wrenches through an optimization process that includes the impedance parameters at the interfaces as optimization variables and the similarity of the LLE’s joint variables trajectories with the motion of the wearer’s articulations. Exploratory tests are conducted using the Wearable Walker LLE in different configurations and measuring the interaction forces. Experimental data are then compared to the optimization outcomes, proving that the proposed method provides contact wrench estimations consistent with the collected measurements and previous outcomes from the literature.

Humans are remarkably adept at imitating other people performing tasks, afforded by their ability to abstract away irrelevant details and focus on the task strategy of the demonstrator. In this paper, we take steps towards enabling robots with this ability, and present a framework, TransAct to do so. TransAct first builds on prior skill learning work to learn temporally abstract representations of common agent-environment interactions in manipulation tasks, e.g., a robot pouring from a cup. Given a human demonstration of an unseen unknown task, TransAct then translates the underlying sequence of interactions (i.e., the human task strategy) to a robot learner. Through experiments on real-world human and robot datasets, we demonstrate TransAct’s ability to accurately represent diverse agent-environment interactions. Moreover, TransAct empowers robots to consume human task demonstrations and compose corresponding interactions with similar environmental effects to perform the tasks themselves in a zero shot manner, without access to paired demonstrations or dense annotations. We present visualizations of our results at https://sites.google.com/view/interaction-abstractions.

In domestic environments, the conventional organization of objects by service robots often relies on the inherent properties of each object, such as placing fragile bowls in enclosed cupboards. However, this approach tends to overlook the importance of the orderly arrangement of objects, neglecting the specific placement order of bowls within the cabinet. In practice, effective object organization necessitates consideration of both individual properties and the relationships defined by these properties. In this paper, we have constructed a specialized dataset encompassing the ontological properties of household objects along with their relationships. Furthermore, we have introduced a graph-based model to explicitly represent these relationships and proposed a novel feature extraction technique that integrates the Graph Attention Network (GAT) with the BERT model to predict the relationships among objects. Subsequently, we utilized the Transformer framework to train a model, enabling it to infer relationships between objects. Experimental validation demonstrates the effectiveness of our approach in accurately predicting relationships between household objects, thus facilitating their orderly organization. Our approach significantly augments the object organization capabilities for service robots by accurately predicting the relationships among household objects. Our code is available at: https://github.com/Li-XD-Pro/Household-Object-Organization

If robots are to work effectively alongside people, they must be able to interpret natural language references to objects in their 3D environment. Understanding 3D referring expressions is challenging—it requires the ability to both parse the 3D structure of the scene and correctly ground free-form language in the presence of distraction and clutter. We introduce Transcrib3D, an approach that brings together 3D detection methods and the emergent reasoning capabilities of large language models (LLMs). Transcrib3D uses text as the unifying medium, which allows us to sidestep the need to learn shared representations connecting multi-modal inputs, which would require massive amounts of annotated 3D data. As a demonstration of its effectiveness, Transcrib3D achieves state-of-the-art results on 3D reference resolution benchmarks, with a great leap in performance from previous multi-modality baselines. To improve upon zero-shot performance and facilitate local deployment on edge computers and robots, we propose self-correction for fine-tuning that trains smaller models, resulting in performance close to that of large models. We show that our method enables a real robot to perform pick-and-place tasks given queries that contain challenging referring expressions. Code will be available at https://ripl.github.io/Transcrib3D.

Aerial-ground vehicles have shown great potential in various fields due to their superior mobility and outstanding endurance. However, most of morphing aerial-ground vehicles consider little about controllability and traversability in ground mode. We present a novel aerial-ground vehicle called TransRotor. By proposing a differential gear joint, we equip TransRotor with omnidirectional mobility in both air and ground mode. Besides, using a four-wheel-steering model in ground mode provides better traversability and ground flexibility. Moreover, we design mid-mode transformation for Trans-Rotor, which provides smooth and rapid mode switching. In this work, we firstly propose a novel design of an aerial-ground vehicle. Then, we propose a decoupled controller considering the four-wheel-steer model to achieve autonomous navigation of the vehicle. Comprehensive experiments and a benchmark comparison are carried out to validate the outstanding performance of the proposed system, where the system shows ground flexibility and saves energy up to more than 95%.

Non-prehensile transportation of unstable objects presents a challenging task in robotics. To ensure the success of the transportation, it is necessary to consider both the object’s stability via contact dynamics and the motion constraints of the robot. We propose two novel trajectory planning methods derived from sampling and dynamic programming algorithms, tested on a 7-DoF Franka Emika robot against common strategies like Model Predictive Control (MPC) and S-curve planning, particularly under the constraint of a non-rotating tray. The results demonstrate the effectiveness of our methodologies in improving transportation speed. This research contributes to advancements in robotic manipulation techniques by tackling non-prehensile manipulation of dynamically unstable objects.

Trajectory generation for quadrotors with limited field-of-view sensors has numerous applications such as aerial exploration, coverage, inspection, videography, and target tracking. Most previous works simplify the task of optimizing yaw trajectories by either aligning the heading of the robot with its velocity, or potentially restricting the feasible space of candidate trajectories by using a limited yaw domain to circumvent angular singularities. In this paper, we propose a novel global yaw parameterization method for trajectory optimization that allows a 360-degree yaw variation as demanded by the underlying algorithm. This approach effectively bypasses inherent singularities by including supplementary quadratic constraints and transforming the final decision variables into the desired state representation. This method significantly reduces the needed control effort, and improves optimization feasibility. Furthermore, we apply the method to several examples of different applications that require jointly optimizing over both the yaw and position trajectories. Ultimately, we present a comprehensive numerical analysis and evaluation of our proposed method in both simulation and real-world experiments.

To track tumors during surgery, information from preoperative CT scans is used to determine their position. However, as the surgeon operates, the tumor may be deformed which presents a major hurdle for accurately resecting the tumor, and can lead to surgical inaccuracy, increased operation time, and excessive margins. This issue is particularly pronounced in robot-assisted partial nephrectomy (RAPN), where the kidney undergoes significant deformations during operation. Toward addressing this, we introduce a occupancy network-based method for the localization of tumors within kidney phantoms undergoing deformations at interactive speeds. We validate our method by introducing a 3D hydrogel kidney phantom embedded with exophytic and endophytic renal tumors. It closely mimics real tissue mechanics to simulate kidney deformation during in vivo surgery, providing excellent contrast and clear delineation of tumor margins to enable automatic threshold-based segmentation. Our findings indicate that the proposed method can localize tumors in moderately deforming kidneys with a margin of 6mm to 10mm, while providing essential volumetric 3D information at over 60Hz. This capability directly enables downstream tasks such as robotic resection.

Aggressive maneuvering is crucial for aerial vehicles to execute adversarial and penetration missions. However, this challenges the accurate tracking control of drones due to uncertainties induced by high-speed flight. Therefore, firstly, a highly dynamic tracking control framework is proposed to actualize the accurate tracking of aggressive trajectories with velocities up to 15 m/s (i.e., 54 km/h) and acceleration of 2 g. Secondly, in order to mitigate the impact of conjoint effects on uncertainty estimation during aggressive flights and to ensure that uncertainty is smoothly compensated, a novel adaptive nonlinear extended state observer (ANESO) with noise suppression and peak attenuation capabilities is designed. Finally, extensive comparative simulation and real-world practical experimental results certify the superiority of the proposed control strategy in tracking aggressive trajectories.

The burgeoning demand for home-care services, driven by a rapidly aging global population, necessitates innovative solutions to alleviate the burden on caregivers and enhance care quality. This paper introduces the development of an Inter-net of Human and Robotic Things (IoHRT) framework, which synergizes cloud computing and the Internet of Robotic Things (IoRT) with human-robot collaborative control mechanisms for home-care applications. The IoHRT framework is designed to enable the seamless integration of customizable robotic platforms with modular, scalable, and compatible features, thereby creating a dynamic and adaptable home-care ecosystem. By leveraging the scalability and computational power of cloud computing, the framework facilitates real-time data analysis and remote monitoring, thus enhancing the efficiency and effectiveness of home-care. We present an in-depth analysis of the key characteristics of IoHRT, supported by evidence embedded in our design, and conduct user studies to evaluate the framework from users’ perspectives. We demonstrate the performance and utility of our proposed framework for the future of home-care applications.

Today, intelligent robotic manufacturing systems are reshaping the production industry. Using robots as actuators, multi-source sensors for perception, and Artificial Intelligence (AI) as decision-making systems, they can perform routine manufacturing tasks, surpassing the capabilities of traditional hard-programmed Computer Numerical Control (CNC) machinery. One specific challenge in footwear manufacturing is sole deburring, traditionally done manually by skilled workers. This paper focuses on developing a robust path-planning pipeline, comprising vision-based and Learning from Demonstrations (LfD) modules for autonomous deburring of soles. The vision-based module exploits Deep Learning (DL) techniques to handle key challenges such as precise segmentation of different soles types across diverse scenarios despite potential occlusions. Additionally, a novel method for burrs identification has been developed leveraging image processing and optimization techniques. Determining the optimal cutting tool orientation during sole deburring relies on human experience. The LfD module aims to impart this knowledge to the robot from videos of expert demonstrations, requiring adaptability to every new incoming sole that needs deburring. Experimental results showcase the method’s performance and flexibility, underlining the potential to advance the field of the proposed approach.

Robot-assisted surgical systems have demonstrated significant potential in enhancing surgical precision and minimizing human errors. However, existing systems cannot accommodate individual surgeons’ unique preferences and requirements. Additionally, they primarily focus on general surgeries (e.g., laparoscopy) and are unsuitable for highly precise microsurgeries, such as ophthalmic procedures. Thus, we propose an image-guided approach for surgeon-centered autonomous agents that can adapt to the individual surgeon’s skill level and preferred surgical techniques during ophthalmic cataract surgery. Our approach trains reinforcement and imitation learning agents simultaneously using curriculum learning approaches guided by image data to perform all tasks of the incision phase of cataract surgery. By integrating the surgeon’s actions and preferences into the training process, our approach enables the robot to implicitly learn and adapt to the individual surgeon’s unique techniques through surgeon-in-the-loop demonstrations. This results in a more intuitive and personalized surgical experience for the surgeon while ensuring consistent performance for the autonomous robotic apprentice. We define and evaluate the effectiveness of our approach in a simulated environment using our proposed metrics and highlight the trade-off between a generic agent and a surgeon-centered adapted agent. Finally, our approach has the potential to extend to other ophthalmic and microsurgical procedures, opening the door to a new generation of surgeon-in-the-loop autonomous surgical robots. We provide an open-source simulation framework for future development and reproducibility at https://github.com/amrgomaaelhady/CataractAdaptSurgRobot.

MEMS Attitude Heading Reference Systems are widely employed to determine a system’s attitude, but sensor measurement biases limit their accuracy. This paper introduces a novel factor graph-based method called MAgnetometer and GYroscope Calibration (MAGYC). MAGYC leverages three-axis angular rate measurements from an angular rate gyroscope to enhance calibration for batch and online applications. Our approach imposes less restrictive conditions for instrument movements required for calibration, eliminates the need for knowledge of the local magnetic field or instrument attitude, and facilitates integration into factor graph algorithms within Smoothing and Mapping frameworks. We evaluate the proposed methods through numerical simulations and in-field experimental assessments using a sensor installed on an underwater vehicle. Ultimately, our proposed methods reduced the underwater vehicle’s heading error standard deviation from 6.21° to 0.57° for a standard seafloor mapping survey.

This paper introduces a new method for online estimating the penetration of the end-effector and the viscoelastic properties of a soft body, through palpation exams using a collaborative robotic arm. The estimator is based on the dimensionality reduction method that simplifies the nonlinear Hunt-Crossley model. In addition, in our algorithm, the model parameters can be found without a force sensor, leveraging only the robotic arm controller data. An extended Kalman filter is employed to achieve online estimation, which embeds the dynamic contact model. The algorithm is tested with various types of silicone, a material that resembles biological tissues, including samples with hard intrusions to simulate cancerous cells within a softer tissue. The results indicate that this technique can accurately determine the model parameters and estimate the penetration of the end-effector into the soft body. These promising preliminary results demonstrate robots’ potential to be an effective tool for early-stage cancer diagnostics.

Visual SLAM with thermal imagery remains a difficult problem for many state of the art (SOTA) algorithms. Compared with visible spectrum imagery, thermal imagery generally has lower contrast, higher noise, and tends to have lower resolution, making for challenging front-end data association. Thermal imagery also presents a difficult problem for long term relocalization and map reuse, because the relative temperatures of objects in thermal imagery tend to change dramatically from day to night. Feature descriptors typically used for relocalization in SLAM are unable to maintain consistency over these diurnal changes. We show that learned feature descriptors can be used within existing bag of word based localization schemes to dramatically improve place recognition across large temporal gaps in thermal imagery. In order to demonstrate the effectiveness of our trained vocabulary, we have developed a baseline SLAM system, integrating learned features and matching into a classical SLAM algorithm. Our system demonstrates good local tracking on challenging thermal imagery, and relocalization that overcomes dramatic day to night thermal appearance changes. Our code and datasets are available here: https://github.com/neufieldrobotics/IRSLAM_Baseline

In the dynamic construction industry, traditional robotic integration has primarily focused on automating specific tasks, often overlooking the complexity and variability of human aspects in construction workflows. This paper introduces a human-centered approach with a "work companion rover" designed to assist construction workers within their existing practices, aiming to enhance safety and workflow fluency while respecting construction labor’s skilled nature. We conduct an in-depth study on deploying a robotic system in carpentry formwork, showcasing a prototype that emphasizes mobility, safety, and comfortable worker-robot collaboration in dynamic environments through a contextual Reinforcement Learning (RL)-driven modular framework. Our research advances robotic applications in construction, advocating for collaborative models where adaptive robots support rather than replace humans and underscores the potential for an interactive and collaborative human-robot workforce.

Miniature rotation actuators have been extensively developed and utilized in optical coherence tomography (OCT) endoscopy, enabling distortion-free OCT imaging in complex and tortuous environments. However, the use of electrical-driven rotation actuators raises safety concerns. Although magnetic-driven rotation actuators have been reported in OCT endoscopy, their use can potentially interfere with other medical devices in clinical settings. Here, we propose a pneumatic miniature rotation actuator that eliminates the electricity and magnetism concerns in circumferential imaging for OCT endoscopy. The rotor of the actuator is designed as a windmill, enabling it to convert air energy into rotation energy. In addition, to maintain the stable rotation, both a sliding bearing with two supporting points and a glass spindle with a half-ball end surface are developed. The rotation speed of our pneumatic actuator can be controlled from 66 to 97 revolutions per second by adjusting the airflow rate from 3.25 to 4.00 liters per minute. By OCT imaging of the human fingers, we demonstrate the feasibility of the pneumatic actuator in electricity-free distal scanning OCT endoscopy. Our pneumatic rotation actuator has wide-ranging potential in various fiber-imaging modalities, including not only OCT but also ultrasound imaging that requires similar rotation capabilities.