IROS 2024 论文列表

In robot-assisted minimally invasive surgery (RAMIS), optimal placement of the surgical robot base is crucial for successful surgery. Improper placement can hinder performance because of manipulator limitations and inaccessible workspaces. Conventional base placement relies on the experience of trained medical staff. This study proposes a novel method for determining the optimal base pose based on the surgeon’s working pattern. The proposed method analyzes recorded end-effector poses using a machine learning-based clustering technique to identify key positions and orientations preferred by the surgeon. We introduce two scoring metrics to address the joint limit and singularity issues: joint margin and manipulability scores. We then train a multi-layer perceptron regressor to predict the optimal base pose based on these scores. Evaluation in a simulated environment using the da Vinci Research Kit shows unique base pose score maps for four volunteers, highlighting the individuality of the working patterns. Results comparing with 20,000 randomly selected base poses suggest that the score obtained using the proposed method is 28.2% higher than that obtained by random base placement. These results emphasize the need for operator-specific optimization during base placement in RAMIS.

In this work, a methodology to find the best view of a grape stem and approach angle in order to crop it is proposed. The control scheme is based only on a classified point cloud obtained by the in-hand camera attached to the robot’s end effector without continuous stem tracking. It is shown that the proposed controller finds and reaches the optimal view point and subsequently the stem fast and efficiently, accelerating the overall harvesting procedure. The proposed control scheme is evaluated through experiments in the lab with a UR5e robot with an in-hand RealSense camera on a mock-up vine.

Multi-agent pathfinding (MAPF) is the problem of finding a set of conflict-free paths for a set of agents. Typically, the agents' moves are limited to a pre-defined graph of possible locations and allowed transitions between them, e.g. a 4-neighborhood grid. We explore how to solve MAPF problems when each agent can move between any pair of possible locations as long as traversing the line segment connecting them does not lead to a collision with the obstacles. This is known as any-angle pathfinding. We present the first optimal any-angle multi-agent pathfinding algorithm. Our planner is based on the Continuous Conflict-based Search (CCBS) algorithm and an optimal any-angle variant of the Safe Interval Path Planning (TO-AA-SIPP). The straightforward combination of those, however, scales poorly since any-angle path finding induces search trees with a very large branching factor. To mitigate this, we adapt two techniques from classical MAPF to the any-angle setting, namely Disjoint Splitting and Multi-Constraints. Experimental results on different combinations of these techniques show they enable solving over 30% more problems than the vanilla combination of CCBS and TO-AA-SIPP. In addition, we present a bounded-suboptimal variant of our algorithm, that enables trading runtime for solution cost in a controlled manner.

Stretchable optical waveguides have been explored as a route to enhancing the sensing capabilities of soft actuators. Certain properties and qualities they possess recommend them for this task – their biological plausibility, compliance, low power consumption, and heightened responsiveness to external stimuli. Though well regarded for their efficiency, their practical application warrants a more detailed examination as regards their sensitivity, robustness, and resilience when integrated with various manipulators. There is a dearth of comprehensive, wide-ranging studies that investigate the relationship between soft sensors and actuators, both in terms of integration and sensor performance – the present study endeavours to fill this void. Here we present a series of findings as to the interdependent relationship at the nexus of soft actuator sensorisation, sensitivity and responsiveness. Building on our previous work and prior waveguide designs, we examine the influence of sensor location and placement along the deformation axis on responsiveness, repeatability, and longevity. Location is key as, during bending, one side experiences tension, while the other compression. Placement is identified as ’straight’ or ’loose’. Three PneuNet-based actuators were used in three design configurations: one without any additional modifications, one with a few rigid exoskeleton reinforcements, and one covered fully with rigid exoskeleton reinforcements. The purpose of applying the exoskeletons is to hold the waveguide-based sensors and to suppress any bubble formation. Each design enables the straightforward integration of sensors, so that the relationship between soft actuator design and sensor performance is easy to assess when applying various pressure intakes (from 0 to 7.1 psi) to actuate the bending motion.

The optimal robotic assembly planning problem entails determining the sequence of actions for a robot to feasibly assemble a product from its components which minimizes a given objective. This problem is made especially challenging as the number of potential sequences increase exponentially with respect to the number of parts in the assembly. Additionally, the optimal sequence must also consider and satisfy a selection of constraints such as attachment precedence or a maximum robot carry weight. Traditionally, robotic assembly planning problems have been solved using heuristics, but these methods are specific to a given robot or cost structure. In this paper, we propose to model robotic assembly planning as a decision-making problem which enables us to use tools from shortest path algorithms and reinforcement learning. We formulate assembly sequencing as a Markov Decision Process and use Dynamic Programming (DP) to find optimal assembly policies that far outperform the state-of-the-art in terms of speed. We further exploit the deterministic nature of assembly planning to introduce a class of optimal Graph Exploration Assembly Planners (GEAPs) and even propose our own ORASP Search Method. We further showcase how we can produce high-reward assembly plans for larger structures using our deep Reinforcement Learning (RL) method. We evaluate this method on large robotic assembly problems such as the assembly of the Hubble Space Telescope, the International Space Station, and the James Webb Space Telescope. We further discuss how our DP, GEAP, and RL methods are capable of finding optimal solutions under a variety of different objective functions and how we translate any form of precedence constraints to branch pruning and further improve performance. We have published our code at https://github.com/labicon/ORASP-Code.

This paper introduces a set of customizable and novel cost functions that enable the user to easily specify desirable robot formations, such as a "high-coverage" infrastructure-inspection formation, while maintaining high relative pose estimation accuracy. The overall cost function balances the need for the robots to be close together for good ranging-based relative localization accuracy and the need for the robots to achieve specific tasks, such as minimizing the time taken to inspect a given area. The formations found by minimizing the aggregated cost function are evaluated in a coverage path planning task in simulation and experiment, where the robots localize themselves and unknown landmarks using a simultaneous localization and mapping algorithm based on the extended Kalman filter. Compared to an optimal formation that maximizes ranging-based relative localization accuracy, these formations significantly reduce the time to cover a given area with minimal impact on relative pose estimation accuracy.

This paper introduces a novel hybrid torque allocation method for improving wearability and mobility in integrated functional electrical stimulation (FES) of the quadriceps muscles and powered exoskeleton systems. Our proposed approach leverages a hierarchical closed-loop controller for knee joint position tracking while addressing limitations of powered exoskeletons and FES systems by reducing power consumption and battery size and by mitigating FES-induced muscle fatigue, respectively. The core component is a model-free optimization algorithm that dynamically distributes torque between FES and the exoskeleton by considering tracking error, effort, and the prediction of muscle fatigue in the cost function, computing allocation gain in an online manner. The online optimization approach interactively changes the optimal allocation gain by taking into account the instantaneous value of error and effort and also penalizing FES-induced fatigue, a common challenge in long-duration experiments. The results demonstrate that this dynamic allocation significantly improves system wearability by reducing power consumption without increasing muscle fatigue during the extension phase of walking. This hybrid control approach contributes to improving exoskeleton wearability and rehabilitation outcomes for individuals with SCI and mobility impairments, enhancing assistive technology and quality of life.

Despite recent advances in semantic Simultaneous Localization and Mapping (SLAM) for terrestrial and aerial applications, underwater semantic SLAM remains an open and largely unaddressed research problem due to the unique sensing modalities and the object classes found underwater. This paper presents an object-based semantic SLAM method for underwater environments that can identify, localize, classify, and map a wide variety of marine objects without a priori knowledge of the object classes present in the scene. The method performs unsupervised object segmentation and object-level feature aggregation, and then uses opti-acoustic sensor fusion for object localization. Probabilistic data association is used to determine observation to landmark correspondences. Given such correspondences, the method then jointly optimizes landmark and vehicle position estimates. Indoor and outdoor underwater datasets with a wide variety of objects and challenging acoustic and lighting conditions are collected for evaluation and made publicly available. Quantitative and qualitative results show the proposed method achieves reduced trajectory error compared to baseline methods, and is able to obtain comparable map accuracy to a baseline closed-set method that requires hand-labeled data of all objects in the scene.

Opinions of individuals in large groups evolve through interactions with neighbors and the environment, which can be modeled with opinion dynamics. In this paper, we propose a distributed opinion-based strategy for large-scale multi-robot task allocation utilizing the convergence behaviors of opinion dynamics. The strategy relies on the specialized opinion dynamics on the unit sphere for robot task selection. We investigate the convergence behaviors of opinion dynamics in the context of regions of attraction. Simulation results with a swarm of 200 homogeneous robots validate the effectiveness of our proposed strategy.

3D reconstruction has been widely used in autonomous navigation fields of mobile robotics. However, the former research can only provide the basic geometry structure without the capability of open-world scene understanding, limiting advanced tasks like human interaction and visual navigation. Moreover, traditional 3D scene understanding approaches rely on expensive labeled 3D datasets to train a model for a single task with supervision. Thus, geometric reconstruction with zero-shot scene understanding i.e. Open vocabulary 3D Understanding and Reconstruction, is crucial for the future development of mobile robots. In this paper, we propose OpenOcc, a novel framework unifying the 3D scene reconstruction and open vocabulary understanding with neural radiance fields. We model the geometric structure of the scene with occupancy representation and distill the pre-trained open vocabulary model into a 3D language field via volume rendering for zero-shot inference. Furthermore, a novel semantic-aware confidence propagation (SCP) method has been proposed to relieve the issue of language field representation degeneracy caused by inconsistent measurements in distilled features. Experimental results show that our approach achieves competitive performance in 3D scene understanding tasks, especially for small and long-tail objects.

The integration of large-scale Vision-Language Models (VLMs) with embodied AI can greatly enhance the generalizability and the capacity to follow open instructions for robots. However, existing studies on object manipulation are not up to full consideration of the 6-DoF requirements, let alone establishing a comprehensive benchmark. In this paper, we propel the pioneer construction of the benchmark and approach for Open-instruction 6-DoF Object Rearrangement (Open6DOR). Specifically, we collect a synthetic dataset of 200+ objects and carefully design 5400+ Open6DOR tasks. These tasks are divided into the Position-track, Rotation-track, and 6-DoF-track for evaluating different embodied agents in predicting the positions and rotations of target objects.Besides, we also propose a VLM-based approach for Open6DOR, named Open6DOR-GPT, which empowers GPT-4V with 3D-awareness and simulation-assistance while exploiting its strengths in generalizability and instruction-following. We compare the existing embodied agents with our Open6DOR-GPT on the proposed Open6DOR benchmark and find that Open6DOR-GPT achieves the state-of-the-art performance. We further show the impressive performance of Open6DOR-GPT in diverse real-world experiments.

The growing interest in human-robot collaboration (HRC), where humans and robots cooperate towards shared goals, has seen significant advancements over the past decade. While previous research has addressed various challenges, several key issues remain unresolved. Many domains within HRC involve activities that do not necessarily require human presence throughout the entire task. Existing literature typically models HRC as a closed system, where all agents are present for the entire duration of the task. In contrast, an open model offers flexibility by allowing an agent to enter and exit the collaboration as needed, enabling them to concurrently manage other tasks. In this paper, we introduce a novel multiagent framework called oDec-MDP, designed specifically to model open HRC scenarios where agents can join or leave tasks flexibly during execution. We generalize a recent multiagent inverse reinforcement learning method - Dec-AIRL to learn from open systems modeled using the oDec-MDP. Our method is validated through experiments conducted in both a simplified toy firefighting domain and a realistic dyadic human-robot collaborative assembly. Results show that our framework and learning method improves upon its closed system counterpart.

The rising demand for customized products necessitates the integration of multiple robotic systems, underscoring the need for advanced production planning and scheduling. This paper introduces an ontology-based, artificial intelligence-enhanced method for dynamic task planning and scheduling, aimed at improving the efficiency of production process, reducing machine downtime, and consequently increasing throughput in assembly operations. Designed to generate and execute feasible production plans dynamically, this method minimizes manual planning and scheduling efforts. We evaluate its effectiveness using two gear assembly use cases with various robot skills, highlighting its flexibility in planning and scheduling and its contributions to the evolution of smart manufacturing. The method’s adaptability suggests its applicability across diverse smart factory environments.

Terrestrial laser scanning (TLS) is the standard technique used to create accurate point clouds for digital forest inventories. However, the measurement process is demanding, requiring up to two days per hectare for data collection, significant data storage, as well as resource-heavy post-processing of 3D data. In this work, we present a real-time mapping and analysis system that enables online generation of forest inventories using mobile laser scanners that can be mounted e.g. on mobile robots. Given incrementally created and locally accurate submaps—data payloads—our approach extracts tree candidates using a custom, Voronoi-inspired clustering algorithm. Tree candidates are reconstructed using an algorithm based on the Hough transform, which enables robust modeling of the tree stem. Further, we explicitly incorporate the incremental nature of the data collection by consistently updating the database using a pose graph LiDAR SLAM system. This enables us to refine our estimates of the tree traits if an area is revisited later during a mission. We demonstrate competitive accuracy to TLS or manual measurements using laser scanners that we mounted on backpacks or mobile robots operating in conifer, broad-leaf and mixed forests. Our results achieve RMSE of 1.93 cm, a bias of 0.65 cm and a standard deviation of 1.81 cm (averaged across these sequences)—with no post-processing required after the mission is complete.

Vertical take-off and landing (VTOL) vehicles are becoming increasingly popular for real-world transport; but, as with any vehicle, guaranteeing safety is both extremely critical and highly challenging due to issues like rotor faults. Existing fault detection and isolation (FDI) techniques usually focus on multirotor systems or fixed wing systems, rather than the hybrid VTOLs. Since VTOLs have both rotors and ailerons, a fault in a rotor may be masked by the (correctly working) ailerons, making it much more difficult to detect faults. However, this masking only works when ailersons are used (e.g., during cruising), leaving the takeoff and landing vulnerable to crashes.This paper presents an online rotor fault detection and isolation (FDI) method for VTOLs. The approach uses pose analysis and aileron command data to quickly and accurately identify the faulty rotor and to compute the severity of the fault. Our method works for hard-to-detect fault scenarios, such as small-severity faults that are masked during cruise flight but not during vertical motion. We evaluated our technique in a SITL PX4 simulation of a modified Deltaquad QuadPlane. The results show that our FDI technique can quickly detect and isolate faults in real time (within 1s-2.5s) and achieve high isolation success rate (91.67%) across six rotors, and that it can estimate the severity of faults to within 2%. When applying a simple recovery process post-isolation, the system consistently achieved safe landing.

This paper presents a general refractive camera model and online co-estimation of odometry and the refractive index of an unknown media. This enables operation in diverse and varying refractive fluids, given only the camera calibration in air. The refractive index is estimated online as a state variable of a monocular visual-inertial odometry framework in an iterative formulation using the proposed camera model. The method was verified on data collected using an underwater robot traversing inside a pool. The evaluations demonstrate convergence to the ideal refractive index for water despite significant perturbations in the initialization. Simultaneously, the approach enables on-par visual-inertial odometry performance in refractive media without prior knowledge of the refractive index or requirement of medium-specific camera calibration.

In multi-agent path finding (MAPF), agents navigate to their target positions without conflict within an environment, typically represented as a graph. Traditionally, the input graph is assumed to be accurate. We investigate MAPF scenarios where the input graph may be inaccurate, containing non-existent edges or missing edges present in the environment. Agents can verify the existence or non-existence of an edge only by moving close to it. To navigate such maps, we propose an online approach where planning and execution are interleaved. As agents gather new information about the environment over time, they replan accordingly. To minimize replanning efforts, we developed methods to identify and replan only for agents affected by observed changes. To scale to larger problems, we defer conflicts resolution expected only in the distant future and adapt single-agent path-finding algorithms to account for map inaccuracies. Experimental results show impressive scalability, solving problems involving over 1000 agents in under 3 minutes.

Managing delivery deadlines in automated warehouses and factories is crucial for maintaining customer satisfaction and ensuring seamless production. This study introduces the problem of online multi-agent pickup and delivery with task deadlines (MAPD-D), an advanced variant of the online MAPD problem incorporating delivery deadlines. In the MAPD problem, agents must manage a continuous stream of delivery tasks online. Tasks are added at any time. Agents must complete their tasks while avoiding collisions with each other. MAPD-D introduces a dynamic, deadline-driven approach that incorporates task deadlines, challenging the conventional MAPD frameworks. To tackle MAPD-D, we propose a novel algorithm named deadline-aware token passing (D-TP). The D-TP algorithm calculates pickup deadlines and assigns tasks while balancing execution cost and deadline proximity. Additionally, we introduce the D-TP with task swaps (D-TPTS) method to further reduce task tardiness, enhancing flexibility and efficiency through task-swapping strategies. Numerical experiments were conducted in simulated warehouse environments to showcase the effectiveness of the proposed methods. Both D-TP and D-TPTS demonstrated significant reductions in task tardiness compared to existing methods. Our methods contribute to efficient operations in automated warehouses and factories with delivery deadlines.

Upper limb amputees face significant challenges in their daily lives due to the loss of hand or arm functionality. Researchers have developed upper limb prostheses to restore normal hand movements for them. Most hand movement recognition systems of prostheses use electromyography (EMG) as the input signal source, but ignore the interrelationship with electroencephalography (EEG), which may contain valuable movement-related information as well. In order to enhance the accuracy of hand movement classification, we proposed a hand movement recognition system based on a one-dimensional convolutional neural network (1D-CNN) that combines EEG and EMG as the input signal sources to increase the quantity of accessible information. In this work, we collected the EEG and EMG of five subjects during the hand movements and used a 1D-CNN based model to classify the preprocessed signals. The average accuracy of using EEG-EMG fusion is 96.59±2.63%, significantly higher than 74.99±8.24% of using single EEG and 90.31±7.16% of using single EMG. Then, we applied the model trained by offline experiment for online recognition, and controlled the Pepper robot to complete the corresponding hand movements. The average accuracy of online recognition can reach 93.00±4.85% by using majority voting method. The results indicate that the method of EEG-EMG fusion can effectively enhance the performance of hand movement recognition system, which promote the development of upper limb prostheses and contribute to the rehabilitation of upper limb amputees.