IROS 2024 论文列表

Tactile sensing is pivotal for enabling effective human-robot interaction, especially in unstructured environments. This work introduces an innovative bioinspired soft robotic finger endowed with shape-adaptive and multi-modal tactile perception capabilities, drawing inspiration from diverse biological tactile sensing modalities. Through an advanced Fin Ray structure, the soft finger features tactile whiskers on its fingertips, facilitating perception of obstacle orientation, fingertip pressure, surface roughness, and grasping ball size. Leveraging distributed optical fiber sensing technology, we develop a sophisticated multi-point, multi-modal tactile perception neural network tailored for the soft finger. Meticulous integration via advanced 3D printing and silicone coating techniques seamlessly embeds optical fiber sensors within the soft robotic finger, creating an intelligent perception-capable bioinspired mechanical system. Experimental validation confirms the soft robotic finger’s sensitive and precise force perception and curvature recognition abilities, achieving accuracies of up to 100%. In summary, our bioinspired robotic finger holds significant promise for applications in intelligent sensing, non-destructive grasping, and fruit classification within unstructured environments, thus advancing the field of robotics and human-robot interaction.

We consider a service robot that offers chocolate treats to people passing in its proximity: it has the capability of predicting in advance a person’s intention to interact, and to actuate an "offering" gesture, subtly extending the tray of chocolates towards a given target. We run the system for more than 5 hours across 3 days and two different crowded public locations; the system implements three possible behaviors that are randomly toggled every few minutes: passive (e.g. never performing the offering gesture); or active, triggered by either a naive distance-based rule, or a smart approach that relies on various behavioral cues of the user. We collect a real-world dataset that includes information on 1777 users with several spontaneous human-robot interactions and study the influence of robot actions on people’s behavior. Our comprehensive analysis suggests that users are more prone to engage with the robot when it proactively starts the interaction. We release the dataset and provide insights to make our work reproducible for the community. Also, we report qualitative observations collected during the acquisition campaign and identify future challenges and research directions in the domain of social human-robot interaction.

As the safe and comfortable human-robot interaction draw more attention, variable stiffness joints are adopted in the rehabilitation robots for the compliant experiences. For patients with joint injury, resistance training has demonstrated its efficacy in concurrently enhancing muscle strength in the rehabilitation process. Here, we introduce a compact passive joint with variable stiffness and damping generation capability for robot-assisted resistance training. The abilities of serial variable stiffness and passive angle tracking were demonstrated by a prototype. The variable stiffness mechanism was mainly formed by 3D-printing, which only weighs 0.1kg with a wide range of stiffness (0.004-11.176Nm/rad). Moreover, the torque-angular displacement characteristics show excellent linearity at high joint stiffness. Finally, the feasibility of being integrated into the wearable rehabilitation robot has been evaluated.

The intersection of robotics and artificial intelligence led to a profound paradigm shift in Robot Learning. Robots have the capacity to replicate human actions and also dynamically adapt, innovate, and excel across a spectrum of tasks. However, the heterogeneity in the deployment of robot platforms and software frameworks poses considerable challenges in terms of systematic testing and comparative analyses. Additionally, the data scarcity of especially force controlled robot manipulation is still restraining the development of advanced foundation models. A reference platform with default software stack can help to increase comparability, reducing development time and collect a large amount of tactile robot manipulation data. To address on this problem, we developed a Parallel and Distributed Robot AI (PD.RAI) framework, comprising a scalable ensemble of Robot Learning Units (RLUs), a global database, and the Robot Cluster Intelligence (RoCI). Each RLU is endowed with robot arms, cameras, and local computational units to autonomously engage in planning, control, and local machine learning of tactile manipulation skills. The RoCI system oversees the learning process and schedules the RLUs tasks. To show the functionality of the system, two black-box optimization algorithms are compared within the robot skill learning domain. An experiment with 24 different optimization tasks is conducted in parallel. The algorithms are incorporated into the same existing default modules acting as a reference environment. This allows for a realistic comparison without sacrificing diversity of possible configurations and testing environments.

Many robot tasks impose constraints on the workspace. For example, a robot may need to move a container without spilling its contents or open a door following the doorknob’s arc. Such constraints may induce narrow volumes in the configuration space, traditionally a challenge for sampling-based methods, and further cause infeasibility. We extend sample-driven connectivity learning (SDCL), a robust approach for planning with narrow passages, to develop a sampling ensemble for workspace constraints. In particular, the ensemble combines SDCL, projection via dual quaternion optimization, and random sampling. These complementary sampling approaches support efficient and robust planning under workspace constraints. Further, this framework offers the ability to determine infeasibility under workspace constraints, which is unaddressed by previous constrained planning methods.

In unstructured environments with complex obstacles and obscure road boundaries, the local planner faces more severe challenges in terms of safety and real-time performance. In order to fulfill these emerging requirements, we propose a novel Timed-Elastic-Band approach for unstructured environments, abbreviated as TEB-U. This approach incorporates a free space extraction optimization module for 2D occupancy grid maps, which efficiently transforms irregular free space boundaries into polygons and restrains robots within the boundaries. Moreover, a dynamic global point adjustment module is designed to adaptively correct the trajectory points obtained from the global planner, thereby enabling robots to travel along the centerline of free space and providing a better initial trajectory for subsequent modules. To reduce the computational cost, we replace the obstacle constraint of TEB with the boundary constraint in hyper-graph optimization. We evaluate our planner in three distinct scenarios, and the results show that TEB-U improves the average success rate by 21% and reduces the planning time by 23% compared to TEB in unstructured road, which demonstrates its safety and efficiency.

For humans, dense bin packing heavily relies on force perception. However, current robotic packing studies only focus on the visual input or adopt auxiliary push-to-place actions to eliminate gaps, suffering from high time expenditure and poor robustness. To address such limitations, we first introduce a novel external force estimation method based on the generalized momentum observer, which can avoid the influence of joint acceleration noises and achieve real-time high-precision monitoring. Second, to obtain compliant interaction and fine robustness, an adaptive variable impedance policy is developed to track dynamic motion and desired force, and compensate for uncertainties. Meanwhile, we perform dissipativity analysis and a virtual energy supply function is augmented to the system for optimization, providing a solid foundation for stability. Third, we propose an efficient packing methodology with three sub- tasks by considering the distinct interaction and constraint states in different areas. Our packing strategies eliminate the need for subsequent auxiliary actions and are proven to enhance efficiency. We perform quantitative evaluations to verify our external force estimation method, conduct comparison studies with current packing methods, and investigate the contribution of our dissipativity-based adaptive controller. The superior results not only prove the robustness and efficiency of our pipeline, but also pave the way for practical applications of packing.

This paper presents a robust visual SLAM system designed for small-scale quadruped robots (ViQu-SLAM) for accurate localization, especially to mitigate the issue of erroneous data association caused by moving objects in dynamic environments. The proposed approach leverages a selfadaptive framework that integrates semantic segmentation with alterations in the spatial location of categorized map points. Besides, combination of leg odometry derived from forward kinematics with IMU provides scale information for positional transformations between keyframes, thus optimizing the overall localization accuracy of quadruped robots. At last, we performed evaluation across various stages and the results demonstrate competitive performance, with 53.16% reduction in average absolute trajectory error compared to that of ORB-SLAM3 in dynamic benchmark datasets. As a result, ViQu-SLAM, including visual and IMU-fused leg odometry, exhibits promising results on a small quadruped robot, reducing positioning errors in dynamic scenes by an average of 29.36% compared to existing state-of-the-art methods.

The goal of this paper is to strike a feasible tracking paradigm that can make 3D human trackers applicable on robot platforms and enable more high-level tasks. Till now, two fundamental problems haven’t been adequately addressed. One is the computational cost lightweight enough for robotic deployment, and the other is the easily-influenced accuracy varied greatly in complex real environments. In this paper, a robotic-centric tracking paradigm called MATNet is proposed that directly matches the LiDAR point clouds and RGB videos through end-to-end learning. To improve the low accuracy of human tracking against disturbance, a coarse-to-fine Transformer along with target-ware augmentation is proposed by fusing RGB videos and point clouds through a pyramid encoding and decoding strategy. To better meet the real-time requirement of actual robot deployment, we introduce the parameter-efficient adaptation tuning that greatly shortens the model’s training time. Furthermore, we also propose a five-step Anti-shake Refinement strategy and have added human prior values to overcome the strong shaking on the robot plat-form. Extensive experiments confirm that MATNet significantly outperforms the previous state-of-the-art on both open-source datasets and large-scale robotic datasets.

Colonoscopy demands multi-finger coordinated motion to achieve safe navigation. As a result, training for colonoscopists is challenging and skill assessment currently relies on subjective scoring by expert proctors. There is a need to provide tools for skill assessment and aid with training new interventionalists. This paper presents a new concept of an in-hand robotic mediation device that can be used for both skill assessment and training. The robotic device can be used to infer the kinematic motion as well as the power input of a user - both of which are proposed to be used for skill assessment and subtask skill classification. Preliminary results collected expert and novice users performing colonoscopy navigation are used to demonstrate this device as a skill assessment tool. A machine-learning model (classification and regression trees) is used for subtask classification of skill and evaluating the most important classification features. A user study demonstrates the effectiveness of this in hand haptic training and assessment tool. We believe that, in the future, this device will enable accelerated skill assessment and training and possible semi-automation of difficult maneuvers.

In order to make robots more useful in a variety of environments, they need to be highly portable so that they can be transported to wherever they are needed, and highly storable so that they can be stored when not in use. We propose "on-site robotics", which uses parts procured at the location where the robot will be active, and propose a new solution to the problem of portability and storability. In this paper, as a proof of concept for on-site robotics, we describe a method for estimating the kinematic model of a robot by using inertial measurement units (IMU) sensor module on rigid links, estimating the relative orientation between modules from angular velocity, and estimating the relative position from the measurement of centrifugal force.At the end of this paper, as an evaluation for this method, we present an experiment in which a robot made up of wooden sticks reaches a target position. In this experiment, even if the combination of the links is changed, the robot is able to reach the target position again immediately after estimation, showing that it can operate even after being reassembled. Our implementation is available on https://github.com/hiroya1224/urdf_estimation_with_imus.

Human Pose Estimation (HPE) is increasingly utilized across various sectors, from healthcare to Industry 5.0. To address the inherent inaccuracies in CNN-based HPE systems, filtering models are commonly employed to refine and improve inference results. However, state-of-the-art filtering models often require substantial computational resources, limiting their applicability in resource-constrained environments. To overcome this limitation, we propose a real-time filtering approach based on denoising diffusion models (DM) specifically optimized for edge devices. Through a micro-benchmarking process, we analyze the DM adaptability to different types and levels of noise and determine the optimal setup for specific application scenarios. We present a real-time filter that takes advantage of the DM setup with two configurations to address different application scenarios. Using a widespread edge device, we evaluate the model’s effectiveness in handling both synthetic and real noise generated by state-of-the-art HPE systems. The results demonstrate a significant improvement in real-time filtering performance with minimal computational overhead. The code is available on github.com/PARCO-LAB/LUT-DM-filters.

Robots performing tasks in challenging environments must be supported by control or planning algorithms that exploit sensor feedback to effectively plan the robot’s actions. In this paper, we propose a reactive control law that simultaneously utilizes proximity and tactile feedback to perform a pick-and-place task in an unknown and cluttered environment. Specifically, the presented solution leverages proximity sensing obtained from distributed Time of Flight (ToF) sensors to avoid collision when this does not interfere with the pick-and-place task. Safety is guaranteed by a higher-priority task using tactile feedback that reduces contact forces when a collision occurs. Additionally, we compare the effectiveness of this control scheme with a collision detection and reaction scheme based solely on tactile sensing. Our results demonstrate that the proposed approach reduces the collisions with the environment and the task execution time of the pick-and-place operation.

Fast and robust initialization is essential for highly accurate monocular visual-inertial odometer (VIO), but at present majority of initialization methods rely only on point features, unstable in low texture and blurring situations. Therefore, we propose a novel point-line features fusion method for monocular visual-inertial initialization, as line features are more stable and provide richer geometric information than point features: 1) a closed-form line features initialization method is presented, and combined with point features to obtain a more integrated and robust linear system; 2) a monocular depth network is adopted to provide learned affine-invariant depth map, requiring only one prior depth map for the first frame, which can improve performance under low-parallax scenarios; 3) we can easily use RANSAC to reject outliers in solving linear system based on our formulation. Moreover, line feature re-projection residual is added to visual-inertial bundle adjustment (VI-BA) to obtain more accurate initial parameters. The proposed method is more accurate and robust than state-of-the-art methods due to the line features, especially under extreme low-parallax scenarios, and extensive experiments on popular datasets have confirmed, 0.5s initialization window on EuRoC MAV, 0.3s initialization window on TUM-VI, while the standard method normally waits for a window of 2s.

Industrial robots and collaborative robots are widely employed in industry and are progressively being utilized to assist individuals in their daily routines. To improve their absolute accuracy, self-calibration methods using portable local measurement devices are cost-effective solutions. However, compared with the conventional external calibration methods, self-calibration methods employing two configurations as a calibration sample introduce more non-kinematic errors to the robot. Therefore, noise reduction is significantly necessary in self-calibration. A novel Piecewise-weighted Random Sample Consensus (RANSAC) method is proposed in this paper. Instead of choosing an optimal model with all inliers, the proposed method employs a general weight considering both the sample and hypothesis model qualities to generate a new model with Weighted Least Square (WLS) method. Besides, the proposed method turns the target of finding an uncontaminated set of inliers into the training of the proper weight coefficient for WLS, which not only improves the accuracy but also greatly enhances the speed. The self-calibration experiment on a 6 degree-of-freedom(DOF) robot CR10 shows that the accuracy of the proposed Piecewise-weighted RANSAC method makes a 27.7% accuracy improvement from that employing Least Square method, a 20.0% accuracy improvement from that employing standard RANSAC method, and a 5.5% accuracy improvement from that employing LO-RANSAC method. Besides, the proposed method is also over 10.9 times faster than the standard RANSAC method and 18.6 times faster than the LO-RANSAC method.

In the last decades, soft robotics has been growing rapidly as an emerging research topic, bringing new paradigms for robotic manipulation, locomotion, and human‒machine interactions. Pneumatic artificial muscle is a powerful, lightweight, rapid response with great design flexibility, making it promising for developing biological muscle-like robotic systems. The PPAM is made of a silicone tube body with double helix coil fiber reinforcement. The double helix coil fiber restricts the radial expansion of the cylinder tube to achieve extension in actuation, and monitors the muscle length change in real time by measuring its inductance. A finite element model was built to simulate the actuation characteristics of the PPAM. A theoretical formula was derived to analyze the inductive length sensing response of the double-helix coil on the PPAM. It is verified that the PPAM can sense its length change regardless of whether it is caused by active driving or external manipulation. Rigorous testing reveals that PPAM has an ultrahigh length sensing resolution of 5.9 μm in relaxed state, with a short response time of 50 ms. The self-length sensing of PPAM is hysteresis free, and highly repeatable, showing no degradation in 1000 operation cycles. In summary, the PPAM shows promising features for developing the next-generation perceptive and responsive soft robots, intelligent hybrid robots, or safer biomedical instruments.

This paper presents a novel robotic system designed for efficient minimally invasive eye surgery. The proposed prototype integrates a concentric tube mechanism(CTM) and a belt-driven remote center of motion(RCM) mechanism, aiming to maximize the internal reachable workspace while minimizing external robot movements. The integrated system provides several advantages, including preventing collisions between surgical tools and the lens, minimizing sclera stress, and having efficient robot motion inside and outside the eyeball. It provides sufficient link motions with roll and pitch angles of ±32° and ±85° respectively at the RCM point, allowing access to 89% of the retina. The experiment evaluates the system's performance, with the RCM point accuracy at 0.718mm, CTM position accuracy at 207 μm, and a repeatability error of 246 μm. To reduce hysteresis errors at the RCM point caused by the belt, a lever-based belt tensioner is used for initial calibration while an optical tracking system tracks each joint’s movement. Targeting experiments highlight that the wider workspace was achieved by the CTM+RCM system compared to the traditional RCM mechanism with a straight tool. The results showed the system's compactness, efficiency, and dexterity, confirming its feasibility and potential for the proposed eye surgery robot.

Mobile manipulators (MM) have proven valuable in assisting humans in industrial settings. However, their strict separation from humans in controlled environments limits their effectiveness. Efforts have been made to bridge this gap for physical human-robot interaction (pHRI), leading to the development of collaborative mobile manipulators (CMM). Nonetheless, unpredictable environments continue to present challenges. This paper introduces an innovative suspension design for mobile bases (MBs) to enhance the safety and autonomy of CMMs. We propose an electromechanical approach leveraging variable stiffness and combining passive springs with adaptive transmission mechanisms. Through simulation, physical prototype development, and experimental validation, we demonstrate the effectiveness of our approach in stabilizing the MB against external disturbances. Our findings provide valuable insights for the development of CMMs in dynamic environments.

In recent years, Deep Reinforcement Learning (DRL) has been a key approach to solving Unmanned Aerial Vehicle (UAV) swarm path planning problems. However, traditional DRL methods often face challenges in the initial learning stage and struggle to learn from variable step-size tasks. This paper introduces a novel training framework for large-scale UAV swarm variable step-size path planning: Rapidly-exploring Variable Step-size Deep Reinforcement Learning (RVSDRL). This framework involves common training on the ground local server and decentralized training on distributed UAVs. In the common training stage, we generate rapidly-exploring random graph samples to accelerate the common agent explore environment. In the decentralized training stages, we utilize the priority replay mechanism to improve efficiency. To enhance convergence stability, we restrict the returns of the equivalent paths and propose the Step-size Consistent Markov Decision Process (SCMDP) path planning model. Our method is compared with traditional methods, and the experiments demonstrate its superior performance in complex obstacle environments.