ICRA 2024 论文列表

The utilization of shared control in the realm of surgical robotics augments precision and safety by amalgamating human expertise with autonomous assistance. This paper proposes a user-centered shared control framework enabling a robot to learn from expert demonstration, predict operators’ intent and modulate control authority to provide natural assistance when needed. We employ deep inverse reinforcement learning (IRL) to enable the robot to learn path planning from expert demonstrations with fast convergence, subsequently enhancing the policy with a potential field method. The control authority is allocated seamlessly between the human operator and the autonomous agent based on the prediction of operators’ movement from an adaptive filter and fuzzy logic inference. The proposed method is executed using the da Vinci Research Kit (dVRK) robot in a simulation environment, and its effectiveness is assessed through user performance evaluation in a trajectory tracking task. Compared to direct control and simple shared control, the proposed shared control scheme exhibits superior tracking accuracy and trajectory smoothness under external disturbances. Subjective responses underscore users’ perception of the method’s efficacy in enhancing their performance.

This paper proposes a new nonlinear factor sparsification paradigm for general feature-based long-term SLAM backend. Given a pose sparsification policy, we aim to scale the SLAM problem with space explored instead of time in a principled way so that the number of time-indexed poses can be limited. At the same time, their influence and the long-lived landmarks are appropriately maintained. To do this, we propose a new two-step sparsification pipeline. Given a pose node to remove, the first step is performed in the Markov blankets of affected landmarks. It transforms pose-landmark constraints into pose-pose constraints while preserving observability and minimizing information loss in the blanket. Moreover, since landmarks are conditionally independent, we can do this in parallel, disconnecting a pose from all the landmarks. The second step marginalizes the pose of interest with pure pose-wise constraints without affecting landmarks. Our method decouples the management of landmarks from pose-only measurements, making it general for any feature-based SLAM. We also give a practical example of how our backend works by concatenating it to a monocular VIO frontend. In simulation and realworld dataset, our sparsified backend is accurate and efficient. We open-source our backend, along with the VIO+Backend example, to contribute to the community’s betterment.

This paper presents a strategy for predicting the turning radius of a sailing robot with consideration of aerodynamic and hydrodynamic interferences from the marine environment. The turning radius is initially obtained based on three consecutive designated points during the turning process, which is regarded as the baseline method. Subsequently, on the basis of our constructed turning datasets, a model is trained using Gaussian process regression (GPR) to achieve radius prediction. The feasibility and effectiveness of the proposed scheme have been validated in both simulation and experiments (conducted with OceanVoy as shown in Fig. 1). Under experimental circumstances, the Mean Absolute Error (MAE) of the turning radius produced by the trained prediction model is 0.58m. Furthermore, it has been observed that during longterm sailing covering a distance of 1200km, apart from wind speed and robot velocity, the tidal range also has a significant impact on the navigation of sailing robots.

In this work, a tube-based nearly optimal solution to motion planning in unknown workspaces is presented. The advantages of reactive motion planning are combined with a Policy Iteration Reinforcement Learning scheme to yield a novel solution for unknown workspaces that inherits provable safety, convergence and optimality. Moreover, in simply-connected workspaces, our method is proven to asymptotically provide the globally optimal path. Our method is compared against a provably asymptotically optimal RRT⋆ method, as well as a relevant reactive method and provides satisfactory performance, closely matching or outperforming the former.

Drone racing has become a popular international competition and has attained wide attention in recent years. However, the requirements of high-level operation keep the novice pilots away from participating in it. This paper presents a trajectory-based flight assistive system that enables various operators to fly the drone in a racing scene at a high speed. The whole system is structured hierarchically, consisting of both offline and online components. In the offline part, a global time-optimal trajectory is generated as the expert reference, and a dense flight corridor is constructed to provide sufficiently large safe region. In the online part, a remote control-mapped primitive is designed to fast encapsulate pilots’ inputs, and the time mapping based trajectory progress is customized to further capture intention. Then, a trajectory planner is proposed to generate intention-aligned, smooth, feasible, and safe trajectories periodically. Additionally, a yaw planning that provides the pilot with the best suitable view angle is employed to further alleviate the operation difficulty. Simulations and real world experiments are implemented to verify the performance of our system. The maximum flight speed can reach 6.0 m/s for a novice drone pilot in a real racing scene. Our code is released as an open-source package 1.

Colon endoscopic robots represent a promising screening modality for the visualization of colon cancers with high sensitivity. However, current colonoscopy robots are often characterized by intricate and bulky mechanical structures, which pose practical challenges when moving through the complex and narrow environment of the colon. Moreover, these robots are typically equipped with a single camera, limiting their ability to accurately estimate the depth of haustral folds in the colon, which is of great importance for the active colonic navigation of the robots. To address these challenges, we develop a track-based stereoscopic endoscopic robot (TSER) which is equipped with four tracks positioned at the corners of its body. This innovative design maximizes the contact between the tracks and the colon wall, enhancing maneuverability. The tracks are constructed from de-molded polydimethylsiloxane (PDMS) and incorporate micro-patterns on their outer surfaces. We have proposed a straightforward strategy for detecting haustral folds using TSER’s stereo camera, which allows for precise identification of their position and depth. The TSER achieves an average motion speed of 9.8 mm/s in a bellows tube that contains silicone oil and a speed of 5.2 mm/s in an exvivo porcine intestinal segment. Impressively, the TSER boasts an 88.11% accuracy rate in haustral fold depth estimation, surpassing the performance of existing geometric shape fitting methods. These results demonstrate that the TSER holds great potential for effective and efficient movement and inspection within the colon, offering a promising solution for improved colon cancer screening.

The human-robot collaboration scenarios are characterized by the presence of human operators and robots that work in close contact with each other. As a consequence, the safety regulations have been updated in order to provide guidelines on how to asses safety in these new scenarios. In particular, Power and Force Limiting (PFL) collaborative mode describes how the energy should be regulated during the collaboration. Based on these guidelines, we propose a new optimal trajectory planner which, by exploiting the variability of the robot’s inertia as a function of its configuration, is able to return trajectories that can be travelled at greater speed and in less time, while guaranteeing the safety limits according to the standard. The proposed planner was validated first in simulation, comparing completion times with other state-of-the-art planning algorithms, and then experimentally, demonstrating the performance of the planned trajectories during physical interaction with the environment. Both validations confirm the effectiveness of the proposed planner, which returns shorter completion times while ensuring safe interaction.

The problem of grasping objects using a multi-finger hand has received significant attention in recent years. However, it remains challenging to handle a large number of unfamiliar objects in real and cluttered environments. In this work, we propose a representation that can be effectively mapped to the multi-finger grasp space. Based on this representation, we develop a simple decision model that generates accurate grasp quality scores for different multi-finger grasp poses using only hundreds to thousands of training samples. We demonstrate that our representation performs well on a real robot and achieves a success rate of 78.64% after training with only 500 real-world grasp attempts and 87% with 4500 grasp attempts. Additionally, we achieve a success rate of 84.51% in a dynamic human-robot handover scenario using a multi-finger hand.

Force-free control (FFC) allows for flexible manipulator motion in response to external forces, making it a vital component of human-robot interaction (HRI). Manual intervention may cause uneven forces on the manipulator or frequencies close to the natural frequency, and mechanical resonance can occur due to the inertia of the manipulator and adjustable equivalent stiffness of the controller. This paper proposes an FFC approach for industrial manipulators using a six-axis force/torque sensor (F/T sensor), implemented through a three-layer control architecture, consisting of motion control layer, Admittance Control layer and force decoupling layer. To mitigate the effects of mechanical resonance, a high-pass filter (HPF) is integrated with the F/T sensor and its impact is investigated. Experimental validation is conducted using both a simulation model and an industrial manipulator. Test results indicate that the proposed FFC architecture enables the manipulator not only to interact smoothly with external forces, but also to distinguish load forces at different frequencies and potentially address the issue of mechanical resonance between the manipulator and the applied load forces.

We show how a variety of techniques from Computer Graphics can be leveraged to intuitively control the shape (configuration) of arbitrary 3D Soft Robots in VR. Our pipeline, Virtual Reality Soft Robot Inverse Kinematics (VR-Soft IK), overcomes fundamental limitations of general-purpose drag-and-drop soft robot control interfaces by leaving the 2D computer screen for 3D Virtual Reality (VR). VR-Soft IK uses a simulation based on the Finite Element Method (FEM) and a control method based on sensitivity analysis. Additionally, we show that our general control pipeline can be fused with techniques from 3D character animation to skin our simulation with a high-resolution surface mesh, pointing a way toward Mixed Reality Soft Robots. This full Skinned VR-Soft IK pipeline uses skeletal animation and GPU picking. We demonstrate the utility of our pipeline by doing real-time, open-loop control of the real-world 3D soft robotic arm Helix.

Aneurysms, balloon-like bulges in blood vessels, present a significant health risk due to their potential to rupture, leading to life-threatening internal bleeding. Current treatments often involve delivering embolic materials or metal coils to fill these bulges, occluding them from the pressure of blood flow. However, clinical micro-catheters that deploy embolic materials used today face limitations, primarily their rigidity and the lack of active control over the bending tip of the catheter. This paper introduces a new soft micro-robotics catheter, with diameter of only 0.8 mm, equipped with a hollow channel. With this new design, the new device can induce bending motions at its tip for active steerability to reach desired aneurysm targets and then perform the delivery of embolic materials and tools. To enhance the control and precise navigation during procedures, a robust mathematical model and image processing techniques are also introduced and validated. Experiments are also performed to characterise and validate the model’s accuracy and the steerability and navigation capabilities of the new micro-catheter.

In this paper, a numerically efficient flexible control scheme for the absolute accuracy of industrial robots is presented and experimentally validated. A model-based controller that leverages all typically available parameters is combined with an online path iterative learning controller (ILC). The ILC law is employed to compensate for the unknown residual error dynamics caused by elastic and transmission effects. The proposed approach combines several benefits, including the possibility of a continuous execution of trials, a straightforward generalization of the learned data to different execution speeds, and learning from partial trials. The experimental validations on a 6-axis industrial robot with a laser tracker absolute measurement system show a 95% improvement in absolute accuracy after two trials. When the laser tracker is removed, the learned feedforward controller can sustain the accuracy achieved even without trial-by-trial learning.

In this work, we propose a semi-autonomous scheme to synergistically share the complicated task of manipulation and cutting of an unknown deformable tissue (U-DT) between a remote surgeon and a surgical robot. Particularly, utilizing the da Vinci Research Kit (dVRK) platform, we have designed and successfully demonstrated a fully functional shared control scheme for an autonomous tensioning and tele-cutting of a U-DT. We have shown the system’s ability to cooperate with a remote surgeon by leveraging an online data-driven learning and adaptive control method coupled with a reduced-order trajectory planning module that depends on just two parameters. By performing 25 experiments on custom-designed silicon phantoms and defining a set of success/failure metrics, we have put forward findings that establish a causal relationship between these two important parameters and the success or failure of the performed experiments.

In safety-critical domains like autonomous driving (AD), errors by the object detector may endanger pedestrians and other vulnerable road users (VRU). As raw evaluation metrics are not an adequate safety indicator, recent works leverage domain knowledge to identify safety-relevant VRU, and to back-annotate the criticality of the interaction to the object detector. However, those approaches do not consider the safety factor in the deep neural network (DNN) training process. Thus, state-of-the-art DNN penalize all misdetections equally irrespective of their importance for the safe driving task. Hence, to mitigate the occurrence of safety-critical failure cases like false negatives, a safety-aware training strategy is needed to enhance the detection performance for critical pedestrians. In this paper, we propose a novel, safety-adapted loss variation that leverages the estimated per-pedestrian criticality during training. Therefore, we exploit the reachable set-based time-to-collision (TTCRSB) metric from the motion domain along with distance information to account for the worst-case threat. Our evaluation results using RetinaNet and FCOS on the nuScenes dataset demonstrate that training the models with our safety-adapted loss function mitigates the misdetection of safety-critical pedestrians with robust performance for the general case, i.e., safety-irrelevant pedestrians.

Tactile servoing is an effective approach to enabling robots to safely interact with unknown environments. One of the core problems in tactile servoing is to robustly converge the contact features to the desired ones via a dedicated controller. This paper proposes a Data-Driven Model Predictive Controller (DDMPC) to compute the motion command given the previous interaction experience and feature deviations in tactile space. Compared with the manually designed PID-based controller, the proposed controller depends on the sound control theory and its convergence is guaranteed from a computational perspective. It is applied to the balancing control of a rolling bottle on a robotic forearm covered by a custom tactile sensor array. The real experiment demonstrates the superior robustness of the proposed approach and shows its great potential for other tactile servoing scenarios with measurement noise, which is inevitable for current tactile sensors.

Images captured by robotics under low-light conditions are often plagued by several challenges, including diminished contrast, increased noise, loss of fine details, and unnatural color reproduction. These factors can significantly hinder the performance of computer vision tasks such as object detection and image segmentation. As a result, improving the quality of low-light images is of paramount importance for practical applications in the computer vision domain. To effectively address these challenges, we present a novel low-light image enhancement model, termed Spatial Consistency Retinex Network (SCRNet), which leverages the Retinex-based structure and is guided by the principle of spatial consistency. Specifically, our proposed model incorporates three levels of consistency: channel level, semantic level, and texture level, inspired by the principle of spatial consistency. These levels of consistency enable our model to adaptively enhance image features, ensuring more accurate and visually pleasing results. Extensive experimental evaluations on various low-light image datasets demonstrate that our proposed SCRNet outshines existing state-of-the-art methods, highlighting the potential of SCRNet as an effective solution for enhancing low-light images.

In the context of interaction with unmodelled systems, it becomes imperative for a robot controller to possess the capability to dynamically adjust its actions in real-time, enhancing its resilience in the face of fluctuating environmental conditions. This adaptation process must be performed in a stability-preserving fashion, and resourcefully exploit the knowledge acquired during the interaction process. In this article, we propose a novel control strategy, based on the synergistic usage of state-of-the-art passivity-based control and Deep Reinforcement Learning (DRL). The concept of energy tank is used to provide stability guarantees for the interaction controller with uncertain environments, while an online learning policy allows to properly estimate the requirements of the task and adapt the controller accordingly, thus simultaneously achieving stability and performance. The proposed architecture is successfully validated through simulations and experiments with a collaborative manipulator in a surface polishing task.

Skid-Steer Wheeled Mobile Robots (SSWMRs) are increasingly being used for off-road autonomy applications. When turning at high speeds, these robots tend to undergo significant skidding and slipping. In this work, using Gaussian Process Regression (GPR) and Sigma-Point Transforms, we estimate the non-linear effects of tire-terrain interaction on robot velocities in a probabilistic fashion. Using the mean estimates from GPR, we propose a data-driven dynamic motion model that is more accurate at predicting future robot poses than conventional kinematic motion models. By efficiently solving a convex optimization problem based on the history of past robot motion, the GPR augmented motion model generalizes to previously unseen terrain conditions. The output distribution from the proposed motion model can be used for local motion planning approaches, such as stochastic model predictive control, leveraging model uncertainty to make safe decisions. We validate our work on a benchmark real-world multi-terrain SSWMR dataset. Our results show that the model generalizes to three different terrains while significantly reducing errors in linear and angular motion predictions. As shown in the attached video, we perform a separate set of experiments on a physical robot to demonstrate the robustness of the proposed algorithm.

Inspection of powerlines is a hard problem that requires humans to operate in remote locations and dangerous conditions. This paper proposes a quadcopter unmanned aerial vehicle (UAV) equipped with rolling-capable perching mechanisms and a depth-vision system for the purpose of autonomous power line inspection. The perching mechanism grips onto the power line, allowing the UAV to withstand external forces such as wind disturbances. Once engaged and applying the desired gripping force, the perching mechanism requires no power through the use of a ratcheting serial elastic transmission, allowing the UAV to perch indefinitely. The depth-vision system automates the perching and unperching procedures by estimating the position and pose of the UAV relative to the powerline. These measurements are sent to a local position controller that guides the UAV to and from the power line. Once perched, rollers in the fingers of the perching mechanism drive the UAV along the powerline, providing a close-up platform for inspection equipment. The proposed system was tested in an outdoor testing environment and shown to autonomously perch and unperch from a steel cable. The grippers force application was analysed and the UAVs powerless robust perch is demonstrated by total disconnect of power while perched. These results suggest that such a system could be a valuable tool for the upkeep of electricity networks.