ICRA 2024 论文列表

In many robotic systems, the holding state consumes power, limits operating time, and increases operating costs. Electrostatic clutches have the potential to improve robotic performance by generating holding torques with low power consumption. A key limitation of electrostatic clutches has been their low specific shear stresses which restrict generated holding torque, limiting many applications. Here we show how combining the Johnsen-Rahbek (JR) effect with the exponential tension scaling capstan effect can produce clutches with the highest specific shear stress in the literature. Our system generated 31.3 N/cm2 sheer stress and a total holding torque of 7.1 N•m while consuming only 2.5 mW/cm2 at 500 V. We demonstrate a theoretical model of an electrostatic adhesive capstan clutch and demonstrate how large angle (θ > 2π) designs increase efficiency over planar or small angle (θ < π) clutch designs. We also report the first unfilled polymeric material, polybenzimidazole (PBI), to exhibit the JR-effect.

In this paper, we present an iterative method to quickly traverse multi-dimensional paths considering jerk constraints. As a first step, we analyze the traversal of each individual path dimension. We derive a range of feasible target accelerations for each intermediate waypoint of a one-dimensional path using a binary search algorithm. Computing a trajectory from waypoint to waypoint leads to the fastest progress on the path when selecting the highest feasible target acceleration. Similarly, it is possible to calculate a trajectory that leads to minimum progress along the path. This insight allows us to control the traversal of a one-dimensional path in such a way that a reference path length of a multi-dimensional path is approximately tracked over time. In order to improve the tracking accuracy, we propose an iterative scheme to adjust the temporal course of the selected reference path length. More precisely, the temporal region causing the largest position deviation is identified and updated at each iteration. In our evaluation, we thoroughly analyze the performance of our method using seven-dimensional reference paths with different path characteristics. We show that our method manages to quickly traverse the reference paths and compare the required traversing time and the resulting path accuracy with other state-of-the-art approaches.

Developing autonomous vehicles that can safely interact with pedestrians requires large amounts of pedestrian and vehicle data in order to learn accurate pedestrian-vehicle interaction models. However, gathering data that include crucial but rare scenarios - such as pedestrians jaywalking into heavy traffic - can be costly and unsafe to collect. We propose a virtual reality human-in-the-loop simulator, JaywalkerVR, to obtain vehicle-pedestrian interaction data to address these challenges. Our system enables efficient, affordable, and safe collection of long-tail pedestrian-vehicle interaction data. Using our proposed simulator, we create a high-quality dataset with vehicle-pedestrian interaction data from safety critical scenarios called CARLA-VR. The CARLA-VR dataset addresses the lack of long-tail data samples in commonly used real world autonomous driving datasets. We demonstrate that models trained with CARLA-VR improve displacement error and collision rate by 10.7% and 4.9%, respectively, and are more robust in rare vehicle-pedestrian scenarios.

We present Jade, a differentiable physics engine for articulated rigid bodies. Jade models contacts as the Linear Complementarity Problem (LCP). Compared to existing differentiable simulations, Jade offers features including intersection-free collision simulation and stable LCP solutions for multiple frictional contacts. We use continuous collision detection to detect the time of impact and adopt the backtracking strategy to prevent intersection between bodies with complex geometry shapes. We derive the gradient calculation to ensure the whole simulation process is differentiable under the backtracking mechanism. We modify the popular Dantzig’s algorithm to get valid solutions under multiple frictional contacts. We conduct extensive experiments to demonstrate the effectiveness of our differentiable physics simulation over a variety of contact-rich tasks. Supplemental materials and videos are available on our project webpage at https://sites.google.com/view/diffsim

In the context of rapid advancements in industrial automation, vision-based robotic grasping plays an increasingly crucial role. In order to enhance visual recognition accuracy, the utilization of large-scale datasets is imperative for training models to acquire implicit knowledge related to the handling of various objects. Creating datasets from scratch is a time and labor-intensive process. Moreover, existing datasets often contain errors due to automated annotations aimed at expediency, making the improvement of these datasets a substantial research challenge. Consequently, several issues have been identified in the annotation of grasp bounding boxes within the popular Jacquard Grasp Dataset [1]. We propose utilizing a Human-In-The-Loop(HIL) method to enhance dataset quality. This approach relies on backbone deep learning networks to predict object positions and orientations for robotic grasping. Predictions with Intersection over Union (IOU) values below 0.2 undergo an assessment by human operators. After their evaluation, the data is categorized into False Negatives(FN) and True Negatives(TN). FN are then subcategorized into either missing annotations or catastrophic labeling errors. Images lacking labels are augmented with valid grasp bounding box information, whereas images afflicted by catastrophic labeling errors are completely removed. The open-source tool Labelbee was employed for 53,026 iterations of HIL dataset enhancement, leading to the removal of 2,884 images and the incorporation of ground truth information for 30,292 images. The enhanced dataset, named the Jacquard V2 Grasping Dataset, served as the training data for a range of neural networks. We have empirically demonstrated that these dataset improvements significantly enhance the training and prediction performance of the same network, resulting in an increase of 7.1% across most popular detection architectures for ten iterations. This refined dataset will be accessible on One Drive and Baidu N... Show More

In this paper, we introduce JacobiGPU, a technique that uses a GPU to improve the efficiency of loop closure in visual-inertial SLAM systems, particularly when approximating Jacobians using the Finite Difference Method (FDM). Traditional FDM techniques often face computational overhead due to repeated perturbations in pose graphs. We address this overhead with a novel methodology, leveraging strategic graph partitioning and an optimized approach to Jacobian approximation. By integrating JacobiGPU into ORB-SLAM3’s g2o, we enhance the linearization process. Our evaluation, conducted on 12 sequences of varying lengths from the EuRoC and TUM-VI datasets, demonstrated a speedup of up to 4.23x in the linearization stage and an overall enhancement of up to 2.08x in the overall optimization process.

Event-based moving object detection is a challenging task, where static background and moving object are mixed together. Typically, existing methods mainly align the background events to the same spatial coordinate system via motion compensation to distinguish the moving object. However, they neglect the potential spatial tailing effect of moving object events caused by excessive motion, which may affect the structure integrity of the extracted moving object. We discover that the moving object has a complete columnar structure in the point cloud composed of motion-compensated events along the timestamp. Motivated by this, we propose a novel joint spatio-temporal reasoning method for event-based moving object detection. Specifically, we first compensate the motion of background events using inertial measurement unit. In spatial reasoning stage, we project the compensated events into the same image coordinate, discretize the timestamp of events to obtain a time image that can reflect the motion confidence, and further segment the moving object through adaptive threshold on the time image. In temporal reasoning stage, we construct the events into a point cloud along timestamp, and use RANSAC algorithm to extract the columnar shape in the cloud for peeling off the background. Finally, we fuse the results from the two reasoning stages to extract the final moving object region. This joint spatio-temporal reasoning framework can effectively detect the moving object from motion confidence and geometric structure. Moreover, we conduct extensive experiments on various datasets to verify that the proposed method can improve the moving object detection accuracy by 13%.

This paper reports on a new real-time robotcentered 3D-2D vascular image alignment algorithm, which is robust to outliers and can align nonrigid shapes. Few works have managed to achieve both real-time and accurate performance for vascular intervention robots. This work bridges high-accuracy 3D-2D registration techniques and computational efficiency requirements in intervention robot applications. We categorize centerline-based vascular 3D-2D image registration problems as an iterative Perspective-n-Point (PnP) problem and propose using the Levenberg-Marquardt solver on the Lie manifold. Then, the recently developed Reproducing Kernel Hilbert Space (RKHS) algorithm is introduced to overcome the "big-to-small" problem in typical robotic scenarios. Finally, an iterative reweighted least squares is applied to solve RKHSbased formulation efficiently. Experiments indicate that the proposed algorithm processes registration over 50 Hz (rigid) and 20 Hz (nonrigid) and obtains competing registration accuracy similar to other works. Results indicate that our Iterative PnP is suitable for future vascular intervention robot applications.

This paper proposed an iterative learning control (ILC) scheme for deformable open-frame cable-driven parallel robots (D-CDPRs). In contrast to the straightforward inverse kinematics of the rigid frame cable-driven parallel robots (CDPRs), accurate modeling of the deformable frame poses challenges due to errors and uncertainties. To address these issues, the authors propose the use of ILC, a control strategy that modifies the control input over iterations based on previous results. ILC has been successfully applied to traditional cable robots, particularly in handling model uncertainty. The paper presents a novel ILC control scheme specifically designed for D-CDPRs, with a focus on reducing tracking errors over repetitive operations. Additionally, hardware experiments are conducted to validate the effectiveness and reliability of the proposed ILC approach. The results demonstrate the efficacy of ILC in mitigating tracking errors, even in scenarios where the dynamic model of the D-CDPRs is unknown.

Robot software is abundant with variables that represent real-world physical units (e.g., meters, seconds). Operations over different units (e.g., adding meters and seconds) may be incorrect and can lead to dangerous system misbehaviors; manually detecting such mistakes is challenging. Current software analysis techniques identify such mismatches using dimensional analysis rules and ROS-specific assumptions to analyze the source code. However, these are ignorant of the fact that physical unit mismatches in robotics code are often intentional (e.g., when operating a differential drive robot), resulting in false positive bug reports that can impede robotics developer trust and productivity. In this work, we study how developers introduce physical unit mismatches by manually inspecting 180 errors detected by the software analysis technique, Phys. We identify three types of physical unit mismatches and present a taxonomy of eight high-level categories of how these errors manifest. We find that developers often make unforced and paradigmatic physical unit mismatches through differential drives, small angle approximations, and controls. We draw insights on current development to inform future research to better detect, categorize, and address meaningful physical unit mismatches.

Cable-driven continuum manipulators have gained considerable attention due to their high dexterity and inherent structural compliance, making them a popular research topic. However, previous studies have overlooked the kinetostatics of these manipulators, which can result in a multi-solution problem. This issue is critical as having multiple equilibrium states can lead to erroneous estimations of the manipulator's profile. To address this issue, the kinetostatic model is presented and simulations based on both the interval analysis method and the commonly used floating-point optimization algorithm are conducted under the same actuating forces and external loads. Results show that there are multiple solutions to the kinetostatics of cable-driven continuum manipulators with constant cross section or variable cross section. This paper fills a gap in the current literature and offers valuable insights for researchers in the field of cable-driven continuum manipulators.

The capability of a quadruped robot to negotiate obstacles is tightly connected to its leg workspace and joint torque limits. When facing terrain where the height of obstacles is close to the leg length, the locomotion robustness and safety are reduced since more dynamic motions are required to traverse it. In this paper, we introduce a new mechanism called the Carpal-Claw, which enables quadruped robots to negotiate higher obstacles and adds safety to the locomotion by allowing the robot to negotiate obstacles under static and quasi-static locomotion and regular joint torque demands. The design of the mechanism is detailed, as well as the methodology to exploit the mechanism in the locomotion control framework. The Carpal-Claw functionality is validated through various experiments on a very high obstacle and stairs-like terrains using an Aliengo robot. We demonstrate how Aliengo can safely descend a step height of 40cm, which is 80% of its leg length. To the best knowledge of the authors, this is the first time a mechanism like the C-Claw is proposed for improving quadruped robot locomotion over high obstacles.

We introduce the CEA Industrial Multi-Scale Object Localization Dataset (CEA-IMSOLD), a new BOP format dataset for 6-DoF object localization, crucial for robotics. This dataset aims to evaluate the current localization methods with respect to a new difficulty: large variations in observation distance and, consequently, large variations in image appearance. Compared to the other publicly available datasets, our dataset provides both images with objects small and completely visible in the image, and images where objects are observed close enough so they appear larger than the field of view of the camera. We also propose to consider the observation distance in the evaluation process and introduce new metrics to do so. Finally, our dataset contains a large variety of industrial objects, from small and simple objects such as bolts to sizable and complex ones such as large car parts. We provide baseline results and the dataset is made publicly available to support the community at https://cea-list.github.io/CEA-IMSOLD/.

Current reinforcement learning algorithms struggle in sparse and complex environments, most notably in long-horizon manipulation tasks entailing a plethora of different sequences. In this work, we propose the Intrinsically Guided Exploration from Large Language Models (IGE-LLMs) framework. By leveraging LLMs as an assistive intrinsic reward, IGE-LLMs guides the exploratory process in reinforcement learning to address intricate long-horizon with sparse rewards robotic manipulation tasks. We evaluate our framework and related intrinsic learning methods in an environment challenged with exploration, and a complex robotic manipulation task challenged by both exploration and long-horizons. Results show IGE-LLMs (i) exhibit notably higher performance over related intrinsic methods and the direct use of LLMs in decision-making, (ii) can be combined and complement existing learning methods highlighting its modularity, (iii) are fairly insensitive to different intrinsic scaling parameters, and (iv) maintain robustness against increased levels of uncertainty and horizons.

This paper presents an intraoperatively iterative Hough transform (IHT) based in-plane hybrid control of extracorporeal ultrasound (US) guided magnetic catheterization for arterial intervention. One uniqueness lies in that both control and tracking of the arterial robotic ultrasound end-effector have been implemented to improve performance. Firstly, the magnetic catheter model and hybrid visual/force servoing control scheme of the extracorporeal ultrasound-integrated tracking arm (EUTA) are derived based on the interaction Jacobian matrix and impedance modeling. Meanwhile, we implement a tracking method of in-plane ultrasound catheter’s tip and detection of vascular boundaries utilizing intensity-level iterative Hough-transform with Iterative End-Ponit Fitting (IEPF). The effectiveness of the proposed control and tracking method has been verified by conducting in vitro experimental studies for catheter steering of a soft tissue-imitating phantom. Results show that an average steering error of 0.56 mm and signal-to-noise-ratio (SNR) of 12.2 are obtained for the ultrasound imaging at high synchronization along with a low target lost rate (15.8%) and constant-force tracking (2.50±1.02 N).

Designing robotic agents to perform open vocabulary tasks has been the long-standing goal in robotics and AI. Recently, Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. However, planning for these tasks in the presence of uncertainties is challenging as it requires "chain-of-thought" reasoning, aggregating information from the environment, updating state estimates, and generating actions based on the updated state estimates. In this paper, we present an interactive planning technique for partially observable tasks using LLMs. In the proposed method, an LLM is used to collect missing information from the environment using a robot, and infer the state of the underlying problem from collected observations while guiding the robot to perform the required actions. We also use a fine-tuned Llama 2 model via self-instruct and compare its performance against a pre-trained LLM like GPT-4. Results are demonstrated on several tasks in simulation as well as real-world environments.

This paper proposes an interactive navigation framework by using large language and vision-language models, allowing robots to navigate in environments with traversable obstacles. We utilize the large language model (GPT-3.5) and the open-set Vision-language Model (Grounding DINO) to create an action-aware costmap to perform effective path planning without fine-tuning. With the large models, we can achieve an end-to-end system from textual instructions like "Can you pass through the curtains to deliver medicines to me?", to bounding boxes (e.g., curtains) with action-aware attributes. They can be used to segment LiDAR point clouds into two parts: traversable and untraversable parts, and then an action-aware costmap is constructed for generating a feasible path. The pre-trained large models have great generalization ability and do not require additional annotated data for training, allowing fast deployment in the interactive navigation tasks. We choose to use multiple traversable objects such as curtains and grasses for verification by instructing the robot to traverse them. Besides, traversing curtains in a medical scenario was tested. All experimental results demonstrated the proposed framework’s effectiveness and adaptability to diverse environments.

For robots to perform assistive tasks in unstructured home environments, they must learn and reason on the semantic knowledge of the environments. Despite a resurgence in the development of semantic reasoning architectures, these methods assume that all the training data is available a priori. However, each user’s environment is unique and can continue to change over time, which makes these methods unsuitable for personalized home service robots. Although research in continual learning develops methods that can learn and adapt over time, most of these methods are tested in the narrow context of object classification on static image datasets. In this paper, we combine ideas from continual learning, semantic reasoning, and interactive machine learning literature and develop a novel interactive continual learning architecture for continual learning of semantic knowledge in a home environment through human-robot interaction. The architecture builds on core cognitive principles of learning and memory for efficient and real-time learning of new knowledge from humans. We integrate our architecture with a physical mobile manipulator robot and perform extensive system evaluations in a laboratory environment over two months. Our results demonstrate the effectiveness of our architecture to allow a physical robot to continually adapt to the changes in the environment from limited data provided by the users (experimenters), and use the learned knowledge to perform object fetching tasks.

Recently, pre-trained vision models have gained significant attention in motor control, showcasing impressive performance across diverse robotic learning tasks. While previous works predominantly concentrate on the significance of the pre-training phase, the equally important task of extracting more effective representations based on existing pre-trained visual models remains unexplored. To better leverage the representation capabilities of pre-trained models for robotic grasping, we propose InterRep, a novel interaction representation method that possesses not only the strengths of pre-trained models, known for their robustness in noisy environments and their proficiency in recognizing essential features, but also the capacity of capturing dynamic interaction details and local geometric features during the grasping process. Based on the novel representation, we introduce a deep reinforcement learning method to learn generalizable grasping policies. The experimental results demonstrate that our proposed representation outperforms the baselines in terms of both training speed and generalization. For the generalized grasping tasks with dexterous robotic hands, our method boasts a success rate nearly 20% higher than methods using the global features of the entire image from pre-trained models. In addition, our proposed representation method demonstrates promising performance when applied to a different robotic hand and task. It also exhibits excellent performance on real robots with a success rate of 70%.