IROS 2024 论文列表

Shared control can ease and enhance a human operator’s ability to teleoperate robots, particularly for intricate tasks demanding fine control over multiple degrees of freedom. However, the arbitration process dictating how much autonomous assistance to administer in shared control can confuse novice operators and impede their understanding of the robot’s behavior. To overcome these adverse side-effects, we propose a novel formulation of shared control that enables operators to tailor the arbitration to their unique capabilities and preferences. Unlike prior approaches to "customizable" shared control where users could indirectly modify the latent parameters of the arbitration function by issuing a feedback command, we instead make these parameters observable and directly editable via a virtual reality (VR) interface. We present our user-customizable shared control method for a teleoperation task in SE(3), known as the buzz wire game. A user study is conducted with participants teleoperating a robotic arm in VR to complete the game. The experiment spanned two weeks per subject to investigate longitudinal trends. Our findings reveal that users allowed to interactively tune the arbitration parameters across trials generalize well to adaptations in the task, exhibiting improvements in precision and fluency over direct teleoperation and conventional shared control.

While different control approaches have been developed for smooth and safe navigation, they are limited by the needs for model-based assumptions, true training target/reward function, and/or large sample data. To overcome these limitations, this study proposes a model-free neural control architecture with a generic plug-and-play online Multiple Proactive Behavior Learning (MPL) module. The MPL adapts robot neural control policy in an online unsupervised manner with small sample data by correlating its sensory inputs to a local planner command. As a result, it allows a mobile robot to autonomously and quickly learn and balance various proactive behaviors related to smooth motion and collision avoidance. It also compensates for the limited planning update rates and the planning model mismatch of an arbitrary local motion planner. Compared with existing control approaches without the MPL, our control architecture with the MPL leads to (1) a 10% improvement in the smoothness of robot motion and 30% fewer collisions in a narrow static environment, and (2) trading motion smoothness for up to 70% fewer collisions in an unknown dynamic environment. Taken together, this study also demonstrates how to apply model-free neural control with unsupervised learning to existing model-based control (e.g., local motion planner) for efficient proactive behavior learning and control of mobile robots.

We propose a novel unsupervised method for motion-based 3D part decomposition of articulated objects using a single monocular video of a dynamic scene. In contrast to existing unsupervised methods relying on optical flow or tracking techniques, our approach addresses this problem without additional information by leveraging Gaussian splatting techniques. We generate a series of Gaussians from a monocular video and analyze their relationships to decompose the dynamic scene into motion-based parts. To decompose dynamic scenes consisting of articulated objects, we design an articulated deformation field suitable for the movement of articulated objects. And to effectively understand the relationships of Gaussians of different shapes, we propose a 3D reconstruction loss using 3D occupied voxel maps generated from the Gaussians. Experimental results demonstrate that our method outperforms existing approaches in terms of 3D part decomposition for articulated objects. More demos and code are available at https://choonsik93.github.io/artnerf/.

A known challenge for computer vision methods applied to the underwater domain is that nonlinear attenuation of light in underwater environments distorts the color signal in captured imagery, resulting in inconsistent color and contrast at varying distances to an imaged target. While surface reflectance can provide a useful cue for classifying imagery of the seafloor by object or substrate types, color inconsistency makes robust classification challenging. We introduce a method that leverages hyperspectral imagery with an underwater light formation model and structure from motion to estimate the intrinsic optical properties of the underwater environment and correct seafloor reflectance estimates from radiance measurements. We show that our method enables consistent surface reflectance estimates under both artificial and ambient lighting conditions and is readily integrated on small underwater vehicle platforms, such as a BlueROV.

In this work, we aim to improve transparency and efficacy in human-robot collaboration by developing machine teaching algorithms suitable for groups with varied learning capabilities. While previous approaches focused on tailored approaches for teaching individuals, our method teaches teams with various compositions of diverse learners using team belief representations. We investigate various group teaching strategies, such as focusing on individual beliefs or the group’s collective beliefs, and assess their impact on learning robot policies for different team compositions. Our findings reveal that team belief strategies produce less variation in learning duration and better accommodate diverse teams compared to individual belief strategies, suggesting their suitability in mixed proficiency settings with limited resources. In contrast, individual belief strategies provide a more uniform knowledge level, particularly effective for homogeneously inexperienced groups. Our study indicates that the effectiveness of the teaching strategy is significantly influenced by team composition and learner proficiency, highlighting the importance of real-time assessment of learner proficiency and adapting teaching approaches based on learner proficiency for optimal teaching outcomes.

The CoM-ZMP model represents the dominant behaviour of bipedal locomotion with surface contact. However, once the centre of mass (CoM) position goes out of a predefined spatial plane, the horizontal dynamics of the model can couple with its vertical dynamics to be nonlinear. This study theoretically investigates the properties of the 3-dimensional zero moment point (ZMP), lying apart from the actual ground to resolve the coupling. The presented discussion includes the compatibility of the 3D ZMP with ZMPs used in preceding research, such as the linear inverted pendulum mode, the existence of a virtual repellent point considering the arbitrary vertical CoM motion, the parameter invariance of the CoM-ZMP model, and feasible regions of the ZMP.

Under-actuated robot grippers, as a pervasive tool of robots, have become a considerable research focus. Despite their simplicity of mechanical design and control strategy, they suffer from poor versatility and weak adaptability, making widespread applications limited. To better address relevant research gaps, we present a novel 3-finger linkage-based gripper that realizes retractable and reconfigurable multi-mode grasps driven by a single motor. Firstly, inspired by the changes occurred in the contact surface with a human finger moving, we artfully design a slider-slide rail mechanism as the phalanx to achieve retraction of each finger, allowing for better performance in the enveloping grasping mode. Secondly, a reconfigurable structure is constructed to broaden the grasping range of objects’ dimensions for the proposed gripper. By adjusting the configuration and gesture of each finger, the gripper can achieve five grasping modes. Thirdly, the proposed gripper is solely actuated by a single motor, yet it can be capable of grasping and reconfiguring simultaneously. Finally, various experiments on grasps of slender, thin, and large-volume objects are implemented to evaluate the performance of the proposed gripper in practical scenarios, which demonstrates the excellent grasping capabilities of the gripper.

Visual relative localization is widely used in multi-robot systems. While semantic key points offer a promising solution for 6DoF pose estimation, manual data labeling for network training remains unavoidable. In this paper, we introduce a novel method that jointly estimates the semantic key point detection model and 6DoF camera pose. Our key idea is to leverage the 3D-2D projection to produce pseudo labels for detection model training while taking the key point predictions as landmarks for 6DoF camera pose estimation. Compared with state-of-the-art works, our method eliminates the need for calibration and time synchronization of multi-camera systems, requiring only a handful of manually labeled data, which significantly improves the training efficiency. The experiment validates the effectiveness and practicality of our method in public datasets and real-world robotic applications. Code and data are made available3.

Large-scale robotic policies trained on data from diverse tasks and robotic platforms hold great promise for enabling general-purpose robots; however, reliable generalization to new environment conditions remains a major challenge. Toward addressing this challenge, we propose a novel approach for uncertainty-aware deployment of pre-trained language-conditioned imitation learning agents. Specifically, we use temperature scaling to calibrate these models and exploit the calibrated model to make uncertainty-aware decisions by aggregating the local information of candidate actions. We implement our approach in simulation using three such pre-trained models, and showcase its potential to significantly enhance task completion rates. The accompanying code is accessible at the link: https://github.com/BobWu1998/uncertainty_quant_all.git

Ultra-wideband (UWB) communication technology has gained attention in robotics due to its ability to provide range measurements possibly with centimeter-level accuracy. Nevertheless, practical UWB range measurements are susceptible to disturbances from multiple sources, including the anisotropic characteristics of antennas, non-line-of-sight (NLOS) conditions, and multipath propagation. In this paper, we introduce a UWB range measurement model that addresses these sources of error. To accommodate the effects of antenna anisotropy, we adopt real spherical harmonics to represent directional bias in the UWB range measurement model. To handle delayed measurements induced by NLOS conditions and multipath propagation, an asymmetric heavy-tailed distribution is utilized to model the measurement noise. We calibrate this measurement model based on the maximum likelihood estimation method and propose a UWB-based localization system based on that. Our localization system provides: 1) anchor self-calibration, which identifies anchor placement by fusing visual-inertial-ranging measurements based on continuous-time state representation; and 2) filtering-based state estimation, which applies our measurement model into Kalman filtering framework via an iterative update algorithm. Experimental validation is conducted to demonstrate the effectiveness of the measurement model for our localization system. We open source our implementation of the proposed UWB-based localization system at https://github.com/INRoL/inrol_uwb_localization.

Nonprehensile manipulation through precise pushing is an essential skill that has been commonly challenged by perception and physical uncertainties, such as those associated with contacts, object geometries, and physical properties. For this, we propose a unified framework that jointly addresses system modeling, action generation, and control. While most existing approaches either heavily rely on a priori system information for analytic modeling, or leverage a large dataset to learn dynamic models, our framework approximates a system transition function via non-parametric learning only using a small number of exploratory actions (ca. 10). The approximated function is then integrated with model predictive control to provide precise pushing manipulation. Furthermore, we show that the approximated system transition functions can be robustly transferred across novel objects while being online updated to continuously improve the manipulation accuracy. Through extensive experiments on a real robot platform with a set of novel objects and comparing against a state-of-the-art baseline, we show that the proposed unified framework is a light-weight and highly effective approach to enable precise pushing manipulation all by itself. Our evaluation results illustrate that the system can robustly ensure millimeter-level precision and can straightforwardly work on any novel object.

Anomaly detection is critical in surveillance systems and patrol robots by identifying anomalous regions in images for early warning. Depending on whether reference data are utilized, anomaly detection can be categorized into anomaly detection with reference and anomaly detection without reference. Currently, anomaly detection without reference, which is closely related to out-of-distribution (OoD) object detection, struggles with learning anomalous patterns due to the difficulty of collecting sufficiently large and diverse anomaly datasets with the inherent rarity and novelty of anomalies. Alternatively, anomaly detection with reference employs the scheme of change detection to identify anomalies by comparing semantic changes between a reference image and a query one. However, there are very few ADr works due to the scarcity of public datasets in this domain. In this paper, we aim to address this gap by introducing the UMAD Benchmark Dataset. To our best knowledge, this is the first benchmark dataset designed specifically for anomaly detection with reference in robotic patrolling scenarios, e.g., where an autonomous robot is employed to detect anomalous objects by comparing a reference and a query video sequences. The reference sequences can be taken by the robot along a specified route when there are no anomalous objects in the scene. The query sequences are captured online by the robot when it is patrolling in the same scene following the same route. Our benchmark dataset is elaborated such that each query image can find a corresponding reference based on accurate robot localization along the same route in the pre-built 3D map, with which the reference and query images can be geometrically aligned using adaptive warping. Besides the proposed benchmark dataset, we evaluate the baseline models of ADr on this dataset. We hope this benchmark dataset will facilitate the advancement of ADr methods in the future. Our UMAD benchmark dataset will be publicly accessible at... Show More

Efficient relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails. Recent advances in Bird’s-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance and in turn, can benefit the relocalization of the vehicle. However, one downside of BEV methods is the heavy computation required to leverage the geometric constraints. This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features. We show that this extension boosts the performance of the U-BEV by up to 4.11 IoU. Additionally, we combine the encoded neural BEV with a differentiable template matcher to perform relocalization on neural SD-map data. The model is fully end-to-end trainable and outperforms transformer-based BEV methods of similar computational complexity by 1.7 to 2.8 mIoU and BEV-based relocalization by over 26% Recall Accuracy on the nuScenes dataset.

Visibility is a fundamental concept in computational geometry, with numerous applications in surveillance, robotics, and games. This software paper presents TřiVis, a C++ library developed by the authors for computing numerous visibility-related queries in highly complex polygonal environments. Adapting the triangular expansion algorithm, TřiVis stands out as a versatile, high-performance, more reliable and easy-to-use alternative to current solutions that is also free of heavy dependencies. Through evaluation on a challenging dataset, TřiVis has been benchmarked against existing visibility libraries. The results demonstrate that TřiVis outperforms the competing solutions by at least an order of magnitude in query times, while exhibiting more reliable runtime behavior. TřiVis is freely available for private, research, and institutional use at https://github.com/janmikulacz/trivis.

The demand for 3D shape measurement of underwater scene is increasing in various applications. Especially, simultaneous localization and mapping (SLAM) technique utilizing remotely operated vehicle (ROV) attached with 3D sensors has been intensively researched. This paper focuses on solving pose optimization problem for underwater robots with camera/multiple-line-lasers setup, especially for the scene with some textures (color information). To this end, a two-stage pose optimization technique is proposed. In the first stage, due to the sparse nature of the reconstructed shape in the light-sectioning method consisting of several 3D curves, we bundle 10 to 20 consecutive frames to form a block shape, refining significant errors in the initial sensor poses using a novel bundle adjustment algorithm. In the second stage, remaining pose errors are corrected by a block-based matching algorithm utilizing iterative closest point (ICP) algorithm with color information. Through experiments in underwater environment with a real system, it was validated that the proposed method demonstrates superior performance compared to past underwater SLAM techniques.

We present a novel unsupervised obstacle segmentation architecture that follows a novel Relation Distillation (RD) paradigm. Our architecture design was inspired by a self-supervised teacher-student approach that relies on the Semantic Distillation originally devised for representation learning. While the teacher in the Semantic Distillation considers a single patch at a time, the teacher within RD takes a ‘pair of patches’ instead to transfer the local Semantic Co-occurrence Localization (SCooL) relationship that focuses more on the segmentation-boosting signals. To further improve the proposed architecture, we introduce the utilization of another teacher that leverages the depth information which inherently separates the entities at different physical distances, often tied with the boundaries of the obstacles. As the depth is distilled towards the student network only at the time of training, it adds zero computational/hardware cost at run-time. As no relevant public dataset is available, we have curated the Avoiding Obstacles In unstructured Driving (AvOID) dataset as a new testbed for unsupervised obstacle segmentation. We have validated that both the Relation Distillation and depth contribute to boosting the no-annotation segmentation performance on AvOID and KITTI-Obstacles.

Tremors, which are prevalent symptoms in both Parkinson’s disease (PD) and essential tremor (ET), substantially diminish the quality of life for those affected. Traditional treatments, including pharmaceutical medications and invasive surgical procedures, often come with limitations and side effects, prompting the need for alternative solutions. In this paper, the Tunable Stiffness Glove (TSG) is developed as a non-invasive exoskeleton to suppress wrist tremors. By employing chain mail fabrics with tunable stiffness, the TSG permits natural wrist movement when in a soft state, while transitioning to a jammed state upon the application of negative pressure, effectively suppressing tremors in two directions. Evaluation of the TSG’s efficacy was conducted through comprehensive three-point bending tests and human trials, utilizing commercial mechanical tester, inertial measurement unit (IMU), and electromyography (EMG) sensors. Weighing a mere 92 grams, the TSG demonstrated remarkable tremor suppression rates of 74.86%±5.52% (in the flexion-extension direction) and 66.80%±15.47% (in the adduction-abduction direction). Future enhancements aim to optimize the design for increased damping force and integrate sophisticated control strategies for improved user-exoskeleton interaction.

Virtual tube is a two-dimensional or three-dimensional strip or tubular area similar to RSFC (Relative Safe Flight Corridor), which can provide smooth, feasible, and safe space for UAV swarm in environments with dense obstacles. In order to address the problem that current virtual tube planning methods are mainly based on complex heuristic algorithm with consuming time complexity, we modify the model architecture by introducing generative adversarial network (GAN), and propose a Tube-GAN model. Tube-GAN takes the key point prompt image and obstacle environment image as inputs, and outputs the image of the virtual tube, which transforms the optimization problem into an image generation problem, leveraging the performance of computational efficiency for the construction of virtual tube. The experimental results demonstrate that the proposed Tube-GAN model can quickly generate virtual tube in random environments (less than 25ms), providing a new direction for the construction of virtual tube in real-time.

Large language models (LLMs) exhibit a wide range of promising capabilities – from step-by-step planning to commonsense reasoning –that provide utility for robot navigation. However, as humans communicate with robots in the real world, ambiguity and uncertainty may be embedded inside spoken instructions. While LLMs are proficient at processing text in human conversations, they often encounter difficulties with the nuances of verbal instructions and, thus, remain prone to hallucinate trust in human command. In this work, we present TrustNavGPT, an LLM-based audio-guided navigation agent that uses affective cues in spoken communication—elements such as tone and inflection that convey meaning beyond words—allowing it to assess the trustworthiness of human commands and make effective, safe decisions. Experiments across a variety of simulation and real-world setups show a 70.46% success rate in catching command uncertainty and an 80% success rate in finding the target, 48.30%, and 55% outperform existing LLM-based navigation methods, respectively. Additionally, TrustNavGPT shows remarkable resilience against adversarial attacks, highlighted by a 22%+ less decrease ratio than the existing LLM navigation method in success rate. Our approach provides a lightweight yet effective approach that extends existing LLMs to model audio vocal features embedded in the voice command and model uncertainty for safe robotic navigation. For more information, visit the TrustNav project page.