IROS 2023 论文列表

Transparent objects are common in daily life. However, depth sensing for transparent objects remains a challenging problem. While learning-based methods can leverage shape priors to improve the sensing quality, the labor-intensive data collection in real world and the sim-to-real domain gap restrict these methods' scalability. In this paper, we propose a method to finetune a stereo network with sparse depth labels automatically collected using a probing system with tactile feedback. We present a novel utility function to evaluate the benefit of touches. By approximating and optimizing the utility function, we can optimize the probing locations given a fixed touching budget to better improve the network's performance on real objects. We further combine tactile depth supervision with a confidence-based regularization to prevent over-fitting during finetuning. To evaluate the effectiveness of our method, we construct a real-world dataset including both diffuse and transparent objects. Experimental results on this dataset show that our method can significantly improve real-world depth sensing accuracy, especially for transparent objects.

Despite radar's popularity in the automotive industry, for fusion-based 3D object detection, most existing works focus on LiDAR and camera fusion. In this paper, we propose TransCAR, a Transformer-based Camera-And-Radar fusion solution for 3D object detection. Our TransCAR consists of two modules. The first module learns 2D features from surround-view camera images and then uses a sparse set of 3D object queries to index into these 2D features. The vision-updated queries then interact with each other via transformer self-attention layer. The second module learns radar features from multiple radar scans and then applies transformer decoder to learn the interactions between radar features and vision-updated queries. The cross-attention layer within the transformer decoder can adaptively learn the soft-association between the radar features and vision-updated queries instead of hard-association based on sensor calibration only. Finally, our model estimates a bounding box per query using set-to-set Hungarian loss, which enables the method to avoid non-maximum suppression. TransCAR improves the velocity estimation using the radar scans without temporal information. The superior experimental results of our TransCAR on the challenging nuScenes datasets illustrate that our TransCAR outperforms state-of-the-art Camera-Radar fusion-based 3D object detection approaches.

In this paper we introduce a novel SLAM system for 2-D indoor environments that relies only on limited sensing. Our fully autonomous system uses only the trajectory of the robot around walls and objects in the environment as landmarks and is capable of robust and long-term exploration and mapping of a broad range of household floor plans. Rank-deficient and full-rank factors are created when the robot observes existing trajectory-based landmarks, and they are filtered and added in a pose graph, which is optimized periodically. The mission space is mapped by efficient adaptive local mapping algorithms. The proposed SLAM system has been extensively tested in various scenarios, and experimental results show its robustness and accuracy.

Animals and insects showcase remarkably robust and adept navigational abilities, up to literally circumnavigating the globe. Primary progress in robotics inspired by these natural systems has occurred in two areas: highly theoretical computational neuroscience models, and handcrafted systems like RatSLAM and NeuroSLAM. In this research, we present work bridging the gap between the two, in the form of Multiscale Continuous Attractor Networks (MCAN), that combine the multiscale parallel spatial neural networks of the previous theoretical models with the real-world robustness of the robot-targeted systems, to enable trajectory tracking over large velocity ranges. To overcome the limitations of the reliance of previous systems on hand-tuned parameters, we present a genetic algorithm-based approach for automated tuning of these networks, substantially improving their usability. To provide challenging navigational scale ranges, we open source a flexible city-scale navigation simulator that adapts to any street network, enabling high throughput experimentation11https://github.com/theresejoseph/Trajectory_ Tracking_via_MCAN/ • In extensive experiments using the city-scale navigation environment and Kitti, we show that the system is capable of stable dead reckoning over a wide range of velocities and environmental scales, where a single-scale approach fails.

Visual Place Recognition (VPR) utilizing patch descriptors from Convolutional Neural Networks (CNNs) has shown impressive performance in recent years. Existing works either perform exhaustive matching of all patch descriptors, or employ complex networks to select good candidate patches for further geometric verification. In this work, we develop a novel two-step training-free patch selection method that is fast, while being robust to large occlusions and extreme viewpoint variations. In the first step, a self-attention mechanism is used to select sparse and evenly distributed discriminative patches in the query image. Next, a novel spatial-matching method is used to rapidly select corresponding patches with high similar appearances between the query and each reference image. The proposed method is inspired by how humans perform place recognition by first identifying prominent regions in the query image, and then relying on back-and-forth visual inspection of the query and reference image to attentively identify similar regions while ignoring dissimilar ones. Extensive experiment results show that our proposed method outperforms state-of-the-art (SOTA) methods in both place recognition precision and runtime, on various challenging conditions.

Traffic accident recognition is essential in developing automated driving and Advanced Driving Assistant System technologies. A large dataset of annotated traffic accidents is necessary to improve the accuracy of traffic accident recognition using deep learning models. Conventional traffic accident datasets provide annotations on the presence or absence of traffic accidents and other teacher labels, improving traffic accident recognition performance. However, the labels annotated in conventional datasets need to be more comprehensive to de-scribe traffic accidents in detail. Therefore, we propose V-TIDB, a large-scale traffic accident recognition dataset annotated with various environmental information as multi-labels. Our proposed dataset aims to improve the performance of traffic accident recognition by annotating ten types of environmental information as teacher labels in addition to the presence or absence of traffic accidents. V-TIDB is constructed by collecting many videos from the Internet and annotating them with appropriate environmental information. In our experiments, we compare the performance of traffic accident recognition when only labels related to the presence or absence of traffic accidents are trained and when environmental information is added as a multi-label. In the second experiment, we compare the performance of the training with only “contact level,” which represents the severity of the traffic accident, and the performance with environmental information added as a multi-label. The results showed that 6 out of 10 environmental information labels improved the performance of recognizing the presence or absence of traffic accidents. In the experiment on the degree of recognition of traffic accidents, the performance of recognition of car wrecks and contacts was improved for all environmental information. These experiments show that V-TIDB can be used to learn traffic accident recognition models that take environmental information into account i... Show More

We present a novel algorithm to solve the stochastic orienteering problem with chance constraints that combines Monte Carlo Tree Search (MCTS) with a best arm identification (BAI) algorithm. This method extends our recently proposed solution that builds a search planning tree considering both an objective function to maximize, as well as a chance constraint on the failure probability, i.e., the probability of violating the assigned budget constraint. By combining these two approaches, we obtain a new planner that tunes the amount of tree search at run time. Extensive simulation results on our benchmark problems show that the new approach is significantly faster than the previous one, while incurring in just marginal decrements in terms of performance.

This paper presents a novel augmented reality (AR)-assisted orthopedic surgical robotic system based on Head-Mounted Display (HMD) devices. The proposed system can overlay the preoperative plans over the patient's anatomy and provide useful guidance for surgeons during interventions, with integrated calibration and registration components. A novel bi-directional generalised point set registration algorithm that utilises robust features is developed to accurately align the pre-operative CT and intra-operative patient spaces, which has been demonstrated to outperform existing registration methods. The efficacy of the system is both qualitatively and quantitatively assessed with an in vitro study simulating a total knee arthroplasty (TKA) procedure. The experimental results showed that 1) the system can successfully align the preoperative and intraoperative spaces, with the mean target registration error (TRE) being 2.7771 mm; 2) the models can be properly overlaid to the physical scenarios with the mean AR visualization accuracy being 6.9726 mm.

Object detection techniques for autonomous Un-manned Aerial Vehicles (UAV) are built upon Deep Neural Networks (DNN), which are known to be vulnerable to adversarial patch perturbation attacks that lead to object detection evasion. Yet, current adversarial patch generation schemes are not designed for UAV imagery settings. This paper proposes a new robust adversarial patch generation attack against object detection with UAVs. We build adversarial patches considering UAV-specific settings such as the UAV camera perspective, viewing angle, distance, and brightness changes. As a result, built patches can also degrade the accuracy of object detector models implemented with different initializations and architectures. Experiments conducted on the VisDrone dataset have shown the proposal's feasibility, achieving an attack success rate of up to 80% in a white-box setting. In addition, we also transfer the patch against DNN models with different initializations and different architectures, reaching attack success rates of up to 75% and 78%, respectively, in a gray-box setting. GitHub: https://github.com/SamSamhuns/yolov5_adversarial

In this paper, we present a framework to generate time-optimal trajectories for dynamic target pick-and-place tasks. We develop an optimization-based trajectory generation method for manipulators, which can conduct spatial-temporal deformation under user-defined requirements. We formulate the problem of dynamic target pick-and-place, in which the trajectory duration and jerk are optimized and terminal states are adjusted instead of being fixed. The motions are constrained within the mechanical limits and to avoid collisions. Constraints transcription is adopted to convert constraints to weighted penalties. Then the problem can be solved based on the trajectory generation method with a high-level optimizer. We integrate the proposed method with online perception into a robot arm platform, in which a conveyor belt is used to transport the objects. Simulations and real-world experiments are conducted under a range of object speeds. Results show that the proposed method achieves online grasping under the object velocity up to 0.5m/s with an average computing time of 190ms.

For various automated palletizing tasks, the detection of packaging units is a crucial step preceding the actual handling of the packaging units by an industrial robot. We propose an approach to this challenging problem that is fully trained on synthetically generated data and can be robustly applied to arbitrary real world packaging units without further training or setup effort. The proposed approach is able to handle sparse and low quality sensor data, can exploit prior knowledge if available and generalizes well to a wide range of products and application scenarios. To demonstrate the practical use of our approach, we conduct an extensive evaluation on real-world data with a wide range of different retail products. Further, we integrated our approach in a lab demonstrator and a commercial solution will be marketed through an industrial partner.

Magnetic resonance imaging (MRI) is increasingly used for robotic needle-based clinical diagnosis and therapy due to its high resolution and high soft tissue contrast. A needle driver therefore becomes an essential part of these MRI -guided robotic systems to perform in-bore needle placement under continuous MR imaging. However, existing actuator designs for needle drivers are constrained by the high magnetic field and the bore size of the MR scanner, and most lack high- performance capability, especially in terms of large output force and long stroke. In this work, we introduce an MR-safe concentric bellows linear actuator (CBLA) with improved performance over the existing designs. This soft actuator achieves 45N output force, 77 % stroke-length ratio for the bellows, and around 250% stroke-diameter ratio. A mathematical model was built to characterize the behavior of the actuator to provide guidance for the actuator design. The fabrication and experimental evaluation of the actuator are also presented. The results demonstrate the effectiveness of the CBLA design and the strong potential of its application in MR-guided surgical procedures.

Scientific exploration of planetary bodies is an activity well-suited for robots. Unfortunately, the regions that are richer in potential discoveries, such as impact craters, caves, and volcanic terraces, are hard to access with wheeled robots. Recent advances in legged-based approaches have shown the potential of the technology to overcome difficult terrains such as slopes and slippery surfaces. In this work, we focus on locomotion for sandy slopes, comparing standard walking policies with a novel crawling-based gait for quadrupedal robots. We fine-tuned a state-of-the-art locomotion framework and introduced hardware modifications to the robot ANYmal, which enables walking on its knees. Moreover, we integrated a novel metric for stability, the stability margin, in the training process to increase robustness in such conditions. We benchmarked the locomotion policies in simulation and in real-world experiments on a martian soil simulant. Our results show a significant improvement in terms of robustness and stability, especially at higher slope angles beyond 15 degrees.

Exploration and monitoring of hazardous environments, such as legacy nuclear storage ponds, constitute safety-critical missions to be performed by small-scale underwater robots. These monitoring tasks require fully actuated robot platforms in order to allow for hovering while inspecting objects of interest in detail. A severe bottleneck arises from the restricted access points commonly encountered in these surveillance missions that pose strict limitations on the vehicle dimensions. However, small-scale underwater robots usually possess underactuated propulsion systems and are, thus, only partially suitable for these missions. In this work, we investigate and exploit the idea of reconfigurability. Following the idea of the whole is more than the sum of its parts, we daisy-chain multiple small-scale underwater robots with revolute joints to enable shape reconfigurations. In combination with a centralized sliding mode control scheme, the robot platform is able to change its shape depending on the current task, see Fig. 1. While the straight configuration fits well through tight passages such as inspection holes, the robot can reconfigure itself towards a triangular shape that enables the neutrally buoyant robot to hover at areas of interest, e. g. for inspection tasks. Finally, we examine our proposed concept in a series of simulations and experiments. Moreover, we demonstrate the performance of key elements such as reconfiguration and navigation, discuss their limitations, and point out future directions.

Automation in the life science research laboratory is a paradigm that has gained increasing relevance in recent years. Current robotic solutions often have a limited scope, which reduces their acceptance and prevents the realization of complex workflows. The transport and manipulation of laboratory supplies with a robot is a particular case where this limitation manifests. In this paper, we deduce a taxonomy of biolaboratory liquid containers that clarifies the need for a flexible grasping solution. Using the taxonomy as a guideline, we design fingers for a parallel robotic gripper which are developed with a monolithic dual-extrusion 3D print that integrates rigid and soft materials to optimize gripping properties. We design fine-tuned fingertips that provide stable grasps of the containers in question. A simple actuation system and a low weight are maintained by adopting a passive compliant mechanism. The ability to resist chemicals and high temperatures and the integration with a tool exchange system render the fingers usable for daily laboratory use and complex workflows. We present the task suitability of the fingers in experiments that show the wide range of vessels that can be handled as well as their tolerance against displacements and their grasp stability.

The cooperation of a human pilot with an autonomous agent during flight control realizes parallel autonomy. We propose an air-guardian system that facilitates cooperation between a pilot with eye tracking and a parallel end-to-end neural control system. Our vision-based air-guardian system combines a causal continuous-depth neural network model with a cooperation layer to enable parallel autonomy between a pilot and a control system based on perceived differences in their attention profiles. The attention profiles for neural networks are obtained by computing the networks' saliency maps (feature importance) through the VisualBackProp algorithm, while the attention profiles for humans are either obtained by eye tracking of human pilots or saliency maps of networks trained to imitate human pilots. When the attention profile of the pilot and guardian agents align, the pilot makes control decisions. Otherwise, the air-guardian makes interventions and takes over the control of the aircraft. We show that our attention-based air-guardian system can balance the trade-off between its level of involvement in the flight and the pilot's expertise and attention. The guardian system is particularly effective in situations where the pilot was distracted due to information overload. We demonstrate the effectiveness of our method for navigating flight scenarios in simulation with a fixed-wing aircraft and on hardware with a quadrotor platform.

The identification of human joint impedance is necessary for various applications, such as improving rehabilitation efficiency or monitoring the human operator's state (fatigue, stress). To this end, in this paper we combine robot's payload identification methods with sliding window recursive least squares algorithm allowing a continuous identification of the varying human joint model without the need for external sensors. We also propose a threshold for detecting fake changes in the identified model parameters due to numerical issues. The presented approach is validated by simulations and experiments using elastic rubber bands representing a simplified passive human joint model attached to a one degree of freedom robotic system. Comparison with simple recursive least squares shows that the proposed method is promising, as it converges to the new parameter in a single window length, whereas the other method takes much longer. In addition, it distinguishes real from fake changes depending on the validity of the used model.

One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Al-though several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully handcrafted obstacle descriptions. In this work, we propose a method combining high-frequency and real-time self and environment collision avoidance of a robotic manipulator with low-frequency, multimodal, and high-resolution environmental perceptions accumulated in a digital twin system. Our method is based on geometric primitives, so-called primitive skeletons. These, in turn, are information-compressed and real-time compatible digital representations of the robot's body and environment, automatically generated from ultra-realistic virtual replicas of the real world provided by the digital twin. Our approach is a key enabler for closing the loop between environment perception and robot control by providing the millisecond real-time control stage with a current and accurate world description, empowering it to react to environmental changes. We evaluate our whole-body collision avoidance on a 9-DOFs robot system through five experiments, demonstrating the functionality and efficiency of our framework.

Autonomous mobile robot navigation in a human populated and encumbered environment is recognized as a hard problem to be solved in real-time. Most of the time, robots face the so-called ‘Freezing Robot Problem’, that occurs when the robot stops because no feasible and safe motion can be found. In order to provide to the robot the capability of proactive navigation, in this work we generalize the classical Social Force Model into a Universal Social Force Model (USFM) that attributes to any object surrounding the robot (humans, robots, obstacles) a social behavior. Nonlinear Model Predictive Control (MPC) can be used to solve the autonomous navigation problem since it can take into account all the possible constraints coming from the interaction model between the robot and the different surrounding objects. However, to be effective, MPC requires a sufficiently large prediction horizon, which generally implies a high computational cost. In order to considerably reduce the computational cost, we propose a new control parametrisation based on Thin Plate Spline Radial Basis Functions that allow us to have a large prediction horizon with fewer parameters. The global control framework is validated in simulation with virtual pedestrians, and in real world environments.