ICRA 2023 论文列表

We propose a framework to enable multipurpose assistive mobile robots to autonomously wipe tables to clean spills and crumbs. This problem is challenging, as it requires planning wiping actions while reasoning over uncertain latent dynamics of crumbs and spills captured via high-dimensional visual observations. Simultaneously, we must guarantee constraints satisfaction to enable safe deployment in unstructured cluttered environments. To tackle this problem, we first propose a stochastic differential equation to model crumbs and spill dynamics and absorption with a robot wiper. Using this model, we train a vision-based policy for planning wiping actions in simulation using reinforcement learning (RL). To enable zero-shot sim-to-real deployment, we dovetail the RL policy with a whole-body trajectory optimization framework to compute base and arm joint trajectories that execute the desired wiping motions while guaranteeing constraints satisfaction. We extensively validate our approach in simulation and on hardware. Video of experiments: https://youtu.be/inORKP4F3EI

Ultrasound is a vital imaging modality utilized for a variety of diagnostic and interventional procedures. However, an expert sonographer is required to make accurate maneuvers of the probe over the human body while making sense of the ultrasound images for diagnostic purposes. This procedure requires a substantial amount of training and up to a few years of experience. In this paper, we propose an autonomous robotic ultrasound system that uses Bayesian Optimization (BO) in combination with the domain expertise to predict and effectively scan the regions where diagnostic quality ultrasound images can be acquired. The quality map, which is a distribution of image quality in a scanning region, is estimated using Gaussian process in BO. This relies on a prior quality map modeled using expert's demonstration of the high-quality probing maneuvers. The ultrasound image quality feedback is provided to BO, which is estimated using a deep convolution neural network model. This model was previously trained on database of images labelled for diagnostic quality by expert radiologists. Experiments on three different urinary bladder phantoms validated that the proposed autonomous ultrasound system can acquire ultrasound images for diagnostic purposes with a probing position and force accuracy of 98.7% and 97.8%, respectively.

In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT intro-duces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.

Condition synthesis is vital for generating data for fault detection and diagnosis studies. Traditional methods rely heavily on human labor. This study proposes a robotic method and its instru-ment to efficiently synthesize faulty conditions and mass-produce data to develop fault detection and diagnosis algorithms. The first contribution is the formalization of a new approach called Robotic Condition Synthesis, which shifts the traditionally labor-intensive task of condition synthesis to a robot-based force control task. The second contribution is developing a new robotic manipulator, which is more effective than current lab-grade robots for the tasks involved in the Robotic Condition Synthesis. The third contribution is empirical evidence of the superiority of this new robot in performing the Robotic Condition Synthesis tasks. This study also explores the potential of the new robot by conducting a three-dimensional system identification of a rotordynamic plant, which lays the foundation for more advanced Robotic Condition Synthesis policies in the future.

The robotic hand is still no match for the human hand on many skills. Manipulation of hand tools, which usually requires sophisticated finger movements and fine controls, not only poses a clear technical challenge but also carries a great potential for enabling the robot to assist humans in a wide range of tasks accomplishable using tools. This paper takes a first step to investigate how a robotic arm mounts a rigidly attached screwdriver onto a screw (pre-mounted in a tapped hole) and then tightens it using the tool. Mounting begins with sliding the screwdriver tip on the screw head along preplanned paths to search for the drive and follows with rotating the screwdriver to drop the tip into the drive. Prevention of a slip off the screw head is achieved via impedance control to install a “virtual fence” along its boundary. Turning of the screw is conducted via hybrid position/admittance control based on modeling the reaction force between the screw and the substrate. Simulation results with a KUKA Arm demonstrate the smoothness of the entire action.

In machine learning, meta-learning methods aim for fast adaptability to unknown tasks using prior knowledge. Model-based meta-reinforcement learning combines reinforcement learning via world models with Meta Reinforcement Learning (MRL) for increased sample efficiency. However, adaption to unknown tasks does not always result in preferable agent behavior. This paper introduces a new Meta Adaptation Controller (MAC) that employs MRL to apply a preferred robot behavior from one task to many similar tasks. To do this, MAC aims to find actions an agent has to take in a new task to reach a similar outcome as in a learned task. As a result, the agent will adapt quickly to the change in the dynamic and behave appropriately without the need to construct a reward function that enforces the preferred behavior.

Tactile sensing is essential for robots to perceive and react to the environment. However, it remains a challenge to make large-scale and flexible tactile skins on robots. Industrial machine knitting provides solutions to manufacture customiz-able fabrics. Along with functional yarns, it can produce highly customizable circuits that can be made into tactile skins for robots. In this work, we present RobotSweater, a machine-knitted pressure-sensitive tactile skin that can be easily applied on robots. We design and fabricate a parameterized multi-layer tactile skin using off-the-shelf yarns, and characterize our sensor on both a flat testbed and a curved surface to show its robust contact detection, multi-contact localization, and pressure sensing capabilities. The sensor is fabricated using a well-established textile manufacturing process with a programmable industrial knitting machine, which makes it highly customizable and low-cost. The textile nature of the sensor also makes it easily fit curved surfaces of different robots and have a friendly appearance. Using our tactile skins, we conduct closed-loop control with tactile feedback for two applications: (1) human lead-through control of a robot arm, and (2) human-robot interaction with a mobile robot.

Robot-assisted eye-hand coordination rehabilitation training system is extremely urgent to study since recent evidence suggests that eye-hand coordination can be brutally disturbed by stroke with critical consequences on motor behavior. In this paper, we develop a robot-assisted eye-hand coordination training system by estimating motion direction using smooth-pursuit eye movements. Firstly, we design a Pong Game, which requires users to extrapolate the direction of a linearly moving ball and to predict whether this ball would be hit. Secondly, the motion direction of the ball is estimated via smooth-pursuit eye movements, allowing the robot quickly establish an assistive force field to hit the ball. Thirdly, adding haptic feedback technology into this training system to make users more immersive. Finally, we conduct a feasibility study with eight healthy subjects to verify the effectiveness of the proposed system. The experimental results show that the mean success rate for hitting the pong ball of the experiment group (assistance turn-on) is 28.33% higher than that of the control group (assistance turn-off), and the mean interception time of the experiment group is 0.35s shorter than that of the control group. Therefore, the developed system may be promising for transferring to the robot-assisted eye-hand coordination rehabilitation training for post-stroke patients.

In the future, robots are expected to autonomously interact and/or collaborate with humans, who will increase the uncertainty during the execution of tasks, provoking online adaptations of robots' plans. Hence, trustworthy robots must be able to store, retrieve and narrate important knowledge about their collaborations and adaptations. In this article, it is proposed a sound methodology that integrates three main elements. First, an ontology for collaborative robotics and adaptation to model the domain knowledge. Second, an episodic memory for time-indexed knowledge storage and retrieval. Third, a novel algorithm to extract the relevant knowledge and generate textual explanatory narratives. The algorithm produces three different types of outputs, varying the specificity, for diverse uses and preferences. A pilot study was conducted to evaluate the usefulness of the narratives, obtaining promising results. Finally, we discuss how the methodology can be generalized to other ontologies and experiences. This work boosts robot explainability, especially in cases where robots need to narrate the details of their short and long-term past experiences.

Role arbitration in human-robot collaboration (HRC) is a dynamically changing process that is affected by many factors such as physical workload, environmental changes and trust. In order to address this dynamic process, a trust-based role arbitration method is studied in this research. A computational model of robot trust and self-confidence (TSC) in physical human-robot collaboration (pHRC) is proposed. The TSC model is defined as a function of objective robot and human co-worker performance. A role arbitration method is then proposed based on the TSC model presented. The human-in-the-loop experiments with a collaborative robot are conducted to verify the TSC-based role arbitration method. The results show that the proposed method could achieve superior human-robot combined performance, reduce human co-workers' workload, and improve subjective preference.

Robot person following (RPF) is a capability that supports many useful human-robot-interaction (HRI) applications. However, existing solutions to person following often as-sume full observation of the tracked person. As a consequence, they cannot track the person reliably under partial occlusion where the assumption of full observation is not satisfied. In this paper, we focus on the problem of robot person following under partial occlusion caused by a limited field of view of a monocular camera. Based on the key insight that it is possible to locate the target person when one or more of hislher joints are visible, we propose a method in which each visible joint contributes a location estimate of the followed person. Experiments on a public person-following dataset show that, even under partial occlusion, the proposed method can still locate the person more reliably than the existing SOTA methods. As well, the application of our method is demonstrated in real experiments on a mobile robot.

Future robots will be very advanced with high flexibility and accurate control performance. They will have the ability to mimic human behaviours or even perform better, which raises the significant risk of robot attack. In this work, we study the robot mimic attack on the current keystroke-dynamic user authentication system. Specifically, we proposed a robot mimicry attack framework for keystroke-dynamics systems. We collected keyboard logging data and acoustical signal data from real users and extracted the timing pattern of keystrokes to understand victim's behaviour for robot imitation attacks. Furthermore, we develop a deep Q-Network (DQN) algorithm to control the velocity of robot which is one of the key challenges of forging the human typing timing features. We tested and evaluated our approach on the real-life robotic testbed. We presented our results considering user identification and user authentication performance. We achieved a 90.3% user identification accuracy with genuine keyboard logging data samples and 89.6% accuracy with robot-forged data samples. Furthermore, we achieved 11.1%, and 36.6% EER for user authentication performance with zero-effort attack, and robot mimicry attack, respectively.

The paper presents a novel domain-specific language, RoboSC, for developing supervisory controllers for robotic applications. RoboSC supports concepts of ROS/ROS2 and supervisory control theory. It enables users to focus on the modeling and the synthesis process of supervisory controllers for ROS applications only because it generates all artifacts needed to connect such controllers to ROS applications and deploy them. Validation tests with actual and simulated robots show the approach's feasibility and indicate reduced coding effort.

Anomaly segmentation on the urban landscape scene is an important task in autonomous driving. This process exploits a pre-trained semantic segmentation network to estimate anomalous regions. Anomaly segmentation approaches implemented with extra requirements such as out-of-domain data, extra network, or network retraining might increase the computational cost or degradation of segmentation performance. In this study, to exploit information from the segmentation network for more robust anomaly segmentation, we propose the use of pixel-wise logit variance, which tends to be small for anomalies as network outputs even logits without confidence. Additionally iterative background highlighting is proposed to robustly detect anomalous objects on the background, which is implemented by feeding the logits back into the linear classifier of the network. We achieved state-of-the-art performance among anomaly segmentation approaches without extra requirements, reaching relative average precision improvements of 21.7% on the Fishyscapes Lost&Found and 17.4% on the Fishyscapes Static compared to the state-of-the-art method. The code of this work is available at our Github repository (https://github.com/hagg30/LogitVar).

This paper considers the problem of audio source separation, where the goal is to isolate a target audio signal (say Alice's speech) from a mixture of multiple interfering signals (e.g., when many people are talking). This problem has gained renewed interest mainly due to the significant growth in voice-controlled devices, including robots in homes, offices, and other public facilities. Although a rich body of work exists on the core topic of source separation, we find that rotational motion of the microphones (e.g., a swiveling robot-head) offers complementary gains. We show that rotating the microphone array to the optimal orientation can produce desirable “delay aliasing” between two interferers, causing the two interferers to appear as one. In general, a mixture of K signals becomes a mixture of (K - 1) signals, a mathematically concrete gain. We show that the gain translates well to practice, provided two rotation-related challenges can be mitigated. This paper is focused on mitigating these challenges and demonstrating the end-to-end performance on a fully functional prototype. We believe that our Rotational Source Separation (RoSS) module could be plugged into actual robot heads or into other devices (like Amazon Show) that are also capable of rotation.

Multi-sensor fusion-based localization technology has achieved high accuracy in autonomous systems. How to improve the robustness is the main challenge at present. The most commonly used LiDAR and camera are weather-sensitive, while the FMCW radar has strong adaptability but suffers from noise and ghost effects. In this paper, we propose a heterogeneous localization method of Radar on LiDAR Map (RoLM), which can eliminate the accumulated error of radar odometry in real-time to achieve higher localization accuracy without dependence on loop closures. We embed the two sensor modalities into a density map and calculate the spatial vector similarity with offset to seek the corresponding place index in the candidates and calculate the rotation and translation. We use the ICP to pursue perfect matching on the LiDAR submap based on the coarse alignment. Extensive experiments on Mulran Radar Dataset, Oxford Radar RobotCar Dataset, and our data verify the feasibility and effectiveness of our approach.

Sensor simulation has emerged as a promising and powerful technique to find solutions to many real-world robotic tasks like localization and pose tracking. However, commonly used simulators have high hardware requirements and are therefore used mostly on high-end computers. In this paper, we present an approach to simulate range sensors directly on embedded hardware of mobile robots that use triangle meshes as environment map. This library, called Rmagine, allows a robot to simulate sensor data for arbitrary range sensors directly on board via ray tracing. Since robots typically only have limited computational resources, Rmagine aims at being flexible and lightweight, while scaling well even to large environment maps. It runs on several platforms like Laptops or embedded computing boards like NVIDIA Jetson by putting an unified API over the specific proprietary libraries provided by the hardware manufacturers. This work is designed to support the future development of robotic applications depending on simulation of range data that could previously not be computed in reasonable time on mobile systems.

In many real-world robotic scenarios, we cannot assume exact knowledge about a robot's state due to unmodeled dynamics or noisy sensors. Planning in belief space addresses this problem by tightly coupling perception and planning modules to obtain trajectories that take into account the environment's stochasticity. However, existing works are often limited to tasks such as the classic reach-avoid problem and do not provide risk awareness. We propose a risk-aware planning strategy in belief space that minimizes the risk of violating a given specification and enables a robot to actively gather information about its state. We use Risk Signal Temporal Logic (RiSTL) as a specification language in belief space to express complex spatio-temporal missions including predicates over Gaussian beliefs. We synthesize trajectories for challenging scenarios that cannot be expressed through classical reach-avoid properties and show that risk-aware objectives improve the uncertainty reduction in a robot's belief.

We introduce and investigate the recharging rendezvous problem for a collaborative team of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs), in which UAVs with limited battery capacity and UGVS persistently monitor an area. The UGVs also act as mobile recharging stations for the UAVs. In contrast to prior work on such problems, we consider the challenge of dealing with stochastic energy consumption in a risk-aware fashion. Specifically, we consider a bi-criteria optimization problem of minimizing the time taken by the UAVs on recharging detours while ensuring that the probability that no UAV runs out of charge is greater than a user-defined risk tolerance. This problem (termed Risk-aware Recharging Rendezvous Problem (RRRP)) is a combinatorial problem with a matching constraint — to ensure UAVs are assigned to the limited UGV recharging slots, and a knapsack constraint — to capture the risk tolerance. We propose a novel bicriteria approximation algorithm to solve RRRP and demonstrate its effectiveness in the context of a persistent monitoring mission compared to baseline methods.