IROS 2022 论文列表

Robowflex is a software library for robot motion planning in industrial and research applications, leveraging the popular Moveit library and Robot Operating System (ROS) middleware. Robowflex provides an augmented API for crafting and manipulating motion planning queries within a single program, making motion planning with Moveit easy. Robowflex's high-level API simplifies many common use-cases while still providing low-level access to the Moveit library when needed. Robowflex is particularly useful for 1) developing new motion planners, 2) evaluating motion planners, and 3) complex problems that use motion planning as a subroutine (e.g., task and motion planning). Robowflex also provides visualization capabilities, integrations to other robotics libraries (e.g., DART and Tesseract), and is complementary to other robotics packages. With our library, the user does not need to be an expert at ROS or Moveit to set up motion planning queries, extract information from results, and directly interface with a variety of software components. We demonstrate its efficacy through several example use-cases.

Grinding materials into a fine powder is a time-consuming task in material science that is generally performed by hand, as current automated grinding machines might not be suitable for preparing small-sized samples. This study presents a robotic powder grinding system for laboratory automation in material science applications that observe the powder's state to improve the grinding outcome. We developed a soft jig consisting of off-the-shelf gel materials and 3D-printed parts, which can be used with any robot arm to perform powder grinding. The jig's physical softness allows for safe grinding without force sensing. In addition, we developed a visual feedback system that observes the powder distribution and decides where to grind and when to gather. The results showed that our system could grind 79 percent of the powder to a particle size smaller than 200 μm by using the soft jig and visual feedback. This ratio was 57% when using only the soft jig without feedback. Our system can be used immediately in laboratories to alleviate the workload of researchers.

Interestingness recognition is crucial for decision making in autonomous exploration for mobile robots. Previous methods proposed an unsupervised online learning approach that can adapt to environments and detect interesting scenes quickly, but lack the ability to adapt to human-informed interesting objects. To solve this problem, we introduce a human-interactive framework, AirInteraction, that can detect human-informed objects via few-shot online learning. To reduce the communication bandwidth, we first apply an online unsupervised learning algorithm on the unmanned vehicle for interestingness recognition and then only send the potential interesting scenes to a base-station for human inspection. The human operator is able to draw and provide bounding box annotations for particular interesting objects, which are sent back to the robot to detect similar objects via few-shot learning. Only using few human-labeled examples, the robot can learn novel interesting object categories during the mission and detect interesting scenes that contain the objects. We evaluate our method on various interesting scene recognition datasets. To the best of our knowledge, it is the first human-informed few-shot object detection framework for autonomous exploration.

In this paper, we present a motion-based robotic communication framework that enables non-verbal communication among autonomous underwater vehicles (AUVs) and human divers. We design a gestural language for AUV-to-AUV communication which can be easily understood by divers observing the conversation - unlike typical radio frequency, light, or audio-based AUV communication. To allow AUVs to visually understand a gesture from another AUV, we propose a deep network (RRCommNet) which exploits a self-attention mechanism to learn to recognize each message by extracting maximally discriminative spatio-temporal features. We train this network on diverse simulated and real-world data. Our experimental evaluations, both in simulation and in closed-water robot trials, demonstrate that the proposed RRCommNet architecture is able to decipher gesture-based messages with an average accuracy of 88-94% on simulated data and 73-83% on real data (depending on the version of the model used). Further, by performing a message transcription study with human participants, we also show that the proposed language can be understood by humans with an overall transcription accuracy of 88 %. Finally, we discuss the inference runtime of RRCommNet on embedded GPU hardware, for real-time use on board AUVs in the field.

During auscultation, patients in difficult age often feel embarrassed and uncomfortable when exposing their chests to doctors of the different gender and being touched physically by doctors. We assume that an auscultation with robot technology can address the aforementioned gender-related issue. Toward eliminating gender bias during auscultation exam, this paper proposes a robotic platform which enables to perform the automated auscultation over clothes. Our developed system is comprised of two folds: a depth image-based estimation system of the listening positions over clothes with RGB-D camera and a contact force adjustment system for minimizing the acoustic attenuation due to the clothes with a passive-actuated end-effector. Our preliminary results demonstrated the robotic platform enables to estimate the listening locations to hear the sounds of four cardiac valves over the clothes by combining the estimated skeletal structure with statistical anatomical data and acquire the maximized acoustic quality over the clothes by adjusting the contact force. The developed robotic platform has the potential to address the gender-related issues in auscultation.

Structural intervention cardiology (SIC) interventions are crucial procedures for correcting heart valves, walls, and muscle form defects. However, the possibility of embolization or perforation, as well as the lack of transparent vision and autonomous surgical equipment, make it difficult for the clinician. This paper proposes a robot-assisted tendon-driven catheter and machine learning-based path planner to overcome these challenges. Firstly, an analytical inverse kinematic model is constructed to convert the tip location in the Cartesian space to the tendons' displacement. Then inverse reinforcement learning algorithm is employed to calculate the optimal path to avoid possible collisions between the catheter tip and the atrial wall. Moreover, a closed-loop feedback controller is adopted to improve positioning accuracy in a direct distal position measurement manner. Simulation and experiments are designed and conducted to demonstrate the feasibility and performance of the proposed system.

Hardware acceleration can revolutionize robotics, enabling new applications by speeding up robot response times while remaining power-efficient. However, the diversity of acceleration options makes it difficult for roboticists to easily deploy accelerated systems without expertise in each specific hardware platform. In this work, we address this challenge with RobotCore, an architecture to integrate hardware acceleration in the widely-used ROS 2 robotics software framework. This architecture is target-agnostic (supports edge, workstation, data center, or cloud targets) and accelerator-agnostic (supports both FPGAs and GPUs). It builds on top of the common ROS 2 build system and tools and is easily portable across different research and commercial solutions through a new firmware layer. We also leverage the Linux Tracing Toolkit next generation (LTTng) to enable low-overhead real-time tracing and benchmarking of accelerated ROS 2 systems. To demonstrate the acceleration enabled by this architecture, we use it to deploy a ROS 2 perception computational graph on a CPU and FPGA. We also employ our integrated tracing and benchmarking to analyze bottlenecks, uncovering insights that guide us to improve FPGA communication efficiency. In particular, we design an intra-FPGA ROS 2 node communication queue template and use it in conjunction with FPGA-accelerated nodes to achieve a 24.42% speedup over a CPU.

Estimating the microbial infestation profile of an area is essential for an effective cleaning process. However, current methods used to inspect the microbial infestation within a spatial region are manual and laborious. For large regions that require automated cleaning, conventional methods of microbial examination are not practical. We propose a novel robot-aided microbial density estimation and mapping framework using an in-house developed biosensor payload onboard a mobile robot. The biosensor estimates the degree of microbial infestation in Relative Light Units (RLU) using the natural bio-luminescence reaction. The global distribution of microbial infestation is approximated through the Radial Basis Function (RBF) and Nearest Neighbour (NN) interpolation algorithms. The proposed method is implemented on an in-house developed mobile robot called Beluga. The framework's validation and usefulness are demonstrated quantitatively through real-world experiment trials.

This paper reports on insights by robotics researchers that participated in a 5-day robot-assisted nuclear disaster response field exercise conducted by Kerntechnische Hilfdienst GmbH (KHG) in Karlsruhe, Germany. The German nuclear industry established KHG to provide a robot-assisted emergency response capability for nuclear accidents. We present a systematic description of the equipment used; the robot operators' training program; the field exercise and robot tasks; and the protocols followed during the exercise. Additionally, we provide insights and suggestions for advancing disaster response robotics based on these observations. Specifically, the main degradation in performance comes from the cognitive and attentional demands on the operator. Furthermore, robotic platforms and modules should aim to be robust and reliable in addition to their ease of use. Last, as emergency response stakeholders are often skeptical about using autonomous systems, we suggest adopting a variable autonomy paradigm to integrate autonomous robotic capabilities with the human-in-the-loop gradually. This middle ground between teleoperation and autonomy can increase end-user acceptance while directly alleviating some of the operator's robot control burden and maintaining the resilience of the human-in-the-loop.

Drilling a hole on a curved surface with a desired angle is prone to failure when done manually, due to the difficulties in drill alignment and also inherent instabilities of the task, potentially causing injury and fatigue to the workers. On the other hand, it can be impractical to fully automate such a task in real manufacturing environments because the parts arriving at an assembly line can have various complex shapes where drill point locations are not easily accessible, making automated path planning difficult. In this work, an adaptive admittance controller with 6 degrees of freedom is developed and deployed on a KUKA LBR iiwa 7 cobot such that the operator is able to manipulate a drill mounted on the robot with one hand comfortably and open holes on a curved surface with haptic guidance of the cobot and visual guidance provided through an AR interface. Real-time adaptation of the admittance damping provides more transparency when driving the robot in free space while ensuring stability during drilling. After the user brings the drill sufficiently close to the drill target and roughly aligns to the desired drilling angle, the haptic guidance module fine tunes the alignment first and then constrains the user movement to the drilling axis only, after which the operator simply pushes the drill into the workpiece with minimal effort. Two sets of experiments were conducted to investigate the potential benefits of the haptic guidance module quantitatively (Experiment I) and also the practical value of the proposed pHRI system for real manufacturing settings based on the subjective opinion of the participants (Experiment II). The results of Experiment I, conducted with 3 naive participants, show that the haptic guidance improves task completion time by 26% while decreasing human effort by 16% and muscle activation levels by 27% compared to no haptic guidance condition. The results of Experiment II, conducted with 3 experienced industrial workers, show that the propos... Show More

In hybrid industrial environments, workers' comfort and positive perception of safety are essential requirements for successful acceptance and usage of collaborative robots. This paper proposes a novel human-robot interaction framework in which the robot behaviour is adapted online according to the operator's cognitive workload and stress. The method exploits the generation of B-spline trajectories in the joint space and formulation of a multi-objective optimisation problem to online adjust the total execution time and smoothness of the robot trajectories. The former ensures human efficiency and productivity of the workplace, while the latter contributes to safeguarding the user's comfort and cognitive ergonomics. The performance of the proposed framework was evaluated in a typical industrial task. Results demonstrated its capability to enhance the productivity of the human-robot dyad while mitigating the cognitive workload induced in the worker.

Hierarchical algorithms have often been used to plan and execute complicated robotic sequential manipulation tasks, where an abstract planner searches for a skill sequence in an abstract space, and each skill generates actual motions on the basis of the planned skill sequences. To generate executable plans, the abstract planner should know the pre-/postconditions of each skill and appropriately choose skills so that the generated plan satisfies their pre-/postconditions. For such hierarchical planning, this paper presents a novel method for robot skill learning that learns not only a control policy but also the learned skill's pre-/postconditions to complete a given task. Our method combines an optimal control method and an active learning approach called level set estimation (LSE) to effectively collect training data for learning control policies and pre-/postconditions. Although there exists a LSE-based policy learning algorithm that identifies preconditions, its performance is limited to cases where the dimension of the search space for pre-/postconditions is low. The main contribution of this paper is the proposal of a new learning method that can handle tasks having a high-dimensional search space for pre-/postconditions. We demonstrate our proposed method in two robotic tasks. The results show that our method can more effectively learn a control policy and its pre-/postconditions compared with the existing LSE-based method.

Learning robotic tasks in the real world is still highly challenging and effective practical solutions remain to be found. Traditional methods used in this area are imitation learning and reinforcement learning, but they both have limitations when applied to real robots. Combining reinforcement learning with pre-collected demonstrations is a promising approach that can help in learning control policies to solve robotic tasks. In this paper, we propose an algorithm that uses novel techniques to leverage offline expert data using offline and online training to obtain faster convergence and improved performance. The proposed algorithm (AWET) weights the critic losses with a novel agent advantage weight to improve over the expert data. In addition, AWET makes use of an automatic early termination technique to stop and discard policy rollouts that are not similar to expert trajectories-to prevent drifting far from the expert data. In an ablation study, AWET showed improved and promising performance when compared to state-of-the-art baselines on four standard robotic tasks.

Neural network (NN)-based methods have emerged as an attractive approach for robot motion planning due to strong learning capabilities of NN models and their inherently high parallelism. Despite the current development in this direction, the efficient capture and processing of important sequential and spatial information, in a direct and simultaneous way, is still relatively under-explored. To overcome the challenge and unlock the potentials of neural networks for motion planning tasks, in this paper, we propose STP-Net, an end-to-end learning framework that can fully extract and leverage important spatio-temporal information to form an efficient neural motion planner. By interpreting the movement of the robot as a video clip, robot motion planning is transformed to a video prediction task that can be performed by STP-Net in both spatially and temporally efficient ways. Empirical evaluations across different seen and unseen environments show that, with nearly 100% accuracy (aka, success rate), STP-Net demonstrates very promising performance with respect to both planning speed and path cost. Compared with existing NN-based motion planners, STP-Net achieves at least 5×, 2.6× and 1.8× faster speed with lower path cost on 2D Random Forest, 2D Maze and 3D Random Forest environments, respectively. Furthermore, STP-Net can quickly and simultaneously compute multiple near-optimal paths in multi-robot motion planning tasks.

Robotic painting tasks in the real world are often made complicated by the highly complex and stochastic nature of the dynamics that underlie, e.g., physical contact between the painting tool and a canvas, color blendings between painting mediums, and many more. Simulation-based inverse graphics algorithms, for example, can not be directly transferred to the real-world due in large to the considerable gap in the viable range of painting strokes the robot can accurately generate onto the physical canvas. In this paper, we aim at minimizing this gap by appealing to a data-driven skill learning approach. The core idea lies in allowing the robot to learn continuous stroke-level skills that jointly encodes action trajectories and painted outcomes from an extensive collection of human demonstrations. We demonstrate the efficacy of our method through extensive real-world experiments using a 4-dof torque-controllable manipulator with a digital canvas(iPad).

Learning from Demonstration (LfD) is a popular method of reproducing and generalizing robot skills from human-provided demonstrations. In this paper, we propose a novel optimization-based LfD method that encodes demon-strations as elastic maps. An elastic map is a graph of nodes connected through a mesh of springs. We build a skill model by fitting an elastic map to the set of demonstrations. The formulated optimization problem in our approach includes three objectives with natural and physical interpretations. The main term rewards the mean squared error in the Cartesian coordinate. The second term penalizes the non-equidistant distribution of points resulting in the optimum total length of the trajectory. The third term rewards smoothness while pe-nalizing nonlinearity. These quadratic objectives form a convex problem that can be solved efficiently with local optimizers. We examine nine methods for constructing and weighting the elastic maps and study their performance in robotic tasks. We also evaluate the proposed method in several simulated and real-world experiments using a UR5e manipulator arm, and compare it to other LfD approaches to demonstrate its benefits and flexibility across a variety of metrics.

Musical dancing is an ubiquitous phenomenon in the human society. Providing robots the ability to dance has the potential to make the human robot co-existence more acceptable in our society. Hence, dancing robots have generated a considerable research interest in the recent years. In this paper, we present a novel formalization of robot dancing as planning and control of optimally timed actions based on beat timings and additional features extracted from the music. We showcase the use of this formulation in three different variations: with input of human expert choreography, imitation of a predefined choreography, and automated generation of a novel choreography. Our method has been validated on four different musical pieces, both in simulation and on a real robot, using the upper-body humanoid robot RH5 Manus.

In order to transform a robot into an intelligent machine it needs to be enabled to react to unforeseen events (most importantly collisions) during task execution and have a plan on how to continue the task afterwards. This requires a flexible operational framework that allows to define adaptive reactions and interactions with the motion generation and task planning stage. Within this work we first reason about the choices the robot has for reactions to unforeseen events such as collisions with respect to safety of humans in the workspace, the robot itself and the environment as well as the successful task execution. We further present a flexible reflex engine together with a concept of integration into the motion generation and control work flow. The reflex engine and it's reflex maneuvers are a combination of state machines and decision trees that take into account the state of the robot and the world. It is capable of choosing safe reactions and can differentiate between different levels of contact severity and according reaction sets. Several reflex maneuvers are evaluated towards safety performance criteria in real robot experiments using an ISO/TS 15066 conform measurement device. Some of the tested reflexes are furthermore integrated into an implementation of the proposed approach for a simple real world example task where the robot needs to pickup a container and dispose it's content into a bin.

To overcome the limitations of the so-called Industry 4.0 focusing on mass production and full automation, a novel paradigm was recently introduced, namely Industry 5.0, which aims at an increased collaboration between humans and machines, and particularly robots, instead of replacing the former with the latter. This challenge requires novel interactive intelligent robots able to perform complex tasks easily and efficiently and to collaborate on the fly with humans whenever required, be it for training or working. In this work, the Robot Companion, a novel demonstrator of this paradigm, is introduced. It combines robotics, Artificial Intelligence, software engineering and embedded systems technologies, and targets industrial assembly tasks. First tests show that this robot can efficiently assemble a representative gear system autonomously or in collaboration with human operators.