IROS 2022 论文列表

Percutaneous coronary intervention (PCI) is a frequently used surgical treatment for cardiovascular disease, one of the leading cause of death in the world. In traditional PCI, a doctor navigates a thin guidewire in a patient's vessel toward a target location by looking into live X-ray angiogram images of the patient. Recently, researchers are using reinforcement learning to automate this guidewire navigation process without attaching any sensor to the guidewire tip. These researchers use a real vessel phantom to train their behavior policy using reinforcement learning. Training a reinforcement learning algorithm on a real setup can give a good guidewire control on that setup, but it is under question whether the trained algorithm can be applied to other vessel structures. We can make various vessel phantoms and train the algorithm on the setups, but it can be really time and money consuming. In this paper, we devise a method for sim-to-real transfer of a guidewire navigation trained by reinforcement learning using only images. We pretrain our behavior policy using data collected by running an expert algorithm in the virtual environment. Then, we train the behavior policy by deep deterministic policy gradient (DDPG) in a virtual environment. With behavior cloning, our method learns to successfully navigate a guidewire in much shorter time than training DDPG from scratch without behavior cloning. After done with the training, we transfer the behavior policy trained in the virtual environment to the guidewire navigation in a real vessel phantom. Our trained behavior policy navigates the guidewire to destinations successfully in all test episodes and navigates faster than the expert algorithm. Experiment video is available at: https: //youtu.be/HCEbIhZsXqw

Although the manipulating of the unmanned aerial manipulator (UAM) has been widely studied, vision-based UAM approaching, which is crucial to the subsequent manipulating, generally lacks effective design. The key to the visual UAM approaching lies in object tracking, while current UAM tracking typically relies on costly model-based methods. Besides, UAM approaching often confronts more severe object scale variation issues, which makes it inappro-priate to directly employ state-of-the-art model-free Siamese-based methods from the object tracking field. To address the above problems, this work proposes a novel Siamese network with pairwise scale-channel attention (SiamSA) for vision-based UAM approaching. Specifically, SiamSA consists of a pairwise scale-channel attention network (PSAN) and a scale-aware anchor proposal network (SA-APN). PSAN acquires valuable scale information for feature processing, while SA-APN mainly attaches scale awareness to anchor proposing. Moreover, a new tracking benchmark for UAM approaching, namely UAMT100, is recorded with 35K frames on a flying UAM platform for evaluation. Exhaustive experiments on the benchmarks and real-world tests validate the efficiency and practicality of SiamSA with a promising speed. Both the code and UAMT100 benchmark are now available at https://github.com/vision4robotics/SiamSA.

Social relations within a group are one of the factors that build the social context. Less attention has been paid to social relations within a group for human-robot interaction design. In this study, we designed different types of service behaviors for social relations in a mixed age group and conducted an experiment to investigate the effect of service behavior types (serving the elderly first versus serving the young first versus serving without priority) on a user's evaluation of a service robot. We only found that service evaluation and appropriateness of the robot were rated less positively under the condition of serving the young first than under other conditions. In addition, we found that the participants evaluated the robot serving the young first as impolite. The effect of the robot's behaviors on service evaluation and appropriateness was mediated by politeness. The current study provides an initial basis for human-robot interaction design for social norms in group-robot interaction.

Autonomous vehicles are designed to elevate the efficiency of assigned tasks and ensure the safety of the environment in which they operate. This paper presents a research study focused on shared autonomy using a multi-layer fuzzy logic framework to build a relationship between an autonomous self-reconfigurable robot and a human user by switching control to the teleoperator to assist the robot when it faces challenging scenarios while keeping a good performance and maintaining a safe environment. A novel multi-layer fuzzy logic decision process with shared autonomy for a safety framework is proposed. It evaluates safety based on the robot's multi-sensor inputs, the teleoperator's attention level, and the configuration state of the self-reconfigurable robot and switches the operation mode, robot speed gain, and configuration state for performance and safety without compromises. The experimental outcome successfully demonstrates the self-reconfigurable robot's capability to navigate safely using shared autonomy in real-world pavement scenarios using the proposed algorithm during autonomous navigation.

Variable stiffness catheters typically rely on thermally induced stiffness transitions with a transition temperature above body temperature. This imposes considerable safety limitations for medical applications. In this work, we present a variable stiffness catheter using a hybrid control strategy capable of actively heating and actively cooling the catheter material. The proposed catheter is made of a single biocompatible shape memory polymer, which significantly increases its manufacturability and scalability compared to existing designs. Potentially increased safety is obtained by ensuring a lower-risk compliant state at body temperature while maintaining higher stiffness ranges in actively controlled states. Additionally, the combined use of variable stiffness and magnetic actuation increases the dexterity and steerability of the device compared to existing robotic tools.

We propose a framework for co-optimizing the shape and motion of rigid robotic effectors for planar tasks. While planning object and robot-object contact trajectories is extensively studied, designing an effector that can execute the planned trajectories receives less attention. As such, our framework synthesizes an object trajectory and object-effector contact trajectory into an effector trajectory and shape that (a) does not penetrate the object, (b) makes contact with the object as specified, and (c) optimizes a user-specified objective. This simplifies manipulator control by encoding task-specific contact information in the effector's geometry. Our key insight is posing these requirements as constraints in the effector's reference frame, preventing the need for explicit parameterization of the effector shape. This prevents artificial restrictions on the shape design space. Importantly, it also facilitates posing the shape and motion design problem as a tractable nonlinear program. Our method is particularly useful for problems where the shape of the effector surface must be precisely chosen to achieve a task. We apply our method to several such problems, including jar-opening and picking up objects in constrained spaces. We evaluate the performance and computational cost of our method, and provide a physical experiment of a robotic arm picking up a screwdriver from a table with a designed tool.

Accurate shape estimation of concentric tube robots (CTRs) using mathematical models remains a challenge, reinforcing the need to develop techniques for accurate and real-time shape sensing of CTRs. In this paper, we develop a fusion algorithm that predicts the robot's shape by combining a mathematical model of the CTR with a measurement of the Cartesian coordinates of the robot's tip using an electro-magnetic sensor. We experimentally validated our method in static and dynamic scenarios with and without external loading. Results demonstrated that the fusion algorithm improves the error of model-based shape prediction by an average of 44.3%, corresponding to 2.43% of the robot's arc length. Furthermore, we demonstrate that our method can be used in real-time to simultaneously track the robot's tip position and predict its shape.

In recent years, robots have become vital to achieving manufacturing competitiveness. Especially in industrial environments, a strong level of interaction is reached when humans and robots form a dynamic system that works together towards achieving a common goal or accomplishing a task. However, the human-robot collaboration can be cognitively demanding, potentially contributing to cognitive fatigue. Therefore, the consideration of cognitive fatigue becomes particularly important to ensure the efficiency and safety in the overall human-robot collaboration. Additionally, sex is an inevitable human factor that needs further investigation for machine learning model development given the perceptual and physiological differences between the sexes in responding to fatigue. As such, this study explored sex differences and labeling strategies in the development of machine learning models for cognitive fatigue detection. Sixteen participants, balanced by sex, recruited to perform a surface finishing task with a UR10 collaborative robot under fatigued and non-fatigued states. Fatigue perception and heart rate activity data collected throughout to create a dataset for cognitive fatigue detection. Equitable machine learning models developed based on perception (survey responses) and condition (fatigue manipulation). The labeling approach had a significant impact on the accuracy and F1-score, where perception-based labels lead to lower accuracy and F1-score for females likely due to sex differences in reporting of fatigue. Additionally, we observed a relationship between heart rate, algorithm type, and labeling approach, where heart rate was the most significant predictor for the two labeling approaches and for all the algorithms utilized. Understanding the implications of label type, algorithm type, and sex on the design of fatigue detection algorithms is essential to designing equitable fatigue-adaptive human-robot collaborations across the sexes. Show More

Reconfigurable mobile robots are well suited for inspection tasks in legacy nuclear facilities where access is restricted and the environment is often cluttered. A reconfig-urable snake robot, MIRRAX, has previously been developed to investigate such facilities. The joints used for the robot's reconfiguration introduce additional constraints on the robot's control, such as balance, on top of the existing actuator and collision constraints. This paper presents a set-point controller for MIRRAX using vector-field inequalities to enforce hard constraints on the robot's balance, actuator limits, and collision avoidance in a single quadratic programming formulation. The controller has been evaluated in simulation and early experiments in some scenarios. The results show that the controller generates feasible control inputs that enable the robot to retain its balance while moving with less oscillation and operating within the actuation and collision constraints.

In this paper, we report on a field study in which we employed two service robots in a bakery store as a sales promotion. Previous studies have explored public applications of service robots public such as shopping malls. However, more evidence is needed that service robots can contribute to sales in real stores. Moreover, the behaviors of customers and service robots in the context of sales promotions have not been examined well. Hence, the types of robot behavior that can be considered effective and the customers' responses to these robots remain unclear. To address these issues, we installed two tele-operated service robots in a bakery store for nearly 2 weeks, one at the entrance as a greeter and the other one inside the store to recommend products. The results show a dramatic increase in sales during the days when the robots were applied. Furthermore, we annotated the video recordings of both the robots' and customers' behavior. We found that although the robot placed at the entrance successfully attracted the interest of the passersby, no apparent increase in the number of customers visiting the store was observed. However, we confirmed that the recommendations of the robot operating inside the store did have a positive impact. We discuss our findings in detail and provide both theoretical and practical recommendations for future research and applications.

The awareness for preserving privacy in in-home monitoring robots is increasing. Although several studies have proposed privacy-preserved in-home monitoring robot systems for adults, only a limited amount of attention has been paid attention to research on privacy-preserved in-home monitoring of babies. Like previous studies, thermal infrared image-based methods could ensure a privacy-preserved monitoring of babies, yet when existing detection methods were applied to thermal images to detect babies and adults, we discovered a frequent occurrence of misdetection due to the presence of thermal residual heat marks. In this research, we propose a sequential thermal image-based detection that conjugated the characteristics of thermal residual heat marks. The proposed detection reduced misdetection caused by thermal residual heat marks by 98.7% when compared to RetinaNet. In addition, we open-source our collected thermal image-based baby and adult dataset via: https://github.com/donkeymouse/ThermalAdultandBaby.

We devise a 3D scene graph representation, contact graph+ (cg+), for efficient sequential manipulation planning. Augmented with predicate-like attributes, this contact graph-based representation abstracts scene layouts with succinct geometric information and valid robot-scene interactions. Goal configurations, naturally specified on contact graphs, can be produced by a genetic algorithm with a stochastic optimization method. A task plan is then initialized by computing the Graph Editing Distance (GED) between the initial contact graph and the goal configuration, which generates graph edit operations corresponding to possible robot actions. We finalize the task plan by imposing constraints to regulate the temporal feasibility of graph edit operations, ensuring valid task and motion correspondences. In a series of simulated and real experiments, robots successfully complete complex sequential object rearrangement tasks that are difficult to specify using conventional planning language like Planning Domain Definition Language (PDDL), demonstrating high potential of planning sequential manipulation tasks on cg+.

Task and Motion Planning has made great progress in solving hard sequential manipulation problems. However, a gap between such planning formulations and control methods for reactive execution remains. In this paper we pro-pose a model predictive control approach dedicated to robustly execute a single sequence of constraints, which corresponds to a discrete decision sequence of a TAMP plan. We decompose the overall control problem into three sub-problems (solving for sequential waypoints, their timing, and a short receding horizon path) that each is a non-linear program solved online in each MPC cycle. The resulting control strategy can account for long-term interdependencies of constraints and reactively plan for a timing-optimal transition through all constraints. We additionally propose phase backtracking when running constraints of the current phase cannot be fulfilled, leading to a fluent re-initiation behavior that is robust to perturbations and interferences by an experimenter.

This paper presents a model-based approach to reconstructing the large deformations of slender flexible beams through strain-gauge deflection sensors. Using the principal axes decomposition of structural compliance, a closed-form kinetostatics model can be obtained to characterize the non-linear force-deformation behavior of the flexible beams under-going large-scale deflection. Owing to analytical derivation of the system Jacobian, the efficient Newton-Raphson method is employed to determine the equilibrium configuration of the flexible beams, as well as the corresponding reaction force. To verify the correctness and effectiveness of the proposed method, an experimental apparatus is built up, on which a variety of experiments are conducted. The results show that for a 300 mm long beam, the tip position can be predicted with an accuracy of 1.27 mm, 4.42 mm, and 1.17°, respectively, for the x, y directions and rotation. Accordingly, the estimation errors for the planar forces and torque are 0.075 N (3.33%), 0.155 N (14.23%), and 0.027 Nm (26.84%), respectively.

In this paper, we propose a preliminary definition and analysis of the novel concept of sensor observability index. The goal is to analyse and evaluate the performance of distributed directional or axial-based sensors to observe specific axes in task space as a function of joint configuration in serial robot manipulators. For example, joint torque sensors are often used in serial robot manipulators and assumed to be perfectly capable of estimating end effector forces, but certain joint configurations may cause one or more task-space axes to be unobservable as a result of how the joint torque sensors are aligned. The proposed sensor observability provides a method to analyse the quality of the current robot configuration to observe the task space. Parallels are drawn between sensor observability and the traditional kinematic Jacobian for the particular case of joint torque sensors in serial robot manipulators. Although similar information can be retrieved from kinematic analysis of the Jacobian transpose in serial manipulators, sensor observability is shown to be more generalizable in terms of analysing non-joint-mounted sensors and other sensor types. In addition, null-space analysis of the Jacobian transpose is susceptible to false observability singularities. Simulations and experiments using the robot Baxter demonstrate the importance of maintaining proper sensor observability in physical interactions.

The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. This complicates tactile tasks like shape reconstruction and exploration that require information about contact geometry. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information. We validate our approach by showing a match between the model-generated unsheared images with their counterparts from vertically tapping onto the object. The model-generated unsheared images give faithful reconstruction of contact-geometry otherwise masked by shear, along with robust estimation of object pose then used for sliding exploration and full reconstruction of several planar shapes. We show that our semi-supervised approach achieves comparable performance to its fully supervised counterpart across all validation tasks with an order of magnitude less supervision. The semi-supervised method is thus more computational and labeled sample-efficient. We expect it will have broad applicability to wide range of complex tactile exploration and manipulation tasks performed via a shear-sensitive sense of touch.

Augmented reality (AR) has the potential to improve the immersion and efficiency of computer-assisted orthopaedic surgery (CAOS) by allowing surgeons to maintain focus on the operating site rather than external displays in the operating theatre. Successful deployment of AR to CAOS requires a calibration that can accurately calculate the spatial relationship between real and holographic objects. Several studies attempt this calibration through manual alignment or with additional fiducial markers in the surgical scene. We propose a calibration system that offers a direct method for the calibration of AR head-mounted displays (HMDs) with CAOS systems, by using infrared-reflective marker-arrays widely used in CAOS. In our fast, user-agnostic setup, a HoloLens 2 detected the pose of marker arrays using infrared response and time-of-flight depth obtained through sensors onboard the HMD. Registration with a commercially available CAOS system was achieved when an IR marker-array was visible to both devices. Study tests found relative-tracking mean errors of 2.03 mm and 1.12° when calculating the relative pose between two static marker-arrays at short ranges. When using the calibration result to provide in-situ holographic guidance for a simulated wire- insertion task, a pre-clinical test reported mean errors of 2.07 mm and 1.54° when compared to a pre-planned trajectory.

Loop closure detection has the potential to correct the drift of trajectories and build a global consistent map in LiDAR SLAM, however it remains a challenging problem in outdoor environment due to the sparsity of 3D point clouds data, large-scale scenes and moving objects. Inspired by the way humans perceive the environment through recognizing objects and identifying their relations, this paper presents a novel descriptor that contains semantic and topological information for loop closure detection. Unlike most existing methods that extract features from the raw point clouds or use all semantic objects, we directly discard point clouds representing pedestrians and vehicles after semantic segmentation. Then, we propose a semantic topological graph representation from the remaining point clouds and convert this graph into a descriptor. Additionally, we propose a two-stage algorithm for matching descriptors to efficiently determine the loop. Our method has been extensively evaluated using the KITTI dataset and outperforms state-of-the-art methods, especially in the challenging situations such as viewpoint changes and dynamic scenes.

Partial observation of indoor scenes (single-viewed RGB-D) carries insufficient spatial information for complex tasks such as autonomous navigation and virtual reality, thus many learning-based methods are proposed to realize semantic scene completion (SSC) from single-viewed input. However, most of them only extract scene-level features of input to generate output, which might lose details. In this paper, a new method that fully utilizes both instance-level and scene-level features is proposed. Firstly, an object detection module is pre-trained to localize indoor objects. Secondly, coarse completion result is obtained from scene-level feature using an encoder-decoder structure. Finally, based on the pre-trained bounding boxes, coarse completion result is refined using a geometric refinement module. Our network's performance is evaluated on both real and synthetic datasets. The results demonstrate that our network is able to reconstruct indoor scenes with more geometric details, get clearer boundaries between instances and outperform most existing SSC methods both intuitively and quantitatively.