ICRA 2023 Papers

Fast motion planning is of great significance, espe-cially when a timely mission is desired. However, the complexity of motion planning can grow drastically with the increase of environment details and mission complexity. This challenge can be further exacerbated if the tasks are coupled with the desired locations in the environment. To address these issues, this work aims at fast motion planning problems with temporal logical specifications. In particular, we develop a hierarchical decoupling framework that consists of three layers: the high-level task planner, the decoupling layer, and the low-level motion planner. The decoupling layer is designed to bridge the high and low layers by providing necessary information exchange. Such a framework enables the decoupling of the task planner and path planner, so that they can run independently, which significantly reduces the search space and enables fast planing in continuous or high-dimension discrete workspaces. In addition, the implicit constraint during task-level planning is taken into account, so that the low-level path planning is guaranteed to satisfy the mission requirements. Numerical simulations demonstrate at least one order of magnitude speed up in terms of computational time over existing methods.

Atrial fibrillation (AF) is mostly treated via robotic catheter-based cardiac ablation procedures. Over the last few decades, cables or tendon mechanisms are at the core of available cardiac catheters. Despite advances, the use of cables often results in considerable force loss, nonlinear hysteresis, and control challenges. Most catheters are not equipped with force sensing, which increases the risk of the ablation process and decreases their efficacy in clinical settings. In addition, current catheters have a poor user interface and therefore the ablation process requires skilled or trained surgeons to steer the complex motion of the catheter tip within the heart chambers. To improve the cardiac ablation procedure, a new robotic catheter that has the ability to extend its working space without moving its flexible body and a real-time force sensor for safe operation is highly desired. In this work, a new handheld and soft robotic catheter for AF ablation is introduced. The new device consists of several improved components such as a soft manipulator for navigation and bending motion, an ergonomic handheld controller, and a soft force sensor for monitoring tool-tissue contact. The design, modeling, and fabrication of the device are presented and followed by experimental characterizations and ex-vivo validation.

The on-ground validation of orbital manipulators is a challenging task because the robot is designed for a gravity-free operational environment, but it is validated under the effect of gravity. As a consequence, joint torque limits can be easily reached in certain configurations when gravity is actively compensated by the joints. Hence, the workspace for on-ground testing is restricted. In this paper, an optimal strategy is proposed for achieving gravity compensation of an orbital manipulator arm on ground. The strategy minimizes the joint torques acting on the manipulator by solving an optimization problem and it computes the necessary forces to be tracked by an external carrier. Hence, full gravity compensation is achieved for the orbital manipulator. Experimental results validate the effectiveness of the method on the DLR CAESAR space robot, which uses a cable suspended system as external carrier to track the desired gravity compensation force, resulting from the proposed method.

Hand-eye calibration is the problem of estimating the spatial transformation between a reference frame, usually the base of a robot arm or its gripper, and the reference frame of one or multiple cameras. Generally, this calibration is solved as a non-linear optimization problem, what instead is rarely done is to exploit the underlying graph structure of the problem itself. Actually, the problem of hand-eye calibration can be seen as an instance of the Simultaneous Localization and Mapping (SLAM) problem. Inspired by this fact, in this work we present a pose-graph approach to the hand-eye calibration problem that extends a recent state-of-the-art solution in two different ways: i) by formulating the solution to eye-on-base setups with one camera; ii) by covering multi-camera robotic setups. The proposed approach has been validated in simulation against standard hand-eye calibration methods. Moreover, a real application is shown. In both scenarios, the proposed approach overcomes all alternative methods. We release with this paper an open-source implementation of our graph-based optimization framework for multi-camera setups.

As a kind of ground mobile robot, deformable robots have many advantages, such as solid terrain adaptability, lightweight, and portability. Among these robots, the radial skeleton robot has better stability and controllability. However, because the friction of foot and ground is hard to be predicted, the accuracy of its gait generation algorithms that have been studied is very low. Furthermore, there is currently no closed-loop control scheme for this kind of robot. We designed a 12-legged radial skeleton robot with high extension ratio legs, proposed a high-precision gait generation algorithm for any multi-legged radial skeleton robot, and first proposed a closed-loop control scheme for this kind of robot. A dynamic model considering contact friction is established. And the robot has the advantages of omnidirectional motion, high-precision trajectory tracking, and motion robustness. By conducting prototype experiments, it is verified that our method achieves the highest accuracy when tracking trajectory and holds robustness in the unknown environment.

Human-robot interaction (HRI) demands efficient time performance along the tasks. However, some interaction approaches may extend the time to complete such tasks. Thus, the time performance in HRI must be enhanced. This work presents an effective way to enhance the time performance in HRI tasks with a mixed reality (MR) method based on a gaze-speech system. In this paper, we design an MR world for pick-and-place tasks. The hardware system includes an MR headset, the Baxter robot, a table, and six cubes. In addition, the holographic MR scenario offers two modes of interaction: gesture mode (GM) and gaze-speech mode (GSM). The input actions during the GM and GSM methods are based on the pinch gesture and gaze with speech commands, respectively. The proposed GSM approach can improve the time performance in pick-and-place scenarios. The GSM system is 21.33 % faster than the traditional system, GM. Also, we evaluated the target- to-target time performance against a reference based on Fitts' law. Our findings show a promising method for time reduction in HRI tasks through MR environments.

The spatial distribution of sensed objects strongly influences the behavior of mobile robots. Yet, as robots evolve in complexity to operate in increasingly rich environments, it becomes much more difficult to specify the underlying relations between sensed object spatial distributions and robot behaviors. We aim to address this challenge by leveraging system trace data to automatically infer relations that help to better characterize these spatial associations. In particular, we introduce SpRinG, a framework for the unsupervised inference of system specifications from traces that characterize the spatial relationships under which a robot operates. Our method builds on a parameterizable notion of reachability to encode relationships of spatial neighborship, which are used to instantiate a language of patterns. These patterns provide the structure to infer, from system traces, the connection between such relationships and robot behaviors. We show that SpRinG can automatically infer spatial relations over two distinct domains: autonomous vehicles in traffic and a surgical robot. Our results demonstrate the power and expressiveness of SpRinG, in its ability to learn existing specifications as machine-checkable first-order logic, uncover previously unstated specifications that are rich and insightful, and reveal contextual differences between executions.

This article presents a novel framework called Simultaneous Registration and Machining (SRAM), a generalized method to improve workpiece registration using real-time acquired data in robotic contouring applications. The method allows for online corrections to the toolpath, while a live covariance estimate is simultaneously leveraged to adaptively tune the force controller aggressively when uncertainty is high, but conservatively otherwise to minimize chatter and instability. The SRAM framework is validated in simulation and shown to significantly reduce the path corrections required from the force controller, while correctly predicting optimal controller tuning adaptations. The SRAM method is proposed to improve force control stability, increase peripheral accuracy, smooth surface finish, and reduce cycle times in contouring applications.

Robotic applications involving people often require advanced perception systems to better understand complex real-world scenarios. To address this challenge, photo-realistic and physics simulators are gaining popularity as a means of generating accurate data labeling and designing scenarios for evaluating generalization capabilities, e.g., lighting changes, camera movements or different weather conditions. We develop a photo-realistic framework built on Unreal Engine and AirSim to generate easily scenarios with pedestrians and mobile robots. The framework is capable to generate random and customized trajectories for each person and provides up to 50 ready-to-use people models along with an API for their metadata retrieval. We demonstrate the usefulness of the proposed framework with a use case of multi-target tracking, a popular problem in real pedestrian scenarios. The notable feature variability in the obtained perception data is presented and evaluated.

Haptic feedback is an important component of creating an immersive mixed reality experience. Traditionally, haptic forces are rendered in response to the user's interactions with the virtual environment. In this work, we explore the idea of rendering haptic forces in a proactive manner, with the explicit intention to influence the user's behavior through compelling haptic forces. To this end, we present a framework for active haptic guidance in mixed reality, using one or more robotic haptic proxies to influence user behavior and deliver a safer and more immersive virtual experience. We provide details on common challenges that need to be overcome when implementing active haptic guidance, and discuss example applications that show how active haptic guidance can be used to influence the user's behavior. Finally, we apply active haptic guidance to a virtual reality navigation problem, and conduct a user study that demonstrates how active haptic guidance creates a safer and more immersive experience for users.

With the increasing demand for new healthcare solutions and technologies, such as those resulting from the COVID-19 crisis, and the growing elderly population, exoskeletons for teleoperation are a promising solution for many future medical applications. In this context, we propose two force- sensitive upper-limb exoskeletons for teleoperation, that are characterized by: i) torque-controlled robotic actuators, ii) rigid-body model compensations, and iii) a lightweight design achieved through the use of Bowden cable transmissions and remotely placed actuators. Specifically, we present a semi-active upper-limb exoskeleton for which we demonstrate human- device interaction control and bilateral teleoperation with force- feedback, evaluated via simulation, in the lab and over the Internet. We also introduce a design for a future fully-active upper-limb exoskeleton with two contact force/torque sensors, for a dual-arm device, which features a novel 3-degrees-of- freedom exoskeleton shoulder design and a contact wrench mitigation controller, as demonstrated through simulation. With this work, we propose the essential technical steps towards a novel teleoperation system for elderly care.

Following biology's lead, soft robotics has emerged as a perfect candidate for actuation within complex environments. While soft actuation has been developed intensively over the last few decades, soft sensing has so far slowed to catch up. A largely unresearched area is the change of the soft material properties through prestress to achieve a degree of mechanical sensitivity tunability within soft sensors. Here, a new 3D force sensor which employs novel hydraulic filament artificial muscles capable of in-situ sensitivity tunability is introduced. Using a neural network (NN) model, the new soft 3D sensor can precisely detect external forces based on the change of the hydraulic pressures with error of \sim 1.0, \sim 1.3\sim 1.0, \sim 1.3, and \sim 0.94\sim 0.94 % in the \text{x, y}\text{x, y}, and z-axis directions, respectively. The sensor is also able to sense large force ranges, comparable to other similar sensors available in the literature. The sensor is then integrated into a soft robotic surgical arm for monitoring the tool-tissue interaction during an ablation process.

Current dynamical models of Cosserat rods often use the finite element method limited by computational efficiency or the finite difference method in a Cartesian framework with a compromise to accuracy. We employ the finite difference method in a geometric framework to develop solutions that are both computationally efficient and accurate. A numerical study is conducted on various backward-differentiation discretization and Runge-Kutta-Munthe-Kaas integration schemes, focusing on their accuracy and computational efficiency. Case studies are conducted on a single-degree-of-freedom joint actuated Cosserat rod to mitigate additional sources of undesired error from the numerical analysis, e.g. multi-body interactions, moving base dynamics, etc. The proposed geometric integrators are demonstrated to improve solution accuracy compared to the published finite difference models. The presented solution is parameterization-free and also computationally efficient with the potential for use in real-time applications, e.g., model-based control of soft manipulators.

Upper limb rehabilitation robots are mainly used as a physical therapy method to passively or actively train the affected side. However, they are rarely implemented in accordance with the occupational therapy theory, which is dedicated to improving the sensorimotor coordination of hemiplegic patients by considering both healthy and affected limbs. To realize the occupational therapy concept in robot-assisted upper limb rehabilitation, we propose a new human-robot collaboration framework for hemiplegic patients that integrates healthy/affected limbs and robot. The strategy aims at achieving patient-specific movement capabilities and improving the participation of the affected limb during rehabilitation. To accomplish this task, we have addressed two essential issues: accurate motion estimation of the healthy limb and the rehabilitation trajectory learning technique. The posture estimation is achieved by introducing the calibration model to reduce static and time dependent errors during the measurement. We also introduce a force term to the conventional imitation learning method to improve the adaptability in integrating the affected side in cooperation with the robot. Various experiments have been conducted to validate the feasibility and effectiveness of our proposed dual-arm collaboration strategy.

In this paper, we propose a distributed algorithm to control a team of cooperating robots aiming to protect a target from a set of intruders. Specifically, we model the strategy of the defending team by means of an online optimization problem inspired by the emerging distributed aggregative framework. In particular, each defending robot determines its own position depending on (i) the relative position between an associated intruder and the target, (ii) its contribution to the barycenter of the team, and (iii) collisions to avoid with its teammates. We highlight that each agent is only aware of local, noisy measurements about the location of the associated intruder and the target. Thus, in each robot, our algorithm needs to (i) locally reconstruct global unavailable quantities and (ii) predict its current objective functions starting from the local measurements. The effectiveness of the proposed methodology is corroborated by simulations and experiments on a team of cooperating quadrotors.

Teleoperated driving (ToD) is increasingly considered as a fallback solution for autonomous driving. Up to now, ToD requires a highly reliable mobile network capable of transmitting multiple video streams with low latency. Recently, significant advancements have been made in vehicular sensors and perception algorithms, which have huge implications for the challenges faced in ToD. We envisage that a real-time digital twin that tracks the remote vehicle's environment will play a crucial role in reducing the required communication band-width and providing a more convenient teleoperator interface, ultimately enhancing safety of ToD in crowded environments. Furthermore, it would allow various degrees of cooperation between automated driving functionalities and human teleoperators. In this paper, the concept of digital twin for ToD is outlined and a proof of concept is implemented using a real-world vehicle simulator and a teleoperator hardware setup. A significant reduction in required bandwidth is reported by transmitting less video data and reconstructing the scene from the digital twin.

Many older adults prefer to stay in their own homes and age-in-place. However, physical and cognitive limitations in independently completing activities of daily living (ADLs) requires older adults to receive assistive support, often necessitating transitioning to care centers. In this paper, we present the development of a novel deep learning human activity recognition and classification architecture capable of autonomously identifying ADLs in home environments to enable long-term deployment of socially assistive robots to aid older adults. Our deep learning architecture is the first to use multimodal inputs to create an embedding vector approach for classifying and monitoring multiple ADLs. It uses spatial mid-fusion to combine geometric, motion and semantic features of users, environments, and objects to classify and track ADLs. We leverage transfer learning to extract generic features using the early layers of deep networks trained on large datasets to apply our architecture to various ADLs. The embedding vector enables identification of unseen ADLs and determines intra-class variance for monitoring user ADL performance. Our proposed unique architecture can be used by socially assistive robots to promote reablement in the home via autonomously supporting the assistance of varying ADLs. Extensive experiments show improved classification accuracy compared to unimodal/dual-modal models and the ADL embedding space also incorporates the ability to distinctly identify and track seen and unseen ADLs.

We propose a robust framework for planar pose graph optimization contaminated by loop closure outliers. Our framework rejects outliers by first decoupling the robust PGO problem wrapped by a Truncated Least Squares kernel into two subproblems. Then, the framework introduces a linear angle representation to rewrite the first subproblem that is originally formulated in rotation matrices. The framework is configured with the Graduated Non-Convexity (GNC) algorithm to solve the two non-convex subproblems in succession without initial guesses. Thanks to the linearity property of the angle representation, our framework requires only a linear solver to optimally solve the optimization problems encountered in GNC. We extensively validate the proposed framework, named DEGNC- LAF (DEcoupled Graduated Non-Convexity with Linear Angle Formulation) in planar PGO benchmarks. It turns out that it runs significantly (sometimes up to over 30 times) faster than the standard and general-purpose GNC while resulting in high-quality estimates.

Ultrasound scanning is an efficient imaging modality preferred for quick medical procedures. However, due to the lack of skilled sonographers, researchers have developed many Robotic Ultrasound System (RUS) prototypes for various procedures. Most of these systems have a human-in-the-loop and require an expert to point the robot to the region of the subject to be scanned. Only a few systems try to incorporate some knowledge from the exterior shape of the subject for ultrasound scanning. Accurate 3D surface reconstruction of a patient's exterior can enable an RUS to perceive subjects more like a clinician would. It can help localize the subject for the robot while eliminating input from an expert. Ultrasound scanning trajectories can be better planned if the RUS first detects critical regions on the surface of the subject and corresponding curvatures. We use an RGB-D sensor to acquire point clouds representing the 3D surface of the subject, which in the present work is for a lower-torso leg phantom. A consolidated pipeline for creating an optimized 3D surface reconstruction of a subject is presented and is used to autonomously identify a region of interest for scanning femoral vessels with an ultrasound probe. To make our system more robust to inter-subject variations in shape and size, we incorporate a trajectory optimization module of the RUS-mounted RGB-D sensor. To this end, we introduce a comprehensive evaluation score to quantify the quality of point cloud reconstructions. The resulting improvements in 3D surface scanning and reconstruction enable near-automation in generating ultrasound scanning trajectories for femoral vessels. Our pipeline produces ultrasound images with an average ZNCC score of 0.86 and our 3D point cloud reconstructions are accurate up to le-5 m from a ground-truth high-resolution CT scan.