IROS 2022 Papers

The purpose of this research is to contribute to social communication between humans and robots in scenes that have been considered difficult due to the limited facial expression capabilities of robots. In order to provide more detailed facial expressions, we designed a novel wire-driven 3D eyebrow using a soft material with a bending structure. We then demonstrated the mechanical properties of the eyebrow design and developed a prototype that could be implemented on the humanoid robot iCub to verify its operation. Lastly, we confirmed that the new design enables the production of more slight changes in facial expressions by allowing the eyebrows to change their shape at more delicate angles and continuity than the LED eyebrows of the conventional iCub.

As a traditional movement assist equipment for people with lower-limb dysfunction, the wheelchair can support and carry users to perform a long-distance movement indoor and outdoor, however, prolonged inactivity can lead to muscle atrophy and deteriorate motion functions. As a promising solution, the lower limb exoskeleton provides people the ability of standing and walking to avoid these problems. However, the exoskeleton has inevitable shortcomings in long-distance movement and balance, which do not exist in a wheelchair. To integrate the advantages of both devices, in this paper, we proposed a wheelchair-exoskeleton hybrid robot (WeHR) that can not only provide users long-time support and long-distance movement but also provide walking training and keep self-balance. Moreover, motion transitions such as sit-to-stand and stand-to-sit can also be implemented by the newly proposed device without help from caregivers. We have developed the prototype to implement the above functions. In this paper, we emphasize the strategy of motion transition including two trajectory planning methods for the Sit-To-Stand (STS) process as well as the mechanism design to implement it. Furthermore, the preliminary experiments of motion transition and walking test are also conducted and the results prove that our device can support users sitting, standing, and walking and the motion transition.

Evaluation of grasps on deformable 3\mathrm{D}3\mathrm{D} objects is a little-studied problem, even if the applicability of rigid object grasp quality measures for deformable ones is an open question. A central issue with most quality measures is their dependence on contact points, which for deformable objects depend on the deformations. This paper proposes a grasp quality measure for deformable objects that uses information about object deformation to calculate the grasp quality. Grasps are evaluated by simulating the deformations during grasping and predicting the contacts between the gripper and the grasped object. The contact information is then used as input for a new grasp quality metric to quantify the grasp quality. The approach is benchmarked against two classical rigid-body quality metrics on over 600 grasps in the Isaac gym simulation and over 50 real-world grasps. Experimental results show an average improvement of 18% in the grasp success rate for deformable objects compared to the classical rigid-body quality metrics. Furthermore, the proposed approach is approximately fifteen times faster to calculate than the shake task, which, to date, is one of the most reliable approaches to quantify a grasp on a deformable object.

As a hard high-altitude work, power transmission line inspection increasingly demands robots to conduct in place of the human being. A variety of robots have been developed to this end, with basic locomotion and inspection implemented on the lines. However, most current line inspection robots (LIRs) are only mobile platforms with complex structures and large weights, lacking sufficiently dexterous locomotion on lines, especially for obstacle overcoming and line transition. Also with sensors fixed on the platform, the inspection range is largely limited. For higher mobility and a larger inspection range, a novel biped robot that can roll and climb on a power transmission line for inspection, called Climbot-L, is proposed in this paper. While the rolling mode has the advantage of high locomotion efficiency, the biped climbing mode makes it possible to easily overcome obstacles on the line, transition to adjacent cables, and have multi-view detection. In this paper, the design of this novel robot is first introduced, the working principle of the wheel-gripper modules is then analyzed, and obstacle overcoming gaits are stated. The effectiveness and high maneuverability of the presented robot are verified by a series of experiments.

Goal-based navigation in public places is critical for independent mobility and for breaking barriers that exist for blind or visually impaired (BVI) people in a sight-centric society. Through this work we present a proof-of-concept system that autonomously leverages goal-based navigation assistance and perception to identify socially preferred seats and safely guide its user towards them in unknown indoor environments. The robotic system includes a camera, an IMU, vibrational motors, and a white cane, powered via a backpack-mounted laptop. The system combines techniques from computer vision, robotics, and motion planning with insights from psychology to perform 1) SLAM and object localization, 2) goal disambiguation and scoring, and 3) path planning and guidance. We introduce a novel 2-motor haptic feedback system on the cane's grip for navigation assistance. Through a pilot user study we show that the system is successful in classifying and providing haptic navigation guidance to socially preferred seats, while optimizing for users' convenience, privacy, and intimacy in addition to increasing their confidence in independent navigation. The implications are encouraging as this technology, with careful design guided by the BVI community, can be adopted and further developed to be used with medical devices enabling the BVI population to better independently engage in socially dynamic situations like seat choice.

While collaborative robotic arms offer significant safety benefits, safety of the overall manipulator system cannot be guaranteed unless equally strict safety requirements are satisfied by the accompanying end-effector. Current robot grippers are not made in a way that fulfills such a requirement, resulting in collaborative robots needing to operate in a protected environment. This paper presents a novel permanent magnet actuator inside of a conventional industrial electric gripper which results in an end-effector that has an unmatched force range of 1-2N to 43N and exhibits interesting characteristics suited to the requirements of a safe gripper such as torque holding without power, variable stiffness and force sensing.

Quadruped robots are usually equipped with ad-ditional arms for manipulation, negatively impacting price and weight. On the other hand, the requirements of legged locomotion mean that the legs of such robots often possess the needed torque and precision to perform manipulation. In this paper, we present a novel design for a small-scale quadruped robot equipped with two leg-mounted manipulators inspired by crustacean chelipeds and knuckle-walker forelimbs. By making use of the actuators already present in the legs, we can achieve manipulation using only 3 additional motors per limb. The design enables the use of small and inexpensive actuators relative to the leg motors, further reducing cost and weight. The moment of inertia impact on the leg is small thanks to an integrated cable/pulley system. As we show in a suite of tele-operation experiments, the robot is capable of performing single- and dual-limb manipulation, as well as transitioning between manipulation modes. The proposed design performs similarly to an additional arm while weighing and costing 5 times less per manipulator and enabling the completion of tasks requiring 2 manipulators.

Visual odometry is an important part of the perception module of autonomous robots. Recent advances in deep learning approaches have given rise to hybrid visual odometry approaches that combine both deep networks and traditional pose estimation methods. One limitation of deep learning approaches is the availability of ground truth data needed to train the neural networks. For example, it is extremely difficult, if not impossible, to obtain a ground truth dense depth map of the environment to be used for stereo visual odometry. Even if unsupervised training of networks has been investigated, supervised training remains more reliable and robust. In this paper, we propose a new hybrid dense stereo visual odometry approach in which a dense depth map is obtained with a network that is supervised using ground truth poses that can be more easily obtained than ground truth depths maps. The depth map obtained from the neural network is used to warp the current image into the reference frame and the optimal pose is obtained by minimizing a cost function that encodes the similarity between the warped image and the reference image. The experimental results show that the proposed approach, not only improves state-of-the-art depth maps estimation networks on some of the standard benchmark datasets, but also outperforms the state-of-the-art visual odometry methods.

Human-robot collaboration (HRC) has been considered as a promising paradigm towards futuristic human-centric smart manufacturing, to meet the thriving needs of mass personalization. In this context, existing robotic systems normally adopt a single-granularity semantic segmentation scheme for environment perception, which lacks the flexibility to be implemented to various HRC situations. To fill the gap, this study proposes a multi-granularity scene segmentation network. Inspired by some recent network designs, we construct an encoder network with two ConvNext-T backbones for RGB and depth respectively, and an decoder network consisting of multi-scale supervision and multi-granularity segmentation branches. The proposed model is demonstrated in a human-robot collaborative battery disassembly scenario and further evaluated in comparison with state-of-the-art RGB-D semantic segmentation methods on the NYU-Depth V2 dataset.

Minimally invasive surgeries (MIS) or natural orifice transluminal endoscopic surgeries (NOTES) such as the transurethral resection of bladder tumor (TURBT) require the surgical robot to be miniaturized to perform surgical procedures in confined spaces. However, the surgical robot's tiny size poses problems in its fabrication and shape sensing. In this paper, a miniature continuum surgical robot is proposed with a unique laminated structure which can be fabricated through a 2D lamination process and converted into 3D through folding. This multi-material laminated structure also facilitates the integration of tiny piezoelectric transducers on the robot's surface as beacons to generate ultrasonic waves for shape detection. A novel beacon total focusing method (b-TFM) algorithm is developed to process the received ultrasonic data and create a high-quality ultrasonic image from which the shape of the continuum robot can be extracted. The proposed robot and the ultrasonic shape detection method are validated through simulations and experiments. The error in the open-loop trajectory control is less than 4 mm without compensation, and the error in the ultrasonic shape detection is less than 1 mm. This confirms the possibility of improving the trajectory control accuracy by using the detected shape as a feedback for closed-loop control.

This article proposes a method for the correction of angular deviations caused during the fixing process of samples prepared for Atomic Force Microscopy (AFM). The correction is done using the angular control of a 6-DOF PPPS parallel platform were the sample is placed, while the AFM scan is performed by a 3-DOF serial cartesian robot with a tuning fork probe designed to perform FM-AFM. The method uses the generic x, y, and z data provided by the AFM after performing a scan on a free surface of the sample substrate. This is used to calculate the plane that closest approximates the points by solving a system of linear equations. This plane is then used to estimate the angular corrections that the 6-DOF parallel robot has to do in order to compensate the deviations. The proposed algorithm can be performed iteratively in order to refine the correction. The method also does not require any special preparation of the substrate. It only requires to have a free surface to scan. Experiments are performed using this algorithm to correct the orientation deviation of a substrate of V1 High-grade mica. The results show that the method is able to correct the angular deviation of the sample relatively to the AFM probe with an error of 0.2° after only two iterations of the algorithm.

Steerable needles are medical devices with the ability to follow curvilinear paths to reach targets while circumventing obstacles. In the deployment process, a human operator typically places the steerable needle at its start position on a tissue surface and then hands off control to the automation that steers the needle to the target. Due to uncertainty in the placement of the needle by the human operator, choosing a start position that is robust to deviations is crucial since some start positions may make it impossible for the steerable needle to safely reach the target. We introduce a method to efficiently evaluate steerable needle motion plans such that they are safe to variation in the start position. This method can be applied to many steerable needle planners and requires that the needle's orientation angle at insertion can be robotically controlled. Specifically, we introduce a method that builds a funnel around a given plan to determine a safe insertion surface corresponding to insertion points from which it is guaranteed that a collision-free motion plan to the goal can be computed. We use this technique to evaluate multiple feasible plans and select the one that maximizes the size of the safe insertion surface. We evaluate our method through simulation in a lung biopsy scenario and show that the method is able to quickly find needle plans with a large safe insertion surface.

Drones can provide a minimally-constrained adapting camera view to support robot telemanipulation. Furthermore, the drone view can be automated to reduce the burden on the operator during teleoperation. However, existing approaches do not focus on two important aspects of using a drone as an automated view provider. The first is how the drone should select from a range of quality viewpoints within the workspace (e.g., opposite sides of an object). The second is how to compensate for unavoidable drone pose uncertainty in determining the viewpoint. In this paper, we provide a nonlinear optimization method that yields effective and adaptive drone viewpoints for telemanipulation with an articulated manipulator. Our first key idea is to use sparse human-in-the-loop input to toggle between multiple automatically-generated drone viewpoints. Our second key idea is to introduce optimization objectives that maintain a view of the manipulator while considering drone uncertainty and the impact on viewpoint occlusion and environment collisions. We provide an instantiation of our drone viewpoint method within a drone-manipulator remote teleoperation system. Finally, we provide an initial validation of our method in tasks where we complete common household and industrial manipulations.

This paper proposes a LiDAR-inertial odometry that properly solves the uncertainty estimation problem, guided by the rules of designing a consistent estimator. Our system is built upon an iterated extended Kalman filter, with multiple states in an optimization window. To survive environments without distinctive geometric structures, we do not track features over time. We only extract planar primitives from the local map and use a direct point-to-plane distance metric as the measurement model. The realistic noise parameters are estimated online by modeling point distributions. We use nullspace projection to remove dependency on the feature planes, which is equivalent to transforming the pose-map measurement into relative pose constraints. To avoid reintegrating all the laser points in the local window after every state correction, we use the Schmidt Kalman update to consider the probabilistic effects of past poses while their values are left unaltered. A collection of octrees with an adaptive resolution is designed to manage measurement points and the map efficiently. The consistency and robustness of our system are verified in both simulation and real-world experiments.

Pseudospectral collocation methods have proven to be powerful tools to solve optimal control problems. While these methods generally assume the dynamics is given in the first order form xx = f(x, u, t), where xx is the state and uu is the control vector, robotic systems are typically governed by second order ODEs of the form qq = g(q, q, u, t), where qq is the configuration. To convert the second order ODE into a first order one, the usual approach is to introduce a velocity variable vv and impose its coincidence with the time derivative of q. Lobatto methods grant this constraint by construction, as their polynomials describing the trajectory for vv are the time derivatives of those for q, but the same cannot be said for the Gauss and Radau methods. This is problematic for such methods, as then they cannot guarantee that qq = g(q, q, u, t) at the collocation points. On their negative side, Lobatto methods cannot be used to solve initial value problems, as given the values of uu at the collocation points they generate an overconstrained system of equations for the states. In this paper, we propose a Legendre-Gauss collocation method that retains the advantages of the usual Lobatto, Gauss, and Radau methods, while avoiding their shortcomings. The collocation scheme we propose is applicable to solve initial value problems, preserves the consistency between the polynomials for vv and q, and ensures that qq= g(q, q, u, t) at the collocation points.

While haptics research has traditionally focused on the fingertips and hands, other locations on the body provide large areas of skin that could be utilized to relay large-area haptic sensations. Researchers have thus developed wearable devices that use distributed vibrotactile actuators and distributed pneumatic force displays, but these methods have limitations. In prior work, we presented a novel actuation technique involving stacking pneumatic pouches and evaluated the actuator output. In this work, we developed a wearable haptic device using this actuation technique and evaluated how the actuator output is perceived. We conducted a user study with 20 participants to evaluate users' perception thresholds, ability to localize, and ability to detect differences in contact area and compare their perception using the stacked pneumatic pouch actuation to traditional single-layer pouch actuation. We also used our device with stacked pneumatic actuation in a demonstration of a haptic hug that replicates the dynamics, pressure profile, and mapping to the human back, showcasing how this actuation technique can be used to create novel haptic stimuli.

Wheeled-legged robots have the potential to navigate in cluttered and irregular scenarios by altering the locomotion modes to adapt to the terrain challenges and effectively reach targeted locations in unstructured spaces. To achieve this functionality, a hybrid locomotion planner is necessary. In this work we present a search-based planner, which explores a set of motion primitives and a 2.5D traversability map extracted from the environment to generate navigation plans for the hybrid mobility robot CENTAURO. The planner explores the map from the current robot position to the goal location requested by the user, considering the most appropriate composition and tuning of locomotion primitives to build up a feasible plan, which is then executed by the robot. The available primitives are prioritized and can be easily modified, added or removed through a configuration file. Our approach was evaluated both in simulation and on the real wheeled-legged robot CENTAURO, demonstrating traversing capabilities in cluttered environments with various obstacles.

Groups of robots can be tasked with identifying a location in an environment where a feature cue is past a threshold, then disseminating this information throughout the group – such as identifying a high-enough elevation location to place a communications tower. This is a continuous-cue target search, where multi-robot search algorithms like particle swarm optimization (PSO) can improve search time through parallelization. However, many robots lack global communication in large spaces, and PSO-based algorithms often fail to consider how robots disseminate target knowledge after a single robot locates it. We present a two-stage hybrid algorithm to solve this task: (1) locating a target with a variation of PSO, and (2) moving to maximize target knowledge across the group. We conducted parameter sweep simulations of up to 32 robots in a grid-based grayscale environment. Pre-decision, we find that PSO with a variable velocity update interval improves target localization. In the post-decision phase, we show that dispersion is the fastest strategy to communicate with all other robots. Our algorithm is also competitive with a coverage sweep benchmark, while requiring significantly less inter-individual coordination.

Given an object, an environment, and a goal pose, how should a robot make contact to move it? Solving this problem requires reasoning about rigid-body dynamics, object and environment geometries, and hybrid contact mechanics. This paper proposes a hierarchical framework that solves this problem in 2D worlds, with polygonal objects and point fingers. To achieve this, we decouple the problem in three stages: 1) a high-level graph search over regions of free-space, 2) a medium-level randomized motion planner for the object motion, and 3) a low-level contact-trajectory optimization for the robot and environment contacts. In contrast to the state of the art, this approach does not rely on handcrafted primitives and can still be solved efficiently. This algorithm does not require seeding and can be applied to complex object shapes and environments. We validate this framework with extensive simulated experiments showcasing long-horizon and contact-rich interactions. We demonstrate how our algorithm can reliably solve complex planar manipulation problems in the order of seconds.