IROS 2021 Papers

This paper reports the design and evaluation of a novel piezo based actuator for needle drive in autonomous Deep Anterior Lamellar Keratoplasty (piezo-DALK). The actuator weighs less than 8g and is 20mm x 20mm x 10.5mm in size, making it ideal for eye-mounted applications. Mean open loop positional deviation was 1.17 ± 3.15um, and system repeatability and accuracy were 17.16um and 18.33um, respectively. Stall force was found to vary linearly with the cooling cycle and the actuator achieved a maximum drive force of 3.98N. When simulating the DALK procedure in synthetic corneal tissue, the piezo-DALK achieved a penetration depth of 643.56um which was equivalent to 92.1% of the total corneal thickness. This correlated closely with our desired depth of 90% ± 5% and took 2.5 hours to achieve. This work represents the first eye mountable actuator capable of "Big Bubble" needle drive for autonomous DALK procedures.

We report the development of novel fault detection and isolation (FDI) methods for model-based fault detection (MB-FD) and quotient-space fault isolation (QS-FI). This FDI approach performs MB-FD and QS-FI of single or multiple con-current faults in plants and actuators simultaneously, without a priori knowledge of fault form, type, or dynamics. To detect faults, MB-FD characterizes deviation from nominal behavior using the plant velocity and plant and actuator parameters estimated by nullspace-based adaptive identification. To isolate (i.e. identify) faults, the QS-FI algorithm compares the estimated parameters to a nominal parameter class in progressively decreasing-dimensional quotient spaces of the parameter space. A preliminary simulation study of these proposed FDI methods applied to a three degree-of-freedom uninhabited underwater vehicle plant model shows their ability to detect as well as isolate faults for the cases of both single and multiple simultaneous faults and suggests the generalizability of the MB-FD and QS-FI approaches to any well-defined second-order plant and actuator model whose parameters enter linearly: a broad class of systems which includes aerial vehicles, marine vehicles, spacecraft, and robot arms.

In this paper, a new robust model-based super-twisting algorithm is proposed as a control solution for parallel kinematic manipulators (PKMs). The conventional super-twisting algorithm for robot manipulators has the structure of a computed-torque control which can be sensitive to measurement noise. This issue may deteriorate the dynamic performance of the manipulator and reduce its robustness towards changes in the operating conditions. The proposed approach, relying on the desired trajectory, is more computationally efficient and more robust. It includes a feedforward dynamic compensator, the super-twisting feedback control, and a feedback stabilizing term. As a validation, real-time experiments have been conducted on a 5-DOF redundantly actuated PKM. Several scenarios have been tested including nominal case and the robustness towards speed variations. The relevance of the proposed control solution is proved through the improvement of the tracking performance at different dynamic operating conditions.

Recent offshore drilling activities have dramatically bloomed oil and gas production. Due to extreme weather, such as hurricanes and tsunamis, offshore oil platforms may need to be constantly monitored in case of unexpected dangers. Using robots to monitor and prevent these dangerous situations is a cost-effective and safer solution compared to any human involvement. However, one major drawback for robotic intervention is the ability of the robot to operate water tight ship doors designed for marina facilities. Current studies have been focused on robots manipulating normal doors in people's daily life. However their presented methodology may not be suitable for opening heavy watertight ship doors since they have a special locking structure which needs much larger torque to open. This study explores the possibility of opening heavy watertight ship doors with an ordinary mobile robot under remote control. Given the watertight ship door’s unique structure, a door opening tool with a pawl mechanism is designed for a mobile robot, which consists of a robotic arm and a wheeled platform, to complete the task. Experimental results have shown that the mobile robot can operate the tool to open the door and transition through the door.

The purpose of this paper is to present a novel curved Gaussian Mixture Model (CGMM) and to study the application of it in motion skill encoding. Primarily, Gaussian mixture model (GMM) has been widely applied on many occasions when a probability density function is needed to approximate a complex probability distribution. However, GMM cannot efficiently approach highly non-linear distributions. Thus, the proposed novel CGMM, as a weighted mixture of curved Gaussian models (CGM), is structured with non-linear transfers, which reshapes the flat GMM into a geo-metrically curved one. As a consequence, CGMM has more freedoms and flexibilities than the flat GMM so a CGMM requires fewer number of components in fitting highly non-linear motion trajectories. Moreover, we derive a dedicated iterative parameter estimation algorithm for the CGMM based on maximum likelihood estimation (MLE) theory. To evaluate the performance of the CGMM and its parameter estimation algorithm, a series of quantitative experiments are carried out. We first test the model performance in the data fitting task with the generated synthetic data. Then a motion skill encoding test is carried out on a human motion trajectory dataset built by a Virtual Reality (VR) based motion tracking system. The empirical results support that CGMM outperforms state-of-the-arts in the model performance test. Meanwhile, CGMM has a significant improvement in encoding high dimensional non-linear trajectory data compared to the GMM in motion skill encoding test with its dedicated parameter estimation algorithm.

A novel 6-DOF parallel manipulator with two spherical-universal-revolute limbs is proposed in this work. Compared with general 6-DOF parallel manipulators of six kinematic limbs, this new manipulator actuated by spherical motion generators has only two limbs, which brings kinematic advantages such as small footprint and large workspace. The inverse position problem of the manipulator is solved by an analytical approach, upon which velocity equations are formulated. Kinematic performance including workspace and also manipulability are calculated to show the advantages of the new design.

Execution of automatically generated programs for accurate robotic machining requires the generated trajectories to be not only accurate with respect to the work piece, but also that the trajectories are continuous differentiable (C1) while avoiding unnecessary large curvatures leading to large accelerations that could compromise machining quality or speed. A widely used work piece representation is 3D triangle meshes as they can be easily generated in any CAD representations and from surface scans, and they are also very suitable for robotics applications. However, they lack the C1 property across the triangle edges.In this paper, a new method for generating C1 surfaces based on 3D triangle meshes is presented. It will be shown by an example that the method is as good as existing methods with respect to the accuracy of the generated surface, and that the problem with large curvatures is much smaller than for existing methods. Moreover, the difficult input specification of derivatives at the vertices is avoided with this method.

Understanding human navigational intent is essential for robots to be able to interact with and navigate around humans safely and naturally. Current methods typically perform inference through only one mode of perception such as human motion trajectory, and a single theoretical framework such as a learning-based or classical approach. In contrast, this paper studies prediction of human navigational intent using multimodal perception within a hybrid framework. Our framework consists of two modules: a) a learning-based prediction module to predict a human’s future goal position, and b) a classical control theory-inspired reconstruction module to reconstruct a possible future trajectory or a set of possible future positions using the predicted future goal position. For the prediction module, we propose an end-to-end LSTM-CNN hybrid neural network for predicting a human’s future position in the real world, given human motion, human body pose and head orientation. This visual information from an egocentric perspective is used to make predictions of a human’s future position in world space, essential for robotic navigation algorithms and planning. In the reconstruction module, we present two control theoretic methods to reconstruct possible future trajectories of human: trajectory generation for differentially flat system and reachability analysis. We evaluate the performance of our framework on a newly collected dataset called SFU-Store-Nav. Experimental results reveal that our method outperforms various baselines especially when a relatively small amount of data is available.

Space robots have played an essential role in space junk removal. Compared with traditional model-based methods, model-free reinforcement learning methods are promising in tackling space capture missions, which is challenging due to the dynamic singular problem and measuring errors of dynamics parameters. Nevertheless, current research mostly focus on the single-target environment. In this paper, we propose a multi-target trajectory planning strategy of a 6-DoF free-floating space robot optimized by the Proximal Policy Optimization (PPO) algorithm. Furthermore, we adopt some augmentation techniques to improve the PPO algorithm on precision and stability of reaching multiple targets. In particular, we introduce an Action Ensembles Based on Poisson Distribution (AEP) method, which facilitates the policy to efficiently approximate the optimal policy. Our method can be easily extended to realize the task that the end-effector tracks a specific trajectory. We evaluate our approach on four tasks: circle trajectory tracking, external disturbances at joints, different masses of the base, and even single joint failure, without any further fine-tuning. The results suggest that the planning strategy has comparably high adaptability and anti-inference capacity. Qualitative results (videos) are available at [36].

Design optimization can lead to the development of robotic end-effectors with optimal grasping and dexterous, in-hand manipulation capabilities. In particular, the finger link dimensions have been identified as one of the primary design parameters that affects the performance of a robotic gripper. The ability of a gripper to manipulate objects is mainly attributed to the interaction between a set of coordinated fingers. This coordination is primarily affected by the inter-finger distance. This paper presents a framework for finding an appropriate distance between the finger bases of a two-fingered robotic gripper so as to increase the dexterous manipulation workspace for a range of object sizes. To do that, a parallel multi-start search algorithm is employed to solve a multiparametric optimization problem. The results demonstrate that different distances lead to completely different workspace shapes and that the ratio defined by the area of the optimized workspace (nominator) and the union of all workspaces (denominator) is always significantly less than 1. This means that the area of the union of all workspaces is always larger than the area of the "optimized" workspace. Based on these results a multi-modal robotic gripper with movable finger bases was developed. The proposed gripper can vary the distance between the finger bases online and it offers an increased dexterous manipulation workspace without sacrificing grasping performance.

In object-based Simultaneous Localization and Mapping (SLAM), 6D object poses offer a compact representation of landmark geometry useful for downstream planning and manipulation tasks. However, measurement ambiguity then arises as objects may possess complete or partial object shape symmetries (e.g., due to occlusion), making it difficult or impossible to generate a single consistent object pose estimate. One idea is to generate multiple pose candidates to counteract measurement ambiguity. In this paper, we develop a novel approach that enables an object-based SLAM system to reason about multiple pose hypotheses for an object, and synthesize this locally ambiguous information into a globally consistent robot and landmark pose estimation formulation. In particular, we (1) present a learned pose estimation network that provides multiple hypotheses about the 6D pose of an object; (2) by treating the output of our network as components of a mixture model, we incorporate pose predictions into a SLAM system, which, over successive observations, recovers a globally consistent set of robot and object (landmark) pose estimates. We evaluate our approach on the popular YCB-Video Dataset and a simulated video featuring YCB objects. Experiments demonstrate that our approach is effective in improving the robustness of object-based SLAM in the face of object pose ambiguity. 1

We present a novel robot end-effector for gripping and haptic exploration. Tactile sensing through suction flow monitoring is achieved with a new suction cup design that contains multiple chambers for air flow. Each chamber connects with its own remote pressure transducer, which enables both absolute and differential pressure measures between chambers. By changing the overall vacuum applied to this smart suction cup, it can perform different functions such as gentle haptic exploration (low pressure) and monitoring breaks in the seal during strong astrictive gripping (high pressure). Haptic exploration of surfaces through sliding and palpation can guide the selection of suction grasp locations and help to identify the local surface geometry. During suction gripping, a trained LSTM network can localize breaks in the suction seal between four quadrants with up to 97% accuracy and detects breaks in the suction seal early enough to avoid total grasp failure.

Tactile sensing can improve end-effector control and grasp quality, especially for free-flying robots where target approach and alignment present particular challenges. However, many current tactile sensing technologies are not suitable for the harsh environment of space. We present a tactile sensor that measures normal and biaxial shear strains in the pads of a gripper using a single optical fiber with Bragg grating (FBG) sensors. Compared to conventional wired solutions, the encapsulated optical fibers are immune to electromagnetic interference — critical in the harsh environment of space. Sampling is possible at over 1 kHz to detect dynamic events. We mount sensor pads on a custom two-fingered gripper with independent control of the distal and proximal phalanges, allowing for grip readjustment based on sensing data. Calibrated sensor data for forces match those from a commercial multiaxial load cell with an average 96.2% RMS for all taxels. We demonstrate the gripper on tasks motivated by the Astrobee free-flying robots in the International Space Station (ISS): gripping corners, detecting misaligned grasps, and improving load sharing over the contact areas in pinch grasps.

For conventional aerial manipulators, the robotic arm is rigidly attached to the quadrotor. Consequently, the maneuver of the quadrotor will affect the motion of the robotic arm when it is used for tasks such as inspection. In this paper, we propose a novel aerial manipulator with a self-locking gimbal system which can switch between motion coupled and decoupled mode. Furthermore, a dynamic gravity compensation mechanism is designed, where the location of the battery and the number of teeth are optimized to minimize the weight imbalance of the robotic arm during its motions. To the best of the authors’ knowledge, this is the first aerial manipulator with a motion-decoupled mechanism. Experimental results demonstrate that the proposed manipulator design can significantly improve the performance of the manipulator for general inspection tasks.

Multirotors can be effectively applied to various tasks, such as transportation, investigation, exploration, and lifesaving, depending on the type of payload. However, due to the nature of multirotors, the payload loaded on the multirotor is limited in its position and weight, which presents a major disadvantage when the multirotor is used in various fields. In this paper, we propose a novel method that greatly improves the restrictions on payload position and weight using a morphing quadrotor system. Our method can estimate the drone’s weight, center of gravity position, and inertia tensor in real-time, which change depending on payload, and determine the optimal morphology for efficient and stable flight. An adaptive control method that can reflect the change in flight dynamics by payload and morphing is also presented. Experiments were conducted to confirm that the proposed morphing quadrotor improves the stability and efficiency in various situations of transporting payloads compared with the conventional quadrotor systems.

Collaborative human-robot field operations rely on timely decision-making and coordination, which can be challenging for heterogeneous teams operating in large-scale deployments. In this work, we present the design of an immersive, mixed reality (MR) interface to support sense-making and situational awareness based on the data collection capabilities of both human and robotic team members. Our solution integrates state-of-the-art methods in environment mapping and MR so that users may gain rapid insights regarding the working environment, the current and previous locations of human and robot team members, and the environment data such team members have collected. We describe the implementation of our system, share lessons learned in collaborating with emergency responders throughout our design process, and offer a vision for the use of immersive displays for human-robot field team deployments in large-scale outdoor environments.

This paper shows analytical and experimental evidence of using the vibration dynamics of a compliant whisker for accurate terrain classification during steady state motion of a mobile robot. A Hall effect sensor was used to measure whisker vibrations due to perturbations from the ground. Analytical results predict that the whisker vibrations will have one dominant frequency at the vertical perturbation frequency of the mobile robot and one with distinct frequency components. These frequency components may come from bifurcation of vibration frequency due to nonlinear interaction dynamics at steady state. Experimental results also exhibit distinct dominant frequency components unique to the speed of the robot and the terrain roughness. This nonlinear dynamic feature is used in a deep multi-layer perceptron neural network to classify terrains. We achieved 85.6% prediction success rate for seven flat terrain surfaces with different textures.

Due to changes in model dynamics or unexpected disturbances, an autonomous robotic system may experience unforeseen challenges during real-world operations which may affect its safety and intended behavior: in particular actuator and system failures and external disturbances are among the most common causes of degraded mode of operation. To deal with this problem, in this work, we present a meta-learning-based approach to improve the trajectory tracking performance of an unmanned aerial vehicle (UAV) under actuator faults and disturbances which have not been previously experienced. Our approach leverages meta-learning to train a model that is easily adaptable at runtime to make accurate predictions about the system’s future state. A runtime monitoring and validation technique is proposed to decide when the system needs to adapt its model by considering a data pruning procedure for efficient learning. Finally, the reference trajectory is adapted based on future predictions by borrowing feedback control logic to make the system track the original and desired path without needing to access the system’s controller. The proposed framework is applied and validated in both simulations and experiments on a faulty UAV navigation case study demonstrating a drastic increase in tracking performance.

We focus on multiple experts performing a task in a Markov decision process (MDP) environment. A probabilistic assignment of trajectories to clusters and a mathematical framework which leverages the utility function are employed to jointly estimate the discount factor and reward. We treat the number of clusters as a hyperparameter which can be "freely" selected by the problem designer. In this work, we specifically treat the cluster of trajectories as a latent variable in the adapted maximum entropy inverse reinforcement learning (IRL) formulation; the introduction of this latent variable adds to the complexity of the IRL problem. To manage such complexity, we optimize a marginal log-likelihood function via Expectation Maximization. To test our approach, we have utilized behavioral data generated from three MDP environments. Experimental works show that our approach is promising towards the estimation of discount factors in IRL for non-interacting multiple experts.