IROS 2022 Papers

Area coverage is demanded from the robots utilized in application domains such as floor cleaning. Even though many advanced coverage algorithms have been developed, the area coverage performance is limited due to the inaccessibility of narrow spaces caused by physical constraints. Reconfigurable robots have been introduced to overcome this limitation where reconfigurability could help in assessing narrow spaces. Nevertheless, the state-of-the-art reconfigurable robots are not capable of changing the morphology size and shape as a single entity. Therefore, this paper proposes a novel design of a reconfigurable robot with a size-adaptive coverage strategy. The reconfiguration mechanism is designed in such a way that the robot can independently expand or shrink its size along the principal planar axes, where the behavior allows the change of size and shape. The coverage strategy is based on boustrophedon motion and the A* algorithm modified for accessing narrow areas using the size adaptability. The design of the robot is detailed in the paper, including electro-mechanical aspects, design considerations, and the coverage path planning method. Experiments have been conducted using a prototype of the proposed design to analyze and evaluate the characteristics and the performance of the robot. The results show that the proposed robot design can improve the productivity of a floor cleaning robot in terms of area coverage and coverage time.

Existing robotic lower-limb prostheses use autonomous control to address cyclic, locomotive tasks, but are inadequate in adapting to variations in non-cyclic and unpredictable tasks. This study aims to address this challenge by designing a novel electromyography (EMG)-driven musculoskeletal model for volitional control of a robotic ankle-foot prosthesis. The proposed controller ensures continuous control of the device, allowing users to freely manipulate the prosthesis behavior. A Hill-type muscle model was implemented to model a dorsiflexor and a plantarflexor to function around a virtual ankle joint. The model parameters for a subject specific model was determined by fitting the model to the experimental data collected from an able-bodied subject. EMG signals recorded from antagonist muscle pairs were used to activate the virtual muscle models. This model-based approach was then validated via offline simulations and real-time prosthesis control. Additionally, the feasibility of the proposed prosthesis control on assisting the user's functional tasks was demonstrated. The present control may further improve the function of robotic prosthesis for supporting versatile activities in individuals with lower-limb amputations.

In endoluminal surgery, the miniature instruments shall be of high accuracy and flexibility for minimal invasive diagnosis and surgical intervention. To this end, continuum robots with flexible joints have been proposed as the mechanism of endoscopic instruments. The compliance and deformability of the continuum joints enable access into the curved lumen. However, the manufacturing tolerances are normally not considered in the design procedure, and led to inaccuracy in the robotic control. To improve the control accuracy and flexibility of endoluminal surgical robots, we propose a novel design of a metal printed continuum joint in this paper, which incorporates a variable pitch design into the spring-like structure. The design can reduce the position errors accumulated on the distal tip of the joint, especially at large bending angles. The specification of variable pitch is investigated and determined with a friction model. In addition, to eliminate the distortion of the joint induced during the metal printing process, an extensive experiment was conducted to access the effect of the variables in the design (pitch, thickness, width and number of coils), with the aim of determining optimal parameters for reducing discrepancy caused by manufacturing variations. The final results indicated that the bending error of a single joint can be reduced from 18.10% to 4.63%, and a multi-segment prototype was developed to verify its effectiveness for potential surgical applications.

This paper presents a novel snake robot with the docking function, which can help the snake robots to connect with each other to achieve a stronger one with double length and double degrees of freedom. First, the mechanical design of the snake robot with docking function is introduced, including the body link and the head-tail passive docking mechanical structure. Second, the control system is built, and the control strategies of locomotion and docking are separately proposed. Then, the visual perception function is implemented for the target recognition during the docking process. Finally, the prototype is developed. The mobility and the docking function are fully verified and analyzed through the physical experiments.

In this study, the infant Cardiac Robotic Surgical System (iCROSS) is developed to assist a surgeon in performing the patent ductus arteriosus (PDA) closure and other infant cardiac surgeries. The iCROSS is a dual-arm robot allowing two surgical instruments to collaborate in a narrow space while keeping a sufficiently large workspace. Compared with the existing surgical robotic systems, the iCROSS meets the specific requirements of infant cardiac surgeries. Its feasibility has been validated through several teleoperated tasks performed in the experiment. In particular, the iCROSS is able to perform surgical ligation successfully within one minute.

The introduction of neuroendoscopy, microneu- rosurgery, neuronavigation, and intraoperative imaging for surgical operations has made significant improvements over other traditionally invasive surgical techniques. The integration of magnetic resonance imaging (MRI)-driven surgical devices with intraoperative imaging and endoscopy can enable further advancements in surgical treatments and outcomes. This work proposes the design and development of an MRI-driven endo- scope leveraging the high (3–7 T), external magnetic field of an MR scanner for heat-mitigated steering within the ventricular system of the brain. It also demonstrates the effectiveness of a Lorentz force-based grasper for diseased tissue manipulation and ablation. Feasibility studies show the neuroendoscope can be steered precisely within the lateral ventricle to locate a tumor using both MRI and endoscopic guidance. Results also indicate grasping forces as high as 31 mN are possible and power inputs as low as 0.69 mW can cause cancerous tissue ablation. These findings enable further developments of steerable devices using MR imaging integrated with endoscopic guidance for improved outcomes.

Soft grippers have the potential to solve many existing manipulation challenges, particularly in agile industry applications. However, existing soft grippers are often limited in the range of objects they can pick, or by cluttered environments. We present a design inspired by the nose and fingers at the end of an elephant's trunk, which can pick both by suction and pinching, allowing increased grasping diversity. In addition, we observe an emergent grasping mode, a hybrid of pinching and suction where the cup aperture is morphed online, using embedded soft fingers, to form a seal over challenging objects. An algorithmic grasping strategy, based-on analytical grasping models and primitive objects, is presented. With this, we predict grasping performance and show increased grasping range compared to other soft gripper designs. Finally, the gripper and grasping strategy are successfully applied to grasping more varied everyday objects, demonstrating exploitation of this multi-modal gripping for adaptive grasping.

Impressive animal locomotion capabilities are mediated by the co-evolution of the skeletal morphology and muscular properties. Legged robot performance would also likely benefit from the co-optimization of actuators and leg morphology. However, development of custom actuators for legged robots is expensive and time consuming, discouraging application-specific actuator optimization. This paper presents open-source designs for two quasi-direct-drive actuators with performance regimes appropriate for an 8–15 kg robot, built from off the shelf and 3D-printed components for less than $200 USD each. The mechanical, electrical, and thermal properties of each actuator are characterized and compared to benchmark data. Actuators subjected to 420k strides of gait data experienced only a 2% reduction in efficiency and 26 mrad in backlash growth, demonstrating viability for rigorous and sustained research applications. We present a thermal solution that nearly doubles the thermally-driven torque limits of our plastic actuator design. The performance results are comparable to traditional metallic actuators for use in high-speed legged robots of the same scale. These 3D printed designs demonstrate an approach for designing and characterizing low-cost, highly customizable and reproducible actuators, democratizing the field of actuator design and enabling co-design and optimization of actuators and robot legs.

Soft sensorised skins are essential for improving robotic manipulation capabilities towards that of humans. Integration of sensors into existing robotic hands is challenging due to rigidity of components, low packing density or poor sensor response. We propose a sensorised skin, based-on barometric sensing, which can be molded over a skeletal robot hand. The sensors connect air chambers embedded in the soft skin to wrist-mounted pressure sensors, allowing sensor spacing 2–4 mm, force ranges from 23 mN to 5700 mN and bandwidth of 20 Hz. Integrating this with a skeletal hand allows us to showcase the potential of these sensors to aid robotic manipulation. We demonstrate 3-axis contact modelling, useful for in-hand manipulation and exploration. In addition, by grasping a chopstick and sensing forces transmitted from the environment, the system can remotely detect small environmental features, e.g., hole finding using tools.

In aerial cooperative transportation missions, it has been recognized that for small-sized but heavy payloads, the cable-suspended framework is a preferred manner. However, to maintain proper safe flight distances, cables always stay inclined, which implies that horizontal force components have to be generated by UAVs, and only partial thrust forces are used for gravity compensation. To overcome this drawback, in this paper, a new cooperative transportation system named Truss Aerial Transportation System (TATS) is proposed, where those horizontal forces can be internally compensated by the bar spherical joint structure. In the TATS, rigid bars can powerfully sustain the desired distances among UAVs for safe flight, resulting in a more compact and effective transportation system. Thanks to the structural advantage of the truss, the rigid bars can be made lightweight so as to minimize their induced gravity burden. The construction method of the proposed TATS is presented. The improvement in energy efficiency is analyzed and compared with the cable-suspended framework. Furthermore, the robustness property of a TATS configuration is evaluated by computing the margin capacity. Finally, a load test experiment is conducted on our made prototype, the results of which show the effectiveness and feasibility of the proposed TATS.

Mantis shrimp produce one of the fastest strikes in the animal kingdom, their striking appendages reaching tip velocities of tens of meters per second underwater. Their ultrafast movement is capable of crushing the shells of prey and generating cavitation bubbles, and has long raised interest from the scientific community. To study the underlying mechanisms and operating principles behind these behaviors, prior research has developed physical models that mimic the motions and speeds of mantis shrimp. That microrobot demonstrated speeds of approximately 5 m/s in water and 26 m/s in air. Here we utilize an accurate dynamical model of the four-bar mechanism and geometric latch observed in biological shrimp in a numerical trajectory optimization approach to find the design changes that can maximize the microrobot's striking velocities. Through a suboptimization problem maximizing the energy loaded in the mechanism's spring, we manage to improve the performance of the microrobot by over 58%, reaching tip velocities of 41.2 \pm 0.641.2 \pm 0.6 m/s.

Motion tracking interfaces are intuitive for free-form teleoperation tasks. However, efficient manipulation control can be difficult with such interfaces because of issues like the interference of unintended motions and the limited precision of human motion control. The limitation in control efficiency reduces the operator's performance and increases their workload and frustration during robot teleoperation. To improve the efficiency, we proposed separating controlled degrees of freedom (DoFs) and adjusting the motion scaling ratio of a motion tracking interface. The motion tracking of handheld controllers from a Virtual Reality system was used for the interface. We separated the translation and rotational control into: 1) two controllers held in the dominant and non-dominant hands and 2) hand pose tracking and trackpad inputs of a controller. We scaled the control mapping ratio based on 1) the environmental constraints and 2) the teleoperator's control speed. We further conducted a user study to investigate the effectiveness of the proposed methods in increasing efficiency. Our results show that the separation of position and orientation control into two controllers and the environment-based scaling methods perform better than their alternatives.

Self-supervised monocular depth estimation has been widely investigated to estimate depth images and relative poses from RGB images. This framework is promising because the depth and pose networks can be trained from just time-sequence images without the need for the ground truth depth and poses. In this work, we estimate the depth around a robot (360° view) using time-sequence spherical camera images, from a camera whose parameters are unknown. We propose a learnable axisymmetric camera model which accepts distorted spherical camera images with two fisheye camera images as well as pinhole camera images. In addition, we trained our models with a photo-realistic simulator to generate ground truth depth images to provide supervision. Moreover, we introduced loss functions to provide floor constraints to reduce artifacts that can result from reflective floor surfaces. We demonstrate the efficacy of our method using the spherical camera images from the GO Stanford dataset and pinhole camera images from the KITTI dataset to compare our method's performance with that of baseline method in learning the camera parameters.

Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sample-efficient result than learning by pixels. This work presents a new approach that extracts information from a depth map estimation to teach an RL agent to perform the mapless navigation of Unmanned Aerial Vehicle (UAV). We propose the Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning (Depth-CUPRL) that estimates the depth of images with a prioritized replay memory. We used a combination of RL and Contrastive Learning to lead with the problem of RL based on images. From the analysis of the results with Unmanned Aerial Vehicles (UAVs), it is possible to conclude that our Depth-CUPRL approach is effective for the decision-making and outperforms state-of-the-art pixel-based approaches in the mapless navigation capability.

While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment environments of RAS pose new challenges on its dependability. Although existing works impose constraints on the DRL policy to ensure successful completion of the mission, it is far from adequate to assess the DRL-driven RAS in a holistic way considering all dependability properties. In this paper, we formally define a set of dependability properties in temporal logic and construct a Discrete-Time Markov Chain (DTMC) to model the dynamics of risk/failures of a DRL-driven RAS interacting with the stochastic environment. We then conduct Probabilistic Model Checking (PMC) on the designed DTMC to verify those properties. Our experimental results show that the proposed method is effective as a holistic assessment framework while uncovering conflicts between the properties that may need trade-offs in training. Moreover, we find that the standard DRL training cannot improve dependability properties, thus requiring bespoke optimisation objectives. Finally, our method offers sensitivity analysis of dependability properties to disturbance levels from environments, providing insights for the assurance of real RAS.

We present DOME, a novel method for one-shot imitation learning, where a task can be learned from just a single demonstration and then be deployed immediately, without any further data collection or training. DOME does not require prior task or object knowledge, and can perform the task in novel object configurations and with distractors. At its core, DOME uses an image-conditioned object segmentation network followed by a learned visual servoing network, to move the robot's end-effector to the same relative pose to the object as during the demonstration, after which the task can be completed by replaying the demonstration's end-effector velocities. We show that DOME achieves near 100% success rate on 7 real-world everyday tasks, and we perform several studies to thoroughly understand each individual component of DOME. Videos and supplementary material are available at: https://www.robot-learning.uk/dome.

This paper presents the DeltaZ robot, a centimeter-scale, low-cost, delta-style robot that allows for a broad range of capabilities and robust functionalities. The DeltaZ robot is 3D-printed from soft and rigid materials with a design that is easy to assemble and maintain, and lowers the barriers to utilize. Functionality of the robot stems from its three translational degrees of freedom and a closed form kinematic solution which makes manipulation problems more intuitive compared to many other manipulators. Moreover, the low cost of the robot presents an opportunity to democratize manipulators for research and education settings. We describe how the robot can be used as a reinforcement learning bench-mark. Open-source 3D-printable designs and code for building and using the robot are available to the public.

Decision-making among groups of humans can benefit from open discussion and inclusion of a diversity of opinions, promoting deliberative democracy. In this work, we test whether a swarm of robots can help facilitate decision-making by visually representing the diversity of opinions. We used a swarm of robots we built, called MOSAIX, that consists of 4-inch touchscreens-on-wheels robots called Tiles. The robots acted as physical avatars for opinions, helping them travel and mix together. We recruited 46 participants split into groups of 7 and 8 to test whether the robot movement had an impact on the decision-making process versus using the robots stationary in the participants' hands akin to smartphones. Furthermore, we wanted to test whether the participants felt comfortable expressing their opinion through the robots. Results show the participants indeed felt comfortable using the robots, and user engagement increased with the movement of the robots. The difference between the participants' first and last opinions also increased with the movement of the robots. We believe that robot swarms have not been used before to facilitate decision-making among a group of people. Therefore, our contribution is in testing the possibility of how and whether using a moving robot swarm helps humans reach a decision.

In this paper, we present methods for capturing 4D body shapes of swimming fish with affordable small training datasets and textureless 2D videos. Automated capture of spatiotemporal animal movements and postures is revolutionizing the study of collective animal behavior. 4D (including 3D space + time) shape data from animals like schooling fish contains a rich array of social and non-social information that can be used to shed light on the fundamental mechanisms underlying collective behavior. However, unlike the large datasets used for 4D shape reconstructions of the human body, there are no large amounts of labeled training datasets for reconstructing fish bodies in 4D, due to the difficulty of underwater data collection. We created a template mesh model using 3D scan data from a real fish, then extracted silhouettes (segmentation masks) and key-points of the fish body using Mask R-CNN and DeepLabCut, respectively. Next, using the Adam optimizer, we optimized the 3D template mesh model for each frame by minimizing the difference between the projected 3D model and the detected silhouettes as well as the key-points. Finally, using an LSTM-based smoother, we generated accurate 4D shapes of schooling fish based on the 3D shapes over each frame. Our results show that the method is effective for 4D shape reconstructions of swimming fish, with greater fidelity than other state-of-the-art algorithms.