IROS 2023 Papers

This paper presents a haptic device with a simple architecture of only two limbs that can provide translational motion in three degrees of freedom (DOF) and one-DOF rotational motion. Actuation redundancy eliminates all forward-kinematic singularities and improves the motion-force transmission property. Thanks to the special structure of the kinematic chains, all actuators are close to the base and full gravity compensation is achieved passively by using springs. Force producibility analysis shows that this haptic device is able to produce long-term continuous force feedback of 15–30 N in each direction. By developing a prototype of the haptic device and a virtual three-dimensional simulator, a preliminary performance evaluation of the haptic device was conducted. In addition, a torque distribution algorithm considering a relaxed form of actuator-torque saturation was experimentally evaluated, and a comparison with other algorithms reveals that this algorithm offers several advantages.

The success rate of robotic-assisted needle-guided interventions for applications such as tissue biopsy and targeted drug delivery relies heavily on the accuracy of the needle placement. Tissue shift and needle tip deflection due to needle-tissue interaction are some factors that can adversely affect the outcome of these procedures. In this paper, we present a novel algorithm for robotically-steered bevel tip needles that provides variable needle curvatures by continuously controlling the rotation speed of the needle in a bidirectional manner. Our algorithm is an extension of the Continuous Rotation and Variable Curvature (CURV) algorithm and extends its use with wired sensorized needles. Additionally, we present algorithms for the implementation of our proposed method for closed-loop needle steering in robotic systems with image or sensor feedback. To validate our approach, we perform two benchtop needle insertion experiments in a gelatin phantom and ex vivo tissue. In the first experiment, we demonstrate the capability of our proposed algorithm in achieving variable curvatures and compare it with the CURV algorithm and our simulation results. The second experiment studies the effect of the unidirectional and bidirectional needle steering on the tissue wind-up using a novel force collection setup. Our results highlight the capability of the proposed algorithm in achieving variable curvature profiles and suggest a potential advantage compared to the original method in terms of reducing the imbalanced forces sensed at the load cell due to the needle-tissue friction buildup.

Existing in-pipe robots have insufficient adaptability when dealing with accidents in unfamiliar pipe environments. Developing a pipe robot that can be designed and manufactured quickly is one solution. The tensegrity structure is a self-stressing spatial structure formed by the interaction of rigid members and flexible cables, which has the advantages of simple structure, good flexibility, deformability, and impact resistance. Inspired by this structure, we design a novel worm-like tensegrity robot for different pipe environments, which can be manufactured rapidly at low cost. Firstly, a robotic module based on the tensegrity structure is designed inspired by the motion patterns of worm-like organisms. Then, the design process of the module is presented based on the mathematical analysis of the deformation. Finally, a prototype of the tensegrity robot is developed using simple and low-cost parts in less than an hour. To test the motion performance, load performance, and inspection capability of the tensegrity robot, we designed a series of experiments on horizontal pipes, vertical pipes, elbows, and steel pipes. Experimental results show that the worm-like tensegrity robot is simple in structure, easy to manufacture, low in cost, and good in performance.

In this paper, with the goal of addressing the high early-detection miss rate of colorectal cancer (CRC) polyps during a colonoscopy procedure, we propose the design and fabrication of a unique inflatable vision-based tactile sensing balloon (VTSB). The proposed soft VTSB can readily be integrated with the existing colonoscopes and provide a radiation-free, safe, and high-resolution textural mapping and morphology characterization of CRC polyps. The performance of the proposed VTSB has been thoroughly characterized and evaluated on four different types of additively manufactured CRC polyp phantoms with three different stiffness levels. Additionally, we integrated the VTSB with a colonoscope and successfully performed a simulated colonoscopic procedure inside a tube with a few CRC polyp phantoms attached to its internal surface.

Pipelines have become one of the most important infrastructures in the city. Over time, they are prone to aging, cracks, corrosion, and the demand for regular inspection is gradually increasing. Robotic solutions are effective methods for in-pipe inspection. However, existing In-pipe Inspection Robots (IPIR) require that the inner diameter of the pipe is fixed in the application scenarios, and need extra labor to control the robot and handle the cable. In this work, we design and develop a deformable robot to adapt to pipes with different inner diameters. Specifically, the passive elastic hinge is used by us to make the robot fully in contact with the pipe, generating enough friction to ensure that the robot is attached to the inner wall of the pipe. An edge device is deployed on the robot, generating velocity commands of wheels through the data from Inertial Measurement Unit (IMU), which eliminates the need for external devices. Experimental results demonstrate that the robot can move in horizontal and vertical pipelines, as well as traverse through pipe joints and scenarios where there is dirty or small obstacle.

This paper presents a design and characterization of a micropositioning stage driven by a reluctance actuator. The stage is constructed with a C-core reluctance actuator and four compression springs. The design of the stage is presented using a CAD model, followed by the fabrication process of the prototype. The mathematical model is formulated to present the interaction among the stage's electrical, magnetic, and mechanical dynamic behaviour. Next, the force-current and force-gap characteristics are obtained by measuring the force under different applied currents and air gaps. After that, the system is analyzed to determine the maximum applied voltage that stabilizes the system in an open-loop configuration, followed by the time-domain and frequency-domain response. Finally, the feedforward controller is presented to linearize the dynamic behavior of the stage over a specific range of motion. The experimental results under the feedforward controller show a linear characteristic between the desired force and the output displacement.

This paper presents the design and control of a ballbot drivetrain that aims to achieve high agility, minimal footprint, and high payload capacity while maintaining dynamic stability. Two hardware platforms and analytical models were developed to test design and control methodologies. The full-scale ballbot prototype (MiaPURE) was constructed using off-the-shelf components and designed to have agility, footprint, and balance similar to that of a walking human. The planar inverted pendulum testbed (PIPTB) was developed as a reduced-order testbed for quick validation of system performance. We then proposed a simple yet robust cascaded LQR-PI controller to balance and maneuver the ballbot drivetrain with a heavy payload. This is crucial because the drivetrain is often subject to high stiction due to elastomeric components in the torque transmission system. This controller was first tested in the PIPTB to compare with traditional LQR and cascaded PI-PD controllers, and then implemented in the ballbot drivetrain. The MiaPURE drivetrain was able to carry a payload of 60 kg, achieve a maximum speed of 2.3 m/s, and come to a stop from a speed of 1.4 m/s in 2 seconds in a selected translation direction. Finally, we demonstrated the omnidirectional movement of the ballbot drivetrain in an indoor environment as a payload-carrying robot and a human-riding mobility device. Our experiments demonstrated the feasibility of using the ballbot drivetrain as a universal mobility platform with agile movements, minimal footprint, and high payload capacity using our proposed design and control methodologies.

This paper presents the design and control of a novel microrobot that utilizes two distinct magnetic locomotion methods, a combination of rotating and gradient field control, for precise micro-object manipulation using multiple end-effectors. Rotating magnetic fields induce a tumbling locomotion mode to increase the movement speed and decrease issues associated with stiction and locomotion over rough surfaces. The gradient field control allows for precise manipulation using the end-effectors, which include a pointed tip for splitting groups of objects and a blunt end for pushing or capturing objects. The microrobot is fabricated using a two-photon polymerization 3D printer, allowing for the precise reproduction of complex geometries and designs. The potential applications of this technology in the medical field are discussed, highlighting the potential for in vitro cellular manipulation.

Human Pose Estimation (HPE) to assess human motion in sports, rehabilitation or work safety requires accurate sensing without compromising the sensitive underlying personal data. Therefore, local processing is necessary and the limited energy budget in such systems can be addressed by Inertial Measurement Units (IMU) instead of common camera sensing. The central trade-off between accuracy and efficient use of hardware resources is rarely discussed in research. We address this trade-off by a simulative Design Space Exploration (DSE) of a varying quantity and positioning of IMU -sensors. First, we generate IMU-data from a publicly available body model dataset for different sensor configurations and train a deep learning model with this data. Additionally, we propose a combined metric to assess the accuracy-resource trade-off. We used the DSE as a tool to evaluate sensor configurations and identify beneficial ones for a specific use case. Exemplary, for a system with equal importance of accuracy and resources, we identify an optimal sensor configuration of 4 sensors with a mesh error of 6.03 cm, increasing the accuracy by 32.7 % and reducing the hardware effort by two sensors compared to state of the art. Our work can be used to design health applications with well-suited sensor positioning and attention to data privacy and resource-awareness.

In this paper, we focus on the kangaroo, which has powerful legs capable of jumping and a soft and strong tail. To incorporate these unique structure into a robot for utilization, we propose a design method that takes into account both the feasibility as a robot and the kangaroo-mimetic structure. Based on the kangaroo's musculoskeletal structure, we determine the structure of the robot that enables it to jump by analyzing the muscle arrangement and prior verification in simulation. Also, to realize a tail capable of body support, we use an articulated, elastic structure as a tail. In order to achieve both softness and high power output, the robot is driven by a direct-drive, high-power wire-winding mechanism, and weight of legs and the tail is reduced by placing motors in the torso. The developed kangaroo robot can jump with its hind legs, moving its tail, and supporting its body using its hind legs and tail.

Accurately estimating the 6D pose of objects is crucial for many applications, such as robotic grasping, autonomous driving, and augmented reality. However, this task becomes more challenging in poor lighting conditions or when dealing with textureless objects. To address this issue, depth images are becoming an increasingly popular choice due to their invariance to a scene's appearance and the implicit incorporation of essential geometric characteristics. However, fully leveraging depth information to improve the performance of pose estimation remains a difficult and under-investigated problem. To tackle this challenge, we propose a novel framework called SwinDePose, that uses only geometric information from depth images to achieve accurate 6D pose estimation. SwinDePose first calculates the angles between each normal vector defined in a depth image and the three coordinate axes in the camera coordinate system. The resulting angles are then formed into an image, which is encoded using Swin Transformer. Additionally, we apply RandLA-Net to learn the representations from point clouds. The resulting image and point clouds embeddings are concatenated and fed into a semantic segmentation module and a 3D keypoints localization module. Finally, we estimate 6D poses using a least-square fitting approach based on the target object's predicted semantic mask and 3D keypoints. In experiments on the LineMod and Occlusion LineMod, SwinDePose outperforms existing state-of-the-art methods for 6D object pose estimation using depth images. We also provide competitive results on the YCB-Video dataset even without post-processing. This demonstrates the effectiveness of our approach and highlights its potential for improving performance in real-world scenarios. Our code is at https://github.com/zhujunli1993/SwinDePose.

In this paper, we tackle the problem of generating a novel image from an arbitrary viewpoint given a single frame as input. While existing methods operating in this setup aim at predicting the target view depth map to guide the synthesis, without explicit supervision over such a task, we jointly optimize our framework for both novel view synthesis and depth estimation to unleash the synergy between the two at its best. Specifically, a shared depth decoder is trained in a self-supervised manner to predict depth maps that are consistent across the source and target views. Our results demonstrate the effectiveness of our approach in addressing the challenges of both tasks allowing for higher-quality generated images, as well as more accurate depth for the target viewpoint.

Some animals exhibit multi-modal locomotion capability to traverse a wide range of terrains and environments, such as amphibians that can swim and walk or birds that can fly and walk. This capability is extremely beneficial for expanding the animal's habitat range and they can choose the most energy efficient mode of locomotion in a given environment. The robotic biomimicry of this multi-modal locomotion capability can be very challenging but offer the same advantages. However, the expanded range of locomotion also increases the complexity of performing localization and path planning. In this work, we present our morphing multi-modal robot, which is capable of ground and aerial locomotion, and the implementation of readily available SLAM and path planning solutions to navigate a complex indoor environment.

The success of re-localisation has crucial implications for the practical deployment of robots operating within a prior map or relative to one another in real-world scenarios. Using single-modality, place recognition and localisation can be compromised in challenging environments such as forests. To address this, we propose a strategy to prevent lidar-based re-localisation failure using lidar-image cross-modality. Our solution relies on self-supervised 2D-3D feature matching to predict alignment and misalignment. Leveraging a deep network for lidar feature extraction and relative pose estimation between point clouds, we train a model to evaluate the estimated transformation. A model predicting the presence of misalignment is learned by analysing image-lidar similarity in the embedding space and the geometric constraints available within the region seen in both modalities in Euclidean space. Experimental results using real datasets (offline and online modes) demonstrate the effectiveness of the proposed pipeline for robust re-localisation in unstructured, natural environments.

Nowadays, transportation networks face the challenge of sub-optimal control policies that can have adverse effects on human health, the environment, and contribute to traffic congestion. Increased levels of air pollution and extended commute times caused by traffic bottlenecks make intersection traffic signal controllers a crucial component of modern transportation infrastructure. Despite several adaptive traffic signal controllers in literature, limited research has been conducted on their comparative performance. Furthermore, despite carbon dioxide (CO2) emissions' significance as a global issue, the literature has paid limited attention to this area. In this report, we propose EcoLight, a reward shaping scheme for reinforcement learning algorithms that not only reduces CO2 emissions but also achieves competitive results in metrics such as travel time. We compare the performance of tabular Q-Learning, DQN, SARSA, and A2C algorithms using metrics such as travel time, CO2 emissions, waiting time, and stopped time. Our evaluation considers multiple scenarios that encompass a range of road users (trucks, buses, cars) with varying pollution levels.

Movement primitives are trainable parametric models that reproduce robotic movements starting from a limited set of demonstrations. Previous works proposed simple linear models that exhibited high sample efficiency and generalization power by allowing temporal modulation of move-ments (reproducing movements faster or slower), blending (merging two movements into one), via-point conditioning (constraining a movement to meet some particular via-points) and context conditioning (generation of movements based on an observed variable, e.g., position of an object). Previous works have proposed neural network-based motor primitive models, having demonstrated their capacity to perform tasks with some forms of input conditioning or time-modulation representations. However, there has not been a single unified deep movement primitive's model proposed that is capable of all previous operations, limiting neural movement primitive's potential applications. This paper proposes a deep movement primitive architecture that encodes all the operations above and uses a Bayesian context aggregator that allows a more sound context conditioning and blending. Our results demonstrate our approach can scale to reproduce complex motions on a larger variety of input choices compared to baselines while maintaining operations of linear movement primitives provide.

Automating leaf detection and physical leaf sample collection using Internet of Things (IoT) technologies is a crucial task in precision agriculture. In this paper, we present a deep learning-based approach for detecting and segmenting crop leaves for robotic physical sampling. We discuss a method for generating a physical dataset of agricultural crops. Our proposed pipeline incorporates using an RGB-D camera for dataset collection, fusing the depth frame along with RGB images to train Mask R-CNN and YOLOv5 models. We also propose our novel leaf pose estimating algorithm for physical sampling and maximizing leaf sample area while using a robotic arm integrated to the P-AgBot platform. The proposed approach has been experimentally validated on corn and sorghum, in both indoor and outdoor environments. Our method has achieved a best-case detection rate of 90.6%, a 9% smaller error compared to our previous method, and approximately 80% smaller error compared to other state-of-the-art methods in estimating the leaf position.

This paper introduces a novel approach to address the problem of Physical Robot Interaction (PRI) during robot pushing tasks. The approach uses a data-driven forward model based on tactile predictions to inform the controller about potential future movements of the object being pushed, such as a strawberry stem, using a robot tactile finger. The model is integrated into a Deep Functional Predictive Control (d-FPC) system to control the displacement of the stem on the tactile finger during pushes. Pushing an object with a robot finger along a desired trajectory in 3D is a highly nonlinear and complex physical robot interaction, especially when the object is not stably grasped. The proposed approach controls the stem movements on the tactile finger in a prediction horizon. The effectiveness of the proposed FPC is demonstrated in a series of tests involving a real robot pushing a strawberry in a cluster. The results indicate that the d-FPC controller can successfully control PRI in robotic manipulation tasks beyond the handling of strawberries. The proposed approach offers a promising direction for addressing the challenging PRI problem in robotic manipulation tasks.

Due to the high complexity of a football match, the opponents' strategies are variable and unknown. Thus predicting the opponents' future intentions accurately based on current situation is crucial for football players' decision-making. To better anticipate the opponents and learn more effective strategies, a deconfounded opponent intention inference (DOII) method for football multi-player policy learning is proposed in this paper. Specifically, opponents' intentions are inferred by an opponent intention supervising module. Furthermore, for some confounders which affect the causal relationship among the players and the opponents, a decon-founded trajectory graph module is designed to mitigate the influence of these confounders and increase the accuracy of the inferences about opponents' intentions. Besides, an opponent-based incentive module is designed to improve the players' sensitivity to the opponents' intentions and further to train reasonable players' strategies. Representative results indicate that DOII can effectively improve the performance of players' strategies in the Google Research Football environment, which validates the superiority of the proposed method.