ICRA 2024 Papers

Severe chronic pulmonary endarterectomy needs a dissector to delicately remove proliferative intima located in the depth of the pulmonary artery. This work proposed a novel endoscopic robotized steerable dissector for this surgery, enabling easier access to curved deep artery branches. The handheld surgical dissector also provides suction and visualization for surgeons to enhance effectiveness. The steerable section is a cable-driven hinged structure, and through an antagonistic mechanism regulating the cable tension, the overall stiffness is adjusted to adapt various surroundings. The mapping between actuation space and shape configuration and tip force estimation model are respectively established for further closed-loop control scheme, achieving adaptive positioning and safe surgery. Experiments first demonstrate the feasibility of the proposed models and ex vitro trials validated the usage and effectiveness of the robotized dissector.

Small-scale robots hold significant promise in the field of minimally invasive surgery (MIS). In this paper, we present a miniature magnetic gripper and develop a data-driven kinematic model. The gripper comprises four fingers, wherein each finger has a maximum size not exceeding 3mm, 4mm and 5.5mm in three dimensions. By integrating permanent magnets and elastic ropes as internal actuation elements into the fingers, the gripper is equipped with the capability to open-close under an external magnetic field, facilitating the manipulation of small objects in confined spaces. Modeling and analysis of the magnetic gripper are undertaken, wherein the relationship between the open angle and the external magnetic field is established. The average error between the experimentally observed open angles and the model-predicted values is 2.31°. Subsequent experiments demonstrated the necessity of the magnetic gripper model for precise manipulation, verified its excellent sensitivity to magnetic fields, and demonstrated its potential for future applications in MIS.

Origami designs and structures have been widely used in many fields, such as morphing structures, robotics, and metamaterials. However, the design and fabrication of origami structures rely on human experiences and skills, which are both time and labor-consuming. In this paper, we present a rapid design and fabrication method for string-driven origami structures and robots. We developed an origami design software to generate desired crease patterns based on analytical models and Evolution Strategies (ES). Additionally, the software can automatically produce 3D models of origami designs. We then used a dual-material 3D printer to fabricate those wrapping-based origami structures with the required mechanical properties. We utilized Twisted String Actuators (TSAs) to fold the target 3D structures from flat plates. To demonstrate the capability of these techniques, we built and tested an origami crawling robot and an origami robotic arm using 3D-printed origami structures driven by TSAs.

This paper presents the design and experimental results of a proprioceptive, high-bandwidth quasi-direct drive (QDD) actuator for highly dynamic robotic applications. A comprehensive review of the mechanical design of the PULSE115-60 actuator is presented, with particular focus on the design parameters affecting the dynamic performance of the actuator and a full specification is provided. Fundamental parameters to describe the dynamic behaviour of an actuator are discussed, and an experimental method to determine speed and torque bandwidth of the actuator is presented. A rigorous method to determine backdrive torque is also explained. Finally, experimental results quantifying the dynamic performance of the PULSE115-60 actuator are discussed. The PULSE115-60 actuator has a highly dynamic response, surpassing the torque bandwidth at low torque amplitudes showcased in state-of-the-art literature. The differences between current and torque bandwidth, two concepts often conflated in literature, are elucidated. Experimental procedures detailed in previous work are discussed and a novel standardised procedure is proposed for robust characterisation and fair comparison of different actuation systems. Finally, performance results for PULSE115-60 are presented, demonstrating a torque bandwidth of 66.3 Hz at an amplitude of 6 N•m, ±0.11° of backlash and 0.37 N•m of backdrive torque.

With the development of society, aging population and the number of stroke patients is increasing year by year. Rehabilitation exoskeleton can help patients to carry out rehabilitation training and improve their activities of daily living (ADL). First of all, a reconfigurable exoskeleton for upper limb rehabilitation is designed in this paper. The exoskeleton combines gravity compensation with left-right arm switching function through its reconfigurability. Secondly, the motion space and singular configuration of the exoskeleton are analyzed. By changing the working mode of the gravity compensation device, the control experiment of the motor is carried out. The influence of gravity compensation device on motor driving torque and energy consumption is analyzed. Finally, the results of experiment show that, in the best case, the gravity compensation device can reduce the energy consumption by 41.15% and the maximum motor current by 33.56% of the driving element.

Motion planning techniques for quadrotors have advanced significantly over the past decade. Most successful planners have two stages: a front-end that determines a path that incorporates geometric (or kinematic or input) constraints and specifies the homotopy class of the trajectory, and a back-end that optimizes this path to respect dynamics and input constraints. While there are many different choices for each stage, the eventual performance depends critically not only on these choices, but also on the environment. Given a new environment, it is difficult to decide a priori how one should design a motion planner. In this work, we develop (i) a procedure to construct parametrized environments, (ii) metrics that characterize the difficulty of motion planning in these environments, and (iii) an open-source software stack that can be used to combine a wide variety of two-stage planners seamlessly. We perform experiments in simulations and a real platform. We find, somewhat conveniently, that geometric front-ends are sufficient for environments with varying complexities if combined with dynamics-aware backends. The metrics we designed faithfully capture the planning difficulty in a given environment. All code is available at https://github.com/KumarRobotics/kr_mp_design.

Soft actuators have shown advantages of adaptiveness, large deformation, and safe human-robot interaction, making them suitable for various applications. Herein, a novel soft flat tube twisting actuator (SFTTA) is proposed. The SFTTA is composed of a folded flat tube sandwiched between two silicone rubber laminates. When inflated by compressed air, the folded corners of the flat tube tend to unfold, resulting in the twist of the actuator to a helical structure. The SFTTA has great scalability. It can be fabricated through simple processes with low-cost materials. For a sample SFTTA with the size of a human finger, it can twist 5400 at an air pressure of 300 kPa. In general, SFTTA based actuators can twist 9.6 degree per millimeter in length, which is significantly larger than previously reported soft twisting actuators. Additionally, the composite-like SFTTA allows mechanical property programming through the alteration of folding patterns of the flat tube and the material structure of the elastomer laminates. Finally, an extensible soft gripper based on flat tube actuators and a robotic wrist module are developed, and their rotation is realized by the proposed SFTTA actuator.

This paper introduces a new wheel-legged robot and develops motion controllers based on central pattern generators (CPGs) for the robot to navigate over a range of terrains. A transformable leg-wheel design is considered and characterized in terms of key locomotion characteristics as a function of the design. Kinematic analysis is conducted based on a generalized four-bar mechanism driven by a coaxial hub arrangement. The analysis is used to inform the design of a central pattern generator to control the robot by mapping oscillator states to wheel-leg trajectories and implementing differential steering within the oscillator network. Three oscillator models are used as the basis of the CPGs, and their performance is compared over a range of inputs. The CPG-based controller is used to drive the developed robot prototype on level ground and over obstacles. Additional simulated tests are performed for uneven terrain negotiation and obstacle climbing. Results demonstrate the effectiveness of CPG control in transformable wheel-legged robots.

Soft manipulators offer the advantages of safety and adaptability. However, due to insufficient stiffness and single motion mode limitations, existing soft manipulators usually exhibit low load capacity and small working space. To address this problem, we propose a novel soft hybrid-driven manipulator with continuous stiffness control capability and multiple motion patterns (omnidirectional bending and extension). Furthermore, we develop kinematic and stiffness models based on the constant curvature assumption. The soft robot consists of a soft bellows actuator and inextensible rigid skeletons, which exhibit a high extension ratio and low drive pressure. With the antagonistic actuation of tendon-pulling and air-pushing, the robot can achieve independent control over stiffness and position in three-dimensional space. The performance associated with the designed soft hybrid-driven manipulator is experimentally verified. The robot can achieve an elongation of 198% and a maximum bending angle of up to 240°. The robot can also increase stiffness by increasing internal air pressure to resist deformation caused by external loads. Additionally, tracking experiments with various trajectories in space verify the accuracy of the kinematic model, which indicates that the soft manipulator can stabilize motion within a broad workspace.

In this paper, we propose a model-mediated tele-operation (MMT) system using an octree-based model (OBM) to spatially map the environment impedance for the emerging use cases in Tactile Internet. Different from the existing just-noticeable-difference (JND) based MMT, our method avoids continuous transmission of environment impedance. Moreover, it allows the local model to generate accurate force feedback and reduces the number of model updates. Furthermore, the OBM can be deployed with or without previous knowledge of the environment. An online estimation of the OBM is proposed using a JND and a rate-of-change threshold. An offline estimation method is also proposed when the geometry and impedance parameters of the remote environment are known. In addition, a point cloud-based force rendering algorithm is tailored to use the OBM, thereby allowing the generating of force feedback for complex environments. An experiment without human-in-the-loop was conducted, showing that for an online estimated OBM, the accuracy of the force feedback was improved by up to 44 percent while using less than half the number of model updates when compared to JND-based MMT. Another experiment with a human operator interacting with a virtual environment showed that using an offline estimated OBM improves the accuracy of the force feedback and is reliable against packet loss and short temporal breakdown of the communication link.

This study presents the design and validation of a variable stiffness actuator incorporating multiple cam mechanisms. The actuator is intended for use in walking assistance, focusing on assisting individuals with diminished ankle function. This study highlights the advantages of variable stiffness actuators over traditional and other modern actuators in mobility assistance. The working principles of the proposed Variable Stiffness Spiral Cam Actuator (VS-SCA) are described, focusing on the cantilever beams with adjustable supports, main cam mechanism, and symmetric support positioning architecture utilizing an Archimedean spiral cam. The design and fabrication process are discussed, considering system design considerations, cantilever beam design, cam design, and spiral cam design. The analytical methodology used for validation is also presented, which connects the subsystems of the actuator and allows for the determination of effective torsional stiffness. The experimental validation showed that the VS-SCA provides a range of stiffness from 20 to 75 Nm/rad for dorsiflexion, necessary for providing ankle assistance during the push-off phase of walking, while maintaining low stiffness (4 - 12 Nm/rad) for plantarflexion not to hinder natural ankle motion in the swing phase.

Community-based locomotor training post-stroke has shown improvements in independent ambulation by increasing dose, intensity, and specificity of walking practice. Robotic ankle exoskeletons hold the potential to facilitate continued rehabilitation at home, but understanding what aspects of the design are most relevant for successful translation to the community presents a challenge. Here, we design a portable rigid ankle exoskeleton to use as a research platform for investigating the effect of assistance on post-stroke gait during overground, community-based walking. We first test our device with stroke survivors and validate its potential for future community use. We then present a systematic method for quantifying power transmission losses at each transmission stage from the battery to the wearer, using data gathered from walking trials with healthy participants. Our evaluation method revealed inefficiencies in power transfer at the interface level, likely resulting from the compliance in the structural components of the system, which motivates future redesign considerations. Overall, our method provides a framework to identify and characterize the components that must be redesigned to lower exoskeleton weight and maximize performance.

When capturing images through the glass during rainy or snowy weather conditions, the resulting images often contain waterdrops adhered on the glass surface, and these waterdrops significantly degrade the image quality and performance of many computer vision algorithms. To tackle these limitations, we propose a method to reconstruct the clear 3D scene implicitly from multi-view images degraded by water-drops. Our method exploits an attention network to predict the location of waterdrops and then train a Neural Radiance Fields to recover the 3D scene implicitly. By leveraging the strong scene representation capabilities of NeRF, our method can render high-quality novel-view images with waterdrops removed. Extensive experimental results on both synthetic and real datasets show that our method is able to generate clear 3D scenes and outperforms existing state-of-the-art (SOTA) image adhesive waterdrop removal methods.

Dexterous manipulation, especially of small daily objects, continues to pose complex challenges in robotics. This paper introduces the DenseTact-Mini, an optical tactile sensor with a soft, rounded, smooth gel surface and compact design equipped with a synthetic fingernail. We propose three distinct grasping strategies: tap grasping using adhesion forces such as electrostatic and van der Waals, fingernail grasping leveraging rolling/sliding contact between the object and fingernail, and fingertip grasping with two soft fingertips. Through comprehensive evaluations, the DenseTact-Mini demonstrates a lifting success rate exceeding 90.2% when grasping various objects, including items such as 1mm basil seeds, thin paperclips, and items larger than 15mm such as bearings. This work demonstrates the potential of soft optical tactile sensors for dexterous manipulation and grasping.

Enabling autonomous robots to operate robustly in challenging environments is necessary in a future with increased autonomy. For many autonomous systems, estimation and odometry remains a single point of failure, from which it can often be difficult, if not impossible, to recover. As such robust odometry solutions are of key importance. In this work a method for tightly-coupled LiDAR-Radar-Inertial fusion for odometry is proposed, enabling the mitigation of the effects of LiDAR degeneracy by leveraging a complementary perception modality while preserving the accuracy of LiDAR in well-conditioned environments. The proposed approach combines modalities in a factor graph-based windowed smoother with sensor information-specific factor formulations which enable, in the case of degeneracy, partial information to be conveyed to the graph along the non-degenerate axes. The proposed method is evaluated in real-world tests on a flying robot experiencing degraded conditions including geometric self-similarity as well as obscurant occlusion. For the benefit of the community we release the datasets presented: https://github.com/ntnu-arl/lidar_degeneracy_datasets.

The system observability analysis is of practical importance, for example, due to its ability to identify the unobservable directions of the estimated state which can influence estimation accuracy and help develop consistent and robust estimators. Recent studies focused on analyzing the observability of the state of various multisensor systems with a particular interest in unobservable directions induced by degenerate motions. However, those studies mostly stay in the specific sensor domain without aiding to extend the understanding to other heterogeneous systems. To this end, in this work, we provide degenerate motion analysis on general local and global sensor-paired systems, offering insights applicable to a wide range of existing navigation systems. Our analysis includes 9 degenerate motion identification including 5 already identified in literature and 4 new motions with both synchronous and asynchronous sensor-pair cases. Comprehensive numerical studies are conducted to verify those identified motions, show the effect of degenerate motion on state estimation, and demonstrate the generalizability of our analysis on various multisensor systems.

Manipulating deformable objects is a ubiquitous task in household environments, demanding adequate representation and accurate dynamics prediction due to the objects’ infinite degrees of freedom. This work proposes DeformNet, which utilizes latent space modeling with a learned 3D representation model to tackle these challenges effectively. The proposed representation model combines a PointNet encoder and a conditional neural radiance field (NeRF), facilitating a thorough acquisition of object deformations and variations in lighting conditions. To model the complex dynamics, we employ a recurrent state-space model (RSSM) that accurately predicts the transformation of the latent representation over time. Extensive simulation experiments with diverse objectives demonstrate the generalization capabilities of DeformNet for various deformable object manipulation tasks, even in the presence of previously unseen goals. Finally, we deploy DeformNet on an actual UR5 robotic arm to demonstrate its capability in real-world scenarios.

Shape servoing, a robotic task dedicated to controlling objects to desired goal shapes, is a promising approach to deformable object manipulation. An issue arises, however, with the reliance on the specification of a goal shape. This goal has been obtained either by a laborious domain knowledge engineering process or by manually manipulating the object into the desired shape and capturing the goal shape at that specific moment, both of which are impractical in various robotic applications. In this paper, we solve this problem by developing a novel neural network DefGoalNet, which learns deformable object goal shapes directly from a small number of human demonstrations. We demonstrate our method’s effectiveness on various robotic tasks, both in simulation and on a physical robot. Notably, in the surgical retraction task, even when trained with as few as 10 demonstrations, our method achieves a median success percentage of nearly 90%. These results mark a substantial advancement in enabling shape servoing methods to bring deformable object manipulation closer to practical real-world applications.

The complementarity between camera and LiDAR data makes fusion methods a promising approach to improve 3D semantic segmentation performance. Recent transformer-based methods have also demonstrated superiority in segmentation. However, multimodal solutions incorporating transformers are underexplored and face two key inherent difficulties: over-attention and noise from different modal data. To overcome these challenges, we propose a Deformable Multimodal Representation Fusion (DefFusion) framework consisting mainly of a Deformable Representation Fusion Transformer and Dynamic Representation Augmentation Modules. The Deformable Representation Fusion Transformer introduces the deformable mechanism in multimodal fusion, avoiding over-attention and improving efficiency by adaptively modeling a 2D key/value set for a given 3D query, thus enabling multimodal fusion with higher flexibility. To enhance the 2D representation and 3D representation, the Dynamic Representation Enhancement Module is proposed to dynamically remove noise in the input representation via Dynamic Grouped Representation Generation and Dynamic Mask Generation. Extensive experiments validate that our model achieves the best 3D semantic segmentation performance on SemanticKITTI and NuScenes benchmarks.