ICRA 2024 Papers

In this paper, a novel mathematical framework is introduced for modeling deformation behavior of Tendon-Driven Continuum Manipulators (TD-CMs) featuring discontinuous cross-sectional geometries (i.e., having flexural patterns). Leveraging this framework, we also introduce the concept of design space by which the deformation-behavior space of a TD-CM can intuitively be analyzed via its geometrical design parameters. To thoroughly evaluate the performance of the proposed modeling framework, we have conducted various simulation studies and experiments.

Preference-based reward learning is a popular technique for teaching robots and autonomous systems how a human user wants them to perform a task. Previous works have shown that actively synthesizing preference queries to maximize information gain about the reward function parameters improves data efficiency. The information gain criterion focuses on precisely identifying all parameters of the reward function. This can potentially be wasteful as many parameters may result in the same reward, and many rewards may result in the same behavior in the downstream tasks. Instead, we show that it is possible to optimize for learning the reward function up to a behavioral equivalence class, such as inducing the same ranking over behaviors, distribution over choices, or other related definitions of what makes two rewards similar. We introduce a tractable framework that can capture such definitions of similarity. Our experiments in a synthetic environment, an assistive robotics environment with domain transfer, and a natural language processing problem with real datasets demonstrate the superior performance of our querying method over the state-of-the-art information gain method.

For autonomous mobile robots, uncertainties in the environment and system model can lead to failure in the motion planning pipeline, resulting in potential collisions. In order to achieve a high level of robust autonomy, these robots should be able to proactively predict and recover from such failures. To this end, we propose a Gaussian Process (GP) based model for proactively detecting the risk of future motion planning failure. When this risk exceeds a certain threshold, a recovery behavior is triggered that leverages the same GP model to find a safe state from which the robot may continue towards the goal. The proposed approach is trained in simulation only and can generalize to real world environments on different robotic platforms. Simulations and physical experiments demonstrate that our framework is capable of both predicting planner failures and recovering the robot to states where planner success is likely, all while producing agile motion.

In complex unstructured environments, accurate terrain traversability analysis is a fundamental requirement for the successful execution of any movements of ground robots, especially given that terrain traversability often exhibits anisotropy. However, the difficulty in obtaining multi-directional terrain labels hinders the emergence of end-to-end multi-directional traversability network. This paper introduces a framework for real-time multi-directional traversability maps (MTraMap) generation tailored for unstructured environments. It involves pre-training a uni-directional traversability classifier, termed UniTraT, through self-supervised learning using ground robot travel simulation. Furthermore, it employs Uni-directional to Multi-directional Traversability Distillation (UMTraDistill) to distill a multi-directional traversability network, termed MultiTCNN, which is capable of directly generating MTraMap. We evaluated both networks on our traversability dataset, achieving an 89% accuracy in terrain traversability classification with the UniTraT. Compared to UniTraT, the accuracy of the MultiTCNN distilled via UMTraDistill only decreases by 1.8%, and it can process 10 m × 10 m elevation map at a speed of 74 fps. Field robotics experiments were also conducted and showed that MultiTCNN can generate MTraMap of the surrounding 20 m × 20 m environment at a rate of 9.39 fps, with a slight reduction of 0.61 fps compared to the lidar data publishing rate, and the generated MTraMap can clearly delineate the multi-directional traversability of the surrounding environments.

Laparoscope-holding robots significantly enhance the stability and precision of visualization in minimally invasive surgeries. Most existing robots of this kind depend on visual servo systems and struggle with efficient, rapid adjustments in the field-of-view (FOV), especially when identifying organs and needles outside the FOV. This paper presents a laparoscope-holding robot system capable of employing both vision-driven and force-driven mechanisms for continuous and large-scale FOV adjustments, respectively. The system features an integrated tactile handle, enabling the reception of human-robot interaction forces during surgical navigation. We propose a hybrid control method that leverages both force and vision inputs for laparoscopic FOV adjustments. This approach integrates a virtual wrench, generated from visual information, and an interaction wrench, obtained from the tactile handle, into the robot's dynamic model, which complies with remote center of motion constraints. The interaction wrench's gain is adjusted with the gripping force on the integrated tactile handle, ensuring that unintended movements caused by accidental contacts are prevented, thus safeguarding operational safety. The proposed method eliminates the need to switch control modes, enabling simultaneous visual tracking and tactile interaction guidance. Experimental results demonstrate that the proposed method not only allows for FOV adjustments with surgical instrument guiding but also adapts well to large-scale FOV adjustment tasks.

Various sensorized grippers have been developed to handle delicate objects safely. These grippers have sensors mounted on their fingers’ surface that provide direct force measurements. However, multiple sensors are often required on one finger, leading to significant sensor placement and wire routing complexity. Finger-based sensors are limited to sensing external gripping force, and fingers cannot be easily replaced to meet the requirements of objects with specific geometries. To overcome the complexity and limitations of finger surface sensors, this paper proposes a force-controlled two-fingered gripper that relies on the deformation sensing of elastic elements in the drivetrain to obtain finger force. By using a minimum number of optical encoders placed in the drivetrain, accurate position and force sensing can be achieved at any location of each finger. When gripping an object, the size and stiffness of the object can thus be accurately measured. Simulation and experimental results demonstrate the proposed gripper’s merits. We expect this new gripper to provide a more competitive solution for robots that need to manipulate objects and check their mechanical qualities at the same time.

Navigating biomedical instruments inside the brain remains challenging and high-risk. The delicate nature of the tissues involved requires the development of cutting-edge robotic technologies to enhance precision and safety. In response to these demands, this paper presents a novel ribbon-shaped, tendon-driven continuum microrobot designed explicitly to navigate through brain tissues. The microrobot has a cross-sectional area of 1 mm2, and its design is readily compatible with conventional microfabrication techniques for further miniaturization. The flat geometry aims to provide superior maneuverability and opens up new challenges for modeling and control. We detail the design methodology and fabrication, followed by in vitro characterization and testing within brain tissue phantoms.

Human following for mobile robots has emerged as a promising technique with widespread applications. To ensure psychological comfort while collaborating, coexisting, and interacting with humans, robots need to respect the social space of the target person. In this study, we propose a dual closed-loop human-following control strategy that combines model predictive control (MPC) and impedance control. The outer-loop MPC ensures precise control of the robot’s posture while tracking the target person’s velocity and direction to coordinate the motion between them. The inner-loop impedance controller is employed to regulate the robot’s motion and interaction force with the target person, enabling the robot to maintain a respectful and comfortable distance from the target person. Concretely, the social interaction dynamics characteristics between the robot and the target person are described by human-robot interaction dynamics, which considers the rules of social space. Furthermore, an obstacle avoidance component constructed using behavioral dynamics is integrated into the impedance controller. Experimental results demonstrate the effectiveness of the proposed method in achieving human following and obstacle avoidance without intruding into the intimate zone of the target person.

Point cloud compression is an essential technology for efficient storage and transmission of 3D data. Previous methods usually use hierarchical tree data structures for encoding the spatial sparseness of point clouds. However, the node context within the tree is not fully discovered since the feature space among nodes varies significantly. To address this problem, we innovatively represent the LiDAR points in a two-octree structure instead of using traditional single-octree coding, and then design the cross-attention model to capture the hierarchical features between different octrees, of which each octree incorporates a transformer-based deep entropy model and an arithmetic encoder. Besides, we introduce the untied cross-aware position encoding with principal component analysis and different projection matrices, which enhances the correlations over two octrees’ attention feature embeddings. Experimental results show that our method outperforms the previous state-of-the-art works, achieving up to 8.2% Bpp savings on point cloud benchmark datasets with different lasers.

Dragonflies show impressive flying skills by achieving both high efficiency and agility. They can perform distinctive flight maneuvers, such as flying backwards, which has proven to be achieved through "force vectoring" mechanism recently. In this paper, to explore the agile flight ability of dragonflies on man-made flapping wing systems, we designed, optimized and fabricated a dragonfly-inspired flapping wing robot (DFWR) with inclinable stroke plane control degrees. The proposed platform employs a four-wing configuration, each of which integrates an extra servo motor to enable the rotation of the flapping plane and imitate the "force vectoring" mechanism. Besides, referring to the flapping kinematics of dragonflies, the installation angle and wing pitch angle of the proposed DFWR are optimized considering the total lift and energy consumption through multiobjective optimization based on NSGA-II method. The "force vector" produced by the proposed platform has been illustrated through both theoretical method and experimental method. Moreover, the feasibility of the design is further verified through a series of operation validation experiments. Such a robot has the potential to provide a highly biomimetic platform to validate the flight mechanism studying of Odonata as well as the relative on-board applications such as bio-inspired vision.

To enhance robots’ applicability in real-world scenarios, it is essential to establish a complex and multi-tasking behaviour, inspired by human nature. To this purpose, from a hardware perspective, a high number of degrees of freedom is necessary, as is the case for humanoids and collaborative mobile manipulators. From a software standpoint instead, complex hierarchical strategies are often used to define a set of behaviours that the robot should reflect in strict hierarchical order. Their main issue however, is related to the lack of continuity when their stack of tasks is changed. Existing works that address this issue clearly present a trade-off between optimality assurance during transition and computational costs. Here, we employ a distributed processing approach that enables not only the minimization of computational costs, but also continuous optimality and constraints feasibility even under sharp transitions. The approach is tested during three task transitions, for different tasks such as constrained trajectory tracking, obstacle avoidance, and postural optimization. Two mobile manipulators are used, each having 10 DoF, and the results confirm the smoothness of the generated solutions.

The distributed coordination of robot teams performing complex tasks is challenging to formulate. The different aspects of a complete task such as local planning for obstacle avoidance, global goal coordination and collaborative mapping are often solved separately, when clearly each of these should influence the others for the most efficient behaviour. In this paper we use the example application of distributed information acquisition as a robot team explores a large space to show that we can formulate the whole problem as a single factor graph with multiple connected layers representing each aspect. We use Gaussian Belief Propagation (GBP) as the inference mechanism, which permits parallel, on-demand or asynchronous computation for efficiency when different aspects are more or less important. This is the first time that a distributed GBP multi-robot solver has been proven to enable intelligent collaborative behaviour rather than just guiding robots to individual, selfish goals. We encourage the reader to view our demos at https://aalpatya.github.io/gbpstack.

Vertical farming, a sustainable key for urban agriculture, has garnered attention for its land use optimization and enhanced food production capabilities. The adoption of automation in vertical farming is a pivotal response to labor shortages, addressing the need for increased efficiency, particularly in labor-intensive tasks like harvesting. Although soft robotic grippers offer a significant promise for delicately handling fragile objects, the absence of sensors has hindered their full potential to execute precise and secure grasping. To address this challenge, we present a new solution: a detachable Fiber Bragg Grating-based flexible contact force sensor to capture gripper-vegetable interactions. The sensing module was 3D printed using soft material, and the FBG fiber was attached to the module using epoxy. From evaluation tests, this lightweight sensor demonstrated a wide measurement range of up to 9.87 N, with a high sensitivity of 141.7 pm/N, good repeatability, and a hysteresis of 7.96%. Compared to commercial load cells, our sensor achieves a small measurement RMSE of 0.41 N and a percentage error of 4.15%. The sensor was integrated into two robotic 3D-printed soft grippers to enable real-time monitoring of dynamic contact force during vegetable harvesting in vertical farming scenarios. By reflecting contact status, this sensor provides a promising glimpse into the future of agricultural automation, enhancing operational efficiency and strengthening situation awareness and decision-making capabilities in vertical farms. Beyond agriculture, the versatility of this sensor extends to application in areas such as warehousing, logistics, and the food and beverage industry.

The determination of the dynamic properties of a robot is especially important for designing highly accurate and efficient control systems. Conventional methods for dynamic model identification have proven to be effective, where deep learning (DL) approaches have shown limits due to data inefficiencies. However, thanks to novel physics-informed DL architectures, such as Deep Lagrangian Networks (DeLaN) [1], it is possible to control and extract interpretable physical information of a robot. This paper introduces an augmented DeLaN architecture for linear viscous and non-symmetrical Coulomb friction identification, which also learns motor parameters such as rotor inertia. An approach is proposed for comparing this method with the conventional dynamic identification and previous DeLaN implementations. Moreover, our friction and rotor inertia identification is validated, and the performance of our model is analyzed with a real robot (UR5e).

It is challenging to perform system identification on soft robots due to their underactuated, high-dimensional dynamics. In this work, we present a data-driven modeling framework, based on geometric mechanics (also known as gauge theory) that can be applied to systems with low-bandwidth control of the system’s internal configuration. This method constructs a series of connected models comprising actuator and locomotor dynamics based on data points from stochastically perturbed, repeated behaviors. By deriving these connected models from general formulations of dissipative Lagrangian systems with symmetry, we offer a method that can be applied broadly to robots with first-order, low-pass actuator dynamics, including swelling-driven actuators used in hydrogel crawlers. These models accurately capture the dynamics of the system shape and body movements of a simplified swimming robot model. We further apply our approach to a stimulus-responsive hydrogel simulator that captures the complexity of chemomechanical interactions that drive shape changes in biomedically relevant micromachines. Finally, we propose an approach of numerically optimizing control signals by iteratively refining models, which is applied to optimize the input waveform for the hydrogel crawler. This transfer to realistic environments provides promise for applications in locomotor design and biomedical engineering.

A Control Barrier Function (CBF)-based motion planning algorithm is proposed. The algorithm explores an unknown environment to reach a target point, providing velocity commands to the robot controller module. CBFs, along with a circulation inequality are used to generate safe paths toward the goal while preventing collisions with obstacles. The proposed global navigation scheme is experimentally verified on a quadruped platform to demonstrate safe, collision-free exploration over long distances.

Benefiting from its hyper-redundant structure, the biomimetic snake-like manipulator retains its remarkable flexibility even within confined spaces. However, its motion planning and control pose significant challenges. This paper imitates the winding uncoiling behavior of snakes to achieve controllable constrained path following. Firstly, based on control points, a recursive computational model and an equivalent planning angle model are established, enabling efficient and analytical determination of joint positions, collision regions, and motion parameters during the path following. Subsequently, the sliding control point algorithm and motion smoothing restriction algorithm are designed. The former ensures that the remaining segments during following strictly remain within the collision-free regions defined by the base and path controls, while the latter smooths the control parameters based on velocity and acceleration limitations. Finally, simulation and practical experiments demonstrate the feasibility of the proposed methods. The prototype that applied our method can reach targets and accomplish tasks, further validating the applicability of the snake-like manipulator.

In the rapidly evolving landscape of Unmanned Aerial Vehicles (UAVs), regulation enforcement is critical. Unfortunately, existing practices are largely manual and reactive in nature. We present Themis1, a novel compiler-directed approach for automated and proactive regulation enforcement. By expressing regulations through a specification language and integrating their enforcement into the compilation process, Themis enables safe and regulation-compliant UAV flights by enforcing prohibited and restricted areas, avoiding flights over humans, and managing maximum limits of altitude and speed. Our framework features a bidirectional interface that allows the concrete algorithms used for enforcement to be customized. Our evaluation shows Themis-compiled autopilots can adhere to regulatory constraints amidst complex flight conditions, while significantly reducing the burden of UAV operators.

Direct teaching is a straightforward way of teaching new motion to robots. Active methods with torque sensors, for example, can be used so that the robot can follow the movements of the human, but such methods introduce delays. Alternatively, series clutch actuators are easily backdrivable without delay. However, vertical joints are subject to gravity torques, which need to be compensated when disengaging the clutch. We implemented passive gravity compensation to counteract the robot’s weight, but this mechanism cannot compensate for varying payloads, as adjustable passive gravity compensation is relatively slow and mechanically complex. The varying payload causes an unintended joint movement, i.e. the arm falls down on its own, which is unacceptable during direct teaching. Therefore, this paper demonstrates how the torque output controlled with series clutch actuators can be used to compensate for varying payloads while maintaining high backdrivability. The proposed method is evaluated on a collaborative robot with a clutch in series for each actuator. Real-world experiments with payloads from 0 to 3 kg are conducted. During the experiments, the operator force is measured to evaluate the proposed method.