ICRA 2024 Papers

Introducing robotics in the healthcare sector revolutionizes medical services by providing advanced treatments, medication management, and robotic assistance while overcoming resource limitations. In the current healthcare domain, an intermediate robotic communication platform is essential for distributing equal medical services, facilitating remote consultations, and maintaining the integrity of medical education, especially in rural areas and during pandemics. This work introduces ARIS, a multitasking medical service robot designed for telemedicine aspects and to facilitate remote medical education activities such as ward rounds. The prototype called ARIS 1.0 was developed, including a three-wheeled omnidirectional mobile platform, a torso and a novel movable neck mechanism with a face. The prototype robot can generate an online summarized report using its integrated language interaction and IoT-based vital sign extraction modules. The ROS-based semi-autonomous navigation facilitates the robot to be an assistive agent, allowing it to either accompany doctors or visit patients individually. Ultimately, ARIS 1.0 serves telepresence and novel regional language capabilities, specifically Sinhala-based self-communication features. This enables inter-party communication among doctors, medical students, and patients. The functionalities of ARIS 1.0 were validated in an emulated indoor environment to evaluate their feasibility. The results indicate that ARIS 1.0 is feasible for providing remote medical services. Furthermore, the paper discusses several promising research directions related to the proposed concept.

Various perception-aware planning approaches have attempted to enhance the state estimation accuracy during maneuvers, while the feature matchability among frames, a crucial factor influencing estimation accuracy, has often been overlooked. In this paper, we present APACE, an Agile and Perception-Aware trajeCtory gEneration framework for quadrotors aggressive flight, that takes into account feature matchability during trajectory planning. We seek to generate a perception-aware trajectory that reduces the error of visual-based estimator while satisfying the constraints on smoothness, safety, agility and the quadrotor dynamics. The perception objective is achieved by maximizing the number of covisible features while ensuring small enough parallax angles. Additionally, we propose a differentiable and accurate visibility model that allows decomposition of the trajectory planning problem for efficient optimization resolution. Through validations conducted in both a photorealistic simulator and real-world experiments, we demonstrate that the trajectories generated by our method significantly improve state estimation accuracy, with root mean square error (RMSE) reduced by up to an order of magnitude. The source code will be released to benefit the community 1.

Quadrotor motion planning in complex environments leverage the concept of safe flight corridor (SFC) to facilitate static obstacle avoidance. Typically, SFCs are constructed through convex decomposition of the environment’s free space into cuboids, convex polyhedra, or spheres. However, such SFCs can be overly conservative when dealing with a quadrotor swarm, substantially limiting the available free space for quadrotors to coordinate. This paper presents an Alternating Minimization-based approach that does not require building a conservative free-space approximation. Instead, both static and dynamic collision constraints are treated in a unified manner. Dynamic collisions are handled based on shared position trajectories of the quadrotors. Static obstacle avoidance is coupled with distance queries from the Octomap, providing an implicit non-convex decomposition of free space. As a result, our approach is scalable to arbitrary complex environments. Through extensive comparisons in simulation, we demonstrate a 60% improvement in success rate, an average 1.8× reduction in mission completion time, and an average 23× reduction in per-agent computation time compared to SFC-based approaches. We also experimentally validated our approach using a Crazyflie quadrotor swarm of up to 12 quadrotors in obstacle-rich environments. The code, supplementary materials, and videos are released for reference.

The multi-agent pathfinding (MAPF) problem seeks collision-free paths for a team of agents from their current positions to their pre-set goals in a known environment, and is an essential problem found at the core of many logistics, transportation, and general robotics applications. Existing learning-based MAPF approaches typically only let each agent make decisions based on a limited field-of-view (FOV) around its position, as a natural means to fix the input dimensions of its policy network. However, this often makes policies shortsighted, since agents lack the ability to perceive and plan for obstacles/agents beyond their FOV. To address this challenge, we propose ALPHA, a new framework combining the use of ground truth proximal (local) information and fuzzy distal (global) information to let agents sequence local decisions based on the full current state of the system, and avoid such myopicity. We further allow agents to make short-term predictions about each others’ paths, as a means to reason about each others’ path intentions, thereby enhancing the level of cooperation among agents at the whole system level. Our neural structure relies on a Graph Transformer architecture to allow agents to selectively combine these different sources of information and reason about their inter-dependencies at different spatial scales. Our simulation experiments demonstrate that ALPHA outperforms both globally-guided MAPF solvers and communication-learning based ones, showcasing its potential towards scalability in realistic deployments.

To address the limitations inherent to conventional automated harvesting robots specifically their suboptimal success rates and risk of crop damage, we design a novel bot named AHPPEBot which is capable of autonomous harvesting based on crop phenotyping and pose estimation. Specifically, In phenotyping, the detection, association, and maturity estimation of tomato trusses and individual fruits are accomplished through a multi-task YOLOv5 model coupled with a detectionbased adaptive DBScan clustering algorithm. In pose estimation, we employ a deep learning model to predict seven semantic keypoints on the pedicel. These keypoints assist in the robot’s path planning, minimize target contact, and facilitate the use of our specialized end effector for harvesting. In autonomous tomato harvesting experiments conducted in commercial green-houses, our proposed robot achieved a harvesting success rate of 86.67%, with an average successful harvest time of 32.46 s, showcasing its continuous and robust harvesting capabilities. The result underscores the potential of harvesting robots to bridge the labor gap in agriculture.

The exceptional mobility and long endurance of air-ground robots are raising interest in their usage to navigate complex environments (e.g., forests and large buildings). However, such environments often contain occluded and unknown regions, and without accurate prediction of unobserved obstacles, the movement of the air-ground robot often suffers a sub-optimal trajectory under existing mapping-based and learning-based navigation methods. In this work, we present AGRNav, a novel framework designed to search for safe and energy-saving air-ground hybrid paths. AGRNav contains a lightweight semantic scene completion network (SCONet) with self-attention to enable accurate obstacle predictions by capturing contextual information and occlusion area features. The framework subsequently employs a query-based method for low-latency updates of prediction results to the grid map. Finally, based on the updated map, the hierarchical path planner efficiently searches for energy-saving paths for navigation. We validate AGRNav’s performance through benchmarks in both simulated and real-world environments, demonstrating its superiority over classical and state-of-the-art methods. The open-source code is available at https://github.com/jmwang0117/AGRNav.

In this paper, we present an approach for coverage path planning for a team of an energy-constrained Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV). Both the UAV and the UGV have predefined areas that they have to cover. The goal is to perform complete coverage by both robots while minimizing the coverage time. The UGV can also serve as a mobile recharging station. The UAV and UGV need to occasionally rendezvous for recharging. We propose a heuristic method to address this NP-Hard planning problem. Our approach involves initially determining coverage paths without factoring in energy constraints. Subsequently, we cluster segments of these paths and employ graph matching to assign UAV clusters to UGV clusters for efficient recharging management. We perform numerical analysis on real-world coverage applications and show that compared with a greedy approach our method reduces rendezvous overhead on average by 11.33%. We demonstrate proof-of-concept with a team of a VOXL m500 drone and a Clearpath Jackal ground vehicle, providing a complete system from the offline algorithm to the field execution.

Offline reinforcement learning has emerged as a promising technology by enhancing its practicality through the use of pre-collected large datasets. Despite its practical benefits, most algorithm development research in offline reinforcement learning still relies on game tasks with synthetic datasets. To address such limitations, this paper provides autonomous driving datasets and benchmarks for offline reinforcement learning research. We provide 19 datasets, including real-world human driver’s datasets, and seven popular offline reinforcement learning algorithms in three realistic driving scenarios. We also provide a unified decision-making process model that can operate effectively across different scenarios, serving as a reference framework in algorithm design. Our research lays the groundwork for further collaborations in the community to explore practical aspects of existing reinforcement learning methods. Dataset and codes can be found in https://sites.google.com/view/ad4rl.

Prosthetic legs have been used to restore function in the lower limbs lost due to amputation. Early designs including prosthetic legs with a passive joint or without any joint as well as the Energy Storing and Releasing (ESR) feet have shown deficiency in push-off torque, which results in asymmetric gait pattern, slower walking speed, and higher cost of transportation. Although powered prosthetic legs address the aforementioned problems, they suffer from lower energy efficiency, higher volume and weight. In this paper, a powered transtibial prosthesis using a Parallel Elastic Actuator (PEA) is proposed in order to generate the joint torque needed for walking with a lower-powered actuator for lighter and more compact design. A non-linear spring mechanism is proposed to generate the spring torque as needed. The implemented prosthetic leg is evaluated with three intact subjects. The experimental results shows that smaller torque is required for the motor with the spring mechanism. Therefore, less electrical power is consumed when the spring mechanism is used, which implies a lower-powered actuator is sufficient to generate the joint torque needed for walking.

This paper proposes a novel form of a three-dimensional coaxial bowtie-shaped mechanical amplifier. The proposed model incorporates a lever mechanism into the Sarrus linkage structure. It allows the target plate to move along one axis with amplified displacement in a parallel manner. The amplifier was assembled after machining the components using a computer numerical control machine. A flexible hinge was incorporated into the amplifier design for simplified fabrication and reduced friction in the actuation mechanism. Castigliano’s theorem is used to build a mathematical model of the proposed mechanical amplifier, and the performance was validated through finite element analysis and prototype fabrication. We achieved the amplification ratio of ×8.44, resulting in the axial displacement up to 86 µm. The demonstrated amplifier is expected to apply to compact microsurgical robots or biomedical imaging apparatus requiring coaxial displacement amplification in confined spaces.

In this paper, we present a fully soft miniaturized continuum robot that integrates 3D optical shape sensing through functionalized tubing used as soft optical waveguides. The sensor is fabricated by laser patterning an off-the-shelf medical tubing, allowing for bidirectional responses to large curvatures in two bending directions, enabling 3D shape sensing and tip tracking of the continuum robot. The robot is able to bend and sense its own shape up to a curvature of 44.7 m-1, corresponding to a bending angle of 102°, having high-accuracy tracking capabilities, resulting in an average tracking error of 3.08 mm, that is 7.7 % of the robot length. The robot’s functionality was shown in validation experiments, including a real-time shape prediction through a graphical user interface.

Research in the area of photo-actuation is growing rapidly, yet there are few examples of photo-actuators with practical use cases. One potential application is for the control of intelligent electromagnetic surfaces, or two-dimensional arrays that could shape and control an incident electromagnetic field in ideally any manner. A promising concept to realize such a surface leverages signal refraction via antenna edges, but requires non-metal actuation, large antenna rotations, and high antenna angular accuracy for long periods of time. Here, we present a nonmetal, light-controlled, multi-position inchworm actuator array that can rotate an antenna 88 degrees in incremental steps of less than 3.4 degrees with zero-power shape-persistence. The design is modular and rapidly manufacturable via a layered laser-cutting technique, such that the actuator can be tiled into an array to control the rotation of many antennas. We control the array with a single focused IR light that rasters across the actuators to precisely control all antenna positions. We characterize the response time, accuracy, and repeatability of a single actuator, and demonstrate the array achieving diverse antenna configurations. This work advances the precision and scalability of photothermal actuation not only for use in intelligent electromagnetic surfaces but for any application benefitting from light-controlled actuation.

Variable Stiffness Mechanisms (VSM) are becoming ubiquitous in mechatronics given the benefit they provide in terms of safety and performance. Despite these assets, VSMs remain fairly complex mechanical devices lacking in compactness, ease of manufacturing and accessibility. In addition, the scarcity of commercially available VSMs requires that such systems are mostly designed in-house. We propose a new type of VSM that improves on the pre-existing Jack Spring concept by making it more compact and robust. The new concept, which we refer to as the Compact Modifier of Active Coils (C-MAC) mechanism, is specifically designed to be manufactured through a monolithic 3D print. This approach enables to modify a minimal set of design features, namely the spring diameter and the coil diameter, to achieve the desired range of stiffness variation. We test the proposed design on six configurations; these show hysteretic energy losses no larger than 35% over the stiffness variation and confirm stiffness to scale according to theory. Stiffness ranging from 0.15 N/mm to 1.02N/mm were measured for an overall device length of 140 mm, including a maximal stroke length of 22 mm. The results confirm excellent scalability and manufacturability of the proposed design, providing a versatile mechanism for fast prototyping and the development of entire 3D printed robotic systems embedding variable stiffness capabilities.

Assistive robots promise to be of great help to wheelchair users with motor impairments, for example for activities of daily living. Using shared control to provide task-specific assistance – for instance with the Shared Control Templates (SCT) framework – facilitates user control, even with low-dimensional input signals. However, designing SCTs is a laborious task requiring robotic expertise. To facilitate their design, we propose a method to learn one of their core components – active constraints – from demonstrated end-effector trajectories. We use a probabilistic model, Kernelized Movement Primitives, which additionally allows adaptation from user commands to improve the shared control skills, during both design and execution. We demonstrate that the SCTs so acquired can be successfully used to pick up an object, as well as adjusted for new environmental constraints, with our assistive robot EDAN.

Unmanned aerial vehicles (UAVs) are becoming an integral part of both industry and society. In particular, the quadrotor is now invaluable across a plethora of fields and recent developments, such as the inclusion of aerial manipulators, only extends their versatility. As UAVs become more widespread, preventing loss-of-control (LOC) is an ever growing concern. Unfortunately, LOC is not clearly defined for quadrotors, or indeed, many other autonomous systems. Moreover, any existing definitions are often incomplete and restrictive. A novel metric, based on actuator capabilities, is introduced to detect LOC in quadrotors. The potential of this metric for LOC detection is demonstrated through both simulated and real quadrotor flight data. It is able to detect LOC induced by actuator faults without explicit knowledge of the occurrence and nature of the failure. The proposed metric is also sensitive enough to detect LOC in more nuanced cases, where the quadrotor remains undamaged but nevertheless losses control through an aggressive yawing manoeuvre. As the metric depends only on system and actuator models, it is sufficiently general to be applied to other systems.

Ensuring safety in autonomous systems is essential as they become more integrated with modern society. One way to accomplish this is to identify and maintain a safe operating space. To this end, much effort has been devoted in the field of reachability analysis to obtaining control-invariant sets which ensure that a system inside of these sets can remain in these sets, and are thus essential for guaranteeing a system’s safety. However, control invariance does not imply that a system can move from any state in the control-invariant set to any other state in the control-invariant set, within a given time horizon. In this paper, we develop an algorithm to obtain a control-invariant set that allows a given system to move from any state in the set to any other state in the set within a given time horizon without having to leave the set. We call this the ‘maneuver set’, \mathcal{M}\mathcal{M}. We substantiate the algorithm’s efficacy through mathematical proof, affirming that the maneuver set obtained through the algorithm is indeed control-invariant. Furthermore, we prove that the system is indeed able to move from any state within this set to any other state in the set. To illustrate the use of our algorithm, we provide the numerical example of a Dubins car, utilising Hamilton-Jacobi-Bellman reachability analysis along with the proposed algorithm in order to obtain \mathcal{M}\mathcal{M}.

A pneumatic linear actuator is presented and evaluated. Designed to operate in demanding environments such as MRI, it is developed to be used with two motion control modes: 1) a step-by-step mode with tooth-based gripping to ensure precision, 2) a continuous mode available locally for fine positioning. The actuator can also be disengaged to enable direct handling by an operator, for example for comanipulation. The design is presented. A prototype, developed in the medical context, is implemented and characterized. A specific step-by-step control sequence is then elaborated based on its characterization. Testing of the dual-mode actuation is finally described. The complementarity between the two motion modes and possible adaptations of the original design are discussed.

Space-filling building blocks of diverse shape permeate nature at all levels of organization, from atoms to honeycombs, and have proven useful in artificial systems, from molecular containers to clay bricks. But, despite the wide variety of space-filling polyhedra known to mathematics, only the cube has been explored in robotics. Thus, here we roboticize a non-cubic space-filling shape: the rhombic dodecahedron. This geometry offers an appealing alternative to cubes as it greatly simplifies rotational motion of one cell about the edge of another, and increases the number of neighbors each cell can communicate with and hold on to. To better understand the challenges and opportunities of these and other space-filling machines, we manufactured 48 rhombic dodecahedral cells and used them to build various superstructures. We report locomotive ability of some of the structures we built, and discuss the dis/advantages of the different designs we tested. We also introduce a strategy for genderless passive docking of cells that generalizes to any polyhedra with radially symmetrical faces. Future work will allow the cells to freely roll/rotate about one another so that they may realize the full potential of their unique shape.

Dexterous manipulation through imitation learning has gained significant attention in robotics research. The collection of high-quality expert data holds paramount importance when using imitation learning. The existing approaches for acquiring expert data commonly involve utilizing a data glove to capture hand motion information. However, this method suffers from limitations as the collected information cannot be directly mapped to the robotic hand due to discrepancies in their degrees of freedom or structures. Furthermore, it fails to accurately capture force feedback information between the hand and objects during the demonstration process. To overcome these challenges, this paper presents a novel solution in the form of a wearable dexterous hand, namely Handover-hand Imitation learning wearable RObotic Hand (HIRO Hand), which integrates expert data collection and enables the implementation of dexterous operations. This HIRO Hand empowers the operator to utilize their own tactile feedback to determine appropriate force, position, and actions, resulting in more accurate imitation of the expert’s actions. We develop both non-learning and visual behavior cloning based controllers allowing HIRO Hand successfully achieves grasping and in-hand manipulation ability.