ICRA 2024 Papers

In this paper we present a fully distributed, asynchronous, and general purpose optimization algorithm for Consensus Simultaneous Localization and Mapping (CSLAM). Multi-robot teams require that agents have timely and accurate solutions to their state as well as the states of the other robots in the team. To optimize this solution we develop a CSLAM back-end based on Consensus ADMM called MESA (Manifold, Edge-based, Separable ADMM). MESA is fully distributed to tolerate failures of individual robots, asynchronous to tolerate communication delays and outages, and general purpose to handle any CSLAM problem formulation. We demonstrate that MESA exhibits superior convergence rates and accuracy compare to existing state-of-the art CSLAM back-end optimizers.

The multi-robot visibility-based pursuit-evasion problem tasks a team of robots with systematically searching an environment to detect (capture) an evader. Previous techniques to generate search strategies for the pursuit team have shown to be either computationally intractable or permit poor solution quality. This paper presents a novel asymptotically optimal algorithm for generating a joint motion strategy for the pursuers. To explore the space of possible pursuer motion strategies, the algorithm utilizes a trio of hierarchical graph data structures that each capture certain elements of the problem such as connectivity (valid single pursuer motion), coordination (multiple pursuer motion), and tracking information (evaluating where an evader may be). The algorithm is inspired by well-known methods in the motion planning literature and inherits its asymptotic optimality from those techniques. In addition, we describe a method that can improve upon solutions found during the formative stages of the main algorithm, using a "fast-forward" approach that foregoes guarantees of asymptotic optimality, implementing heuristics that concentrate future samples into improving the path quality of the nominal solution. The algorithms were validated in simulation and results are provided.

Socially assistive robots are potentially to be integrated with human daily lives in the near future, and expected to be able to improve group dynamics when interacting with groups of people in social settings. In this paper, we developed a system with desktop robot Haru to assist group discussions. The system consists of three modules: a dialogue assistance module which facilitates Haru to speak to users and answer questions in a free way; a dialogue balance module to encourage participation of users in the discussion with verbal behaviors; an autonomous gazing behavior module trained via deep reinforcement learning in simulation and deployed on physical Haru in reality, which can show politeness during group discussion, e.g., gazing to the speaking member, looking to the middle when both members are talking or silent, looking at the least spoken person when encouraging her. Results of user study with 40 subjects show the significant effectiveness of our system in assisting group discussion.

Industrial exoskeletons are a potential solution for reducing work-related musculoskeletal disorders during carrying or lifting tasks. Having sensors, electrical/pneumatic actuators, and control systems, active exoskeletons present a more versatile control system because it is possible to select different assistive strategies based on the performed task. From this perspective, human-machine interaction is required to safely open basic exoskeleton domains to the user and provide an adaptable setup system. This article presents the assessment and benchmarking of the novel XoLab Natural Language Interface, a voice user interface for interaction and configuration of industrial active exoskeletons. The evaluation of the novel interface was performed by 17 participants who completed the setup and operational activities while wearing the XoTrunk exoskeleton. The benchmark consisted of a comparison of the presented device with previous adaptable interfaces for the exoskeleton: the user command interface and the monitor system interface. The results showed that although the novel interface demonstrated a considerable lag in the time response, it was more attractive, stimulating and novel than the standard one. However, the standard interface obtained favourable results over the user command interface and the voice interface perspicuity and efficiency.

When agents in a multi-robot team have limited knowledge about their relative performance, their teammates, or the environment, robots must observe individual performance variations and adapt accordingly. We propose robot reputation to assess the historical performance of agents and make future adaptations in a persistent coverage task. We consider a heterogeneous multi-robot team, where robots are equipped with different capabilities to serve discrete events in an environment. We utilize a heterogeneous coverage control approach to partition the space according to robot capabilities and the estimated probability density, such that the robot is responsible for serving the events in its assigned region. As the team serves events, we assign each robot a reputation, which is then used to adjust the size of a robot’s region, thus adjusting the amount of space a robot is responsible for serving. Our simulations show that using reputation to weigh the size of the Voronoi cells outperforms the case where we neglect reputation.

3D articulated objects are inherently challenging for manipulation due to the varied geometries and intricate functionalities associated with articulated objects. Point-level affordance, which predicts the per-point actionable score and thus proposes the best point to interact with, has demonstrated excellent performance and generalization capabilities in articulated object manipulation. However, a significant challenge remains: while previous works use perfect point cloud generated in simulation, the models cannot directly apply to the noisy point cloud in the real-world. To tackle this challenge, we leverage the property of real-world scanned point cloud that, the point cloud becomes less noisy when the camera is closer to the object. Therefore, we propose a novel coarse-to-fine affordance learning pipeline to mitigate the effect of point cloud noise in two stages. In the first stage, we learn the affordance on the noisy far point cloud which includes the whole object to propose the approximated place to manipulate. Then, we move the camera in front of the approximated place, scan a less noisy point cloud containing precise local geometries for manipulation, and learn affordance on such point cloud to propose fine-grained final actions. The proposed method is thoroughly evaluated both using large-scale simulated noisy point clouds mimicking real-world scans, and in the real world scenarios, with superiority over existing methods, demonstrating the effectiveness in tackling the noisy real-world point cloud problem.

We present ArrayBot, a distributed manipulation system consisting of a 16 × 16 array of vertically sliding pillars integrated with tactile sensors. Functionally, ArrayBot is designed to simultaneously support, perceive, and manipulate the tabletop objects. Towards generalizable distributed manipulation, we leverage reinforcement learning (RL) algorithms for the automatic discovery of control policies. In the face of the massively redundant actions, we propose to reshape the action space by considering the spatially local action patch and the low-frequency actions in the frequency domain. With this reshaped action space, we train RL agents that can relocate diverse objects through tactile observations only. Intriguingly, we find that the discovered policy can not only generalize to unseen object shapes in the simulator but also have the ability to transfer to the physical robot without any sim-to-real fine-tuning. |Leveraging the deployed policy, we derive more real-world manipulation skills on ArrayBot to further illustrate the distinctive merits of our proposed system.

The tremendous hype around autonomous driving is eagerly calling for emerging and novel technologies to support advanced mobility use cases. As car manufactures keep developing SAE level 3+ systems to improve the safety and comfort of passengers, traffic authorities need to establish new procedures to manage the transition from human-driven to fully-autonomous vehicles while providing a feedback-loop mechanism to fine-tune envisioned autonomous systems. Thus, a way to automatically profile autonomous vehicles and differentiate those from human-driven ones is a must.In this paper, we present a fully-fledged framework that monitors active vehicles using camera images and state information in order to determine whether vehicles are autonomous, without requiring any active notification from the vehicles themselves. Essentially, it builds on the cooperation among vehicles, which share their data acquired on the road feeding a machine learning model to identify autonomous cars. We extensively tested our solution and created the NexusStreet dataset, by means of the CARLA simulator, employing an autonomous driving control agent and a steering wheel maneuvered by licensed drivers. Experiments show it is possible to discriminate the two behaviors by analyzing video clips with an accuracy of ~ 80%, which improves up to ~ 93% when the target’s state information is available. Lastly, we deliberately degraded the state to observe how the framework performs under non-ideal data collection conditions.

Many computations in robotics can be dramatically accelerated if the robot configuration space is described as a collection of simple sets. For example, recently developed motion planners rely on a convex decomposition of the free space to design collision-free trajectories using fast convex optimization. In this work, we present an efficient method for approximately covering complex configuration spaces with a small number of polytopes. The approach constructs a visibility graph using sampling and generates a clique cover of this graph to find clusters of samples that have mutual line of sight. These clusters are then inflated into large, full-dimensional, polytopes. We evaluate our method on a variety of robotic systems and show that it consistently covers larger portions of free configuration space, with fewer polytopes, and in a fraction of the time compared to previous methods.

In this paper, we focus on the autonomous multiagent taxi routing problem for a large urban environment where the location and number of future ride requests are unknown a-priori, but can be estimated by an empirical distribution. Recent theory has shown that a rollout algorithm with a stable base policy produces a near-optimal stable policy. In the routing setting, a policy is stable if its execution keeps the number of outstanding requests uniformly bounded over time. Although, rollout-based approaches are well-suited for learning cooperative multiagent policies with considerations for future demand, applying such methods to a large urban environment can be computationally expensive due to the large number of taxis required for stability. In this paper, we aim to address the computational bottleneck of multiagent rollout by proposing an approximate multiagent rollout-based two phase algorithm that reduces computational costs, while still achieving a stable near-optimal policy. Our approach partitions the graph into sectors based on the predicted demand and the maximum number of taxis that can run sequentially given the user’s computational resources. The algorithm then applies instantaneous assignment (IA) for re-balancing taxis across sectors and a sector-wide multiagent rollout algorithm that is executed in parallel for each sector. We provide two main theoretical results: 1) characterize the number of taxis m that is sufficient for IA to be stable; 2) derive a necessary condition on m to maintain stability for IA as time goes to infinity. Our numerical results show that our approach achieves stability for an m that satisfies the theoretical conditions. We also empirically demonstrate that our proposed two phase algorithm has equivalent performance to the one-at-a-time rollout over the entire map, but with significantly lower runtimes.

Towards flexible object-centric visual perception, we propose a one-shot instance-aware object keypoint (OKP) extraction approach, AnyOKP, which leverages the powerful representation ability of pretrained vision transformer (ViT), and can obtain keypoints on multiple object instances of arbitrary category after learning from a support image. An off-the-shelf petrained ViT is directly deployed for generalizable and transferable feature extraction, which is followed by training-free feature enhancement. The best-prototype pairs (BPPs) are searched for in support and query images based on appearance similarity, to yield instance-unaware candidate keypoints. Then, the entire graph with all candidate keypoints as vertices are divided into sub-graphs according to the feature distributions on the graph edges. Finally, each sub-graph represents an object instance. AnyOKP is evaluated on real object images collected with the cameras of a robot arm, a mobile robot, and a surgical robot, which not only demonstrates the cross-category flexibility and instance awareness, but also show remarkable robustness to domain shift and viewpoint change.

Assistive agents performing household tasks such as making the bed or cooking breakfast often compute and execute actions that accomplish one task at a time. However, efficiency can be improved by anticipating upcoming tasks and computing an action sequence that jointly achieves these tasks. State-of-the-art methods for task anticipation use data-driven deep networks and Large Language Models (LLMs), but they do so at the level of high-level tasks and/or require many training examples. Our framework leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as goals in a classical planning system to compute a sequence of finer-granularity actions that jointly achieve these goals. We ground and evaluate our framework’s abilities in realistic scenarios in the VirtualHome environment and demonstrate a 31% reduction in execution time compared with a system that does not consider upcoming tasks.

Ankle exoskeletons can assist the ankle joint and reduce the metabolic cost of walking. However, many existing ankle exoskeletons constrain the natural 3 degrees of freedom (DoF) of the ankle to limit the exoskeleton’s weight and mechanical complexity, thereby compromising comfort and kinematic compatibility with the user.This paper presents a novel ankle exoskeleton frame design that allows for 3 DoF ankle motion using a symmetric parallel frame design principle resulting in a strong frame while weighing 1.8 kg. Furthermore, a cable routing method is proposed to actuate the plantarflexion of the ankle. The kinematic compatibility of the proposed exoskeleton frame is evaluated in straight- and curve-walking scenarios with four users. The study demonstrates that the exoskeleton frame adapts to the natural 3 DoF ankle motion and the range of motion (RoM) during walking. The actuation in plantarflexion is evaluated in a stationary torque experiment demonstrating the ability of the frame to transfer large torque loads of up to 57.4 Nm. This work contributes to the design and development of more flexible and adaptable ankle exoskeletons for walking assistance.

Sidewinding, a locomotion strategy characterized by the coordination of lateral and vertical body undulations, is frequently observed in rattlesnakes and has been successfully implemented by limbless robotic systems for effective movement across diverse terrestrial terrains. However, the integration of compliant mechanisms into sidewinding limbless robots remains less explored, posing challenges for navigation in complex, rheologically diverse environments. Inspired by a notable control simplification via mechanical intelligence in lateral undulation [1], which offloads feedback control to passive body mechanics and interactions with the environment, we present an innovative design of a mechanically intelligent limbless robot for sidewinding. This robot features a decentralized bilateral cable actuation system that resembles organismal muscle actuation mechanisms. We develop a feedforward controller that incorporates programmable body compliance into the sidewinding gait template. Our experimental results highlight the emergence of mechanical intelligence when the robot is equipped with an appropriate level of body compliance. This allows the robot to 1) locomote more energetically efficiently, as evidenced by a reduced cost of transport, and 2) navigate through terrain heterogeneities, all achieved in an open-loop manner, without the need for environmental awareness.

Developing autonomous intervention capabilities for lightweight underwater vehicle manipulator systems (UVMS) has garnered significant attention within recent years because of the opportunity for these systems to reduce intervention operating costs. Developing autonomous UVMS capabilities is challenging, however, because of the lack of available standardized software frameworks and pipelines. Previous works offer simulation environments and deployment pipelines for underwater vehicles, but fall short of providing a complete UVMS software framework. We address this gap by creating Angler: a software framework for developing localization, control, and decision-making algorithms with support for sim-to-real transfer. We validate this framework by implementing a state-of-the-art control architecture and demonstrate the ability to perform station keeping with a mean error below 0.25 m and waypoint tracking with an average final error of 0.398 m.

Several robotic frameworks have been recently developed to assist ophthalmic surgeons in performing complex vitreoretinal procedures such as subretinal injection. However, in order to intuitively integrate robots into the surgical workflow, it is crucial to emphasize that an accessibility analysis framework for vitreoretinal surgery must be considered as an essential component. Such a framework, ideally, considers the comprehensive factors of the eye anatomy and its positioning, the insertion point, and the initial pose and position of the robot. By combining the mobilization of the eyeball and adjusting the pose and position of the robot, the accessibility of such systems is significantly optimized. At the same time, the accessible-visible area is better and faster matched to the working volume of the robot. This paper presents an analysis of an expansion strategy for the robot’s accessibility and visibility area. The outcomes of this method demonstrate the promising potential to enhance the robot’s accessibility, as evidenced in our analytical and experimental findings from 22.4% to 99.0% of the required working area on an adjustable phantom model.

The cable-driven redundant manipulator (CDRM) has significant potential for applications in narrow and hazardous spaces. However, traditional CDRMs have limited stiffness and load capacity due to their cable routing method. To address these limitations, several scholars have proposed new mechanisms and control strategies. Nevertheless, the cable routing method has not changed, and CDRMs continue to suffer from their limitations. Recently, a nematode-inspired cable routing method was proposed; however, stiffness calculations, derivation of inverse kinematics, and validation of stiffness and load capacity were incomplete. In this paper, we calculate the analytic equivalent stiffness of the nematode-inspired cable routing method and compare it with other cable routing methods. Additionally, we derived and simulate the kinematics and an effective inverse kinematics algorithm. Finally, we validate the stiffness and load capacity using a developed prototype.

Offline learning usually requires a large dataset for training. In this paper, we focus on vision-based robotic manipulation tasks and utilize certain task properties to achieve offline learning with a small dataset. We propose a two-stage agent consisting of a tentative decision stage and a correction stage, where the tentative decision stage determines a tentative action from the original camera image, and the correction stage determines a correction to the tentative action based on the cropped image according to the tentative action. The correction stage utilizes task properties to obtain the cropped image with task-relevant features, enabling efficient correction. In particular, the training of the two stages can be performed individually, which enables a straightforward application of general offline learning algorithms. We conduct experiments by combining the two-stage agent with conventional offline reinforcement learning and imitation learning algorithms. In both cases, we benchmark the proposed method using RLBench and demonstrate that the task performance is significantly improved by the correction stage.

This paper presents a method to compute smooth ankle trajectories for lower limb exoskeletons with powered ankle joints. The proposed approach defines ankle trajectories using four polynomial functions, each representing one of the four primary phases of gait. These polynomials are computed according to different safety constraints. During the single support phase, ground contact constraints are enforced. In the swing phase, an optimization problem is solved to achieve minimum jerk planning while respecting a set of equality and inequality constraints designed to minimize the risk of stumbling. The used approach focuses on making the ankle joint able to smoothly adapt in real-time to different walking styles defined by user-selected gait parameters such as step length and clearance. The primary aim is to improve the user experience by producing a secure and comfortable walking pattern. To validate the effectiveness of the proposed method, the new ankle trajectories were tested on a group of healthy volunteers using the TWIN lower limb exoskeleton.