ICRA 2024 论文列表

There exists a capability gap in the design of currently available autonomous underwater vehicles (AUV). Most AUVs use a set of thrusters, and optionally control surfaces, to control their depth and pose. AUVs utilizing thrusters can be highly maneuverable, making them well-suited to operate in complex environments such as in close-proximity to coral reefs. However, they are inherently power-inefficient and produce significant noise and disturbance. Underwater gliders, on the other hand, use changes in buoyancy and center of mass, in combination with a control surface to move around. They are extremely power efficient but not very maneuverable. Gliders are designed for long-range missions that do not require precision maneuvering. Furthermore, since gliders only activate the buoyancy engine for small time intervals, they do not disturb the environment and can also be used for passive acoustic observations. In this paper we present ReefGlider, a novel AUV that uses only buoyancy for control but is still highly maneuverable from additional buoyancy control devices. ReefGlider bridges the gap between the capabilities of thruster-driven AUVs and gliders. These combined characteristics make ReefGlider ideal for tasks such as long-term visual and acoustic monitoring of coral reefs. We present the overall design and implementation of the system, as well as provide analysis of some of its capabilities.

Federated learning has been widely applied in autonomous driving since it enables training a learning model among vehicles without sharing users’ data. However, data from autonomous vehicles usually suffer from the non-independent-and-identically-distributed (non-IID) problem, which may cause negative effects on the convergence of the learning process. In this paper, we propose a new contrastive divergence loss to address the non-IID problem in autonomous driving by reducing the impact of divergence factors from transmitted models during the local learning process of each silo. We also analyze the effects of contrastive divergence in various autonomous driving scenarios, under multiple network infrastructures, and with different centralized/distributed learning schemes. Our intensive experiments on three datasets demonstrate that our proposed contrastive divergence loss significantly improves the performance over current state-of-the-art approaches. Our source code is available at https://github.com/aioz-ai/CDL.

This study presents a recursive algorithm for solving the regularised least squares problem for online identification of rigid body dynamic model parameters with emphasis on the physical consistency of estimated inertial parameters. One of the geometric approaches is to use a regulariser that represents how close the pseudo-inertia matrix is to a given reference on the feasible manifold in the regression problem. The proposed extension enables memory-efficient online learning in addition to the benefits of geometry-aware convex regularisation using the log-determinant divergence of the pseudo-inertia matrix. Also, the recursive version endows the estimator with the capability to deal with time-variation of parameters by introducing an optional forgetting mechanism. The characteristics of the recursive regularised least squares algorithm is demonstrated using the MIT Cheetah 3 leg swinging experiment dataset and compared to the existing batch optimisation method.

We present progress on the problem of reconfiguring a 2D arrangement of building material by a cooperative group of robots. These robots must avoid collisions, deadlocks, and are subjected to the constraint of maintaining connectivity of the structure. We develop two reconfiguration methods, one based on spatio-temporal planning, and one based on target swapping, to increase building efficiency. The first method can significantly reduce planning times compared to other multi-robot planners. The second method helps to reduce the amount of time robots spend waiting for paths to be cleared, and the overall distance traveled by the robots.

Natural and robotic swarms often exhibit nonreciprocal interactions; agents do not exhibit equal and opposite forces on each other. By studying the effects of reciprocal and non-reciprocal interactions we are better able to design emergent behaviors in robot collectives composed of agents that exert attractive and repulsive forces on each other. Moreover, by controlling agent-specific coupling forces on-demand, we can enable a collective to exhibit desired behaviors previously not possible. We use a general form of the swarming oscillator, swarmalator, model to study reciprocal and non-reciprocal interactions among agents that affect each other’s motions over long and short distances, we use non-reciprocal coupling to elicit collective locomotion toward or away from target sites, and we use the control barrier function method to optimize the non-reciprocal interactions for a desired spatial formation. This work addresses the interests of the active matter, swarm robotics, and control barrier functions communities and demonstrates various collective behaviors with strong potential to be realized in macro- and micro- length scale robot swarms.

Human memory of a robot’s competence, and resulting subjective perceptions of that robot, are influenced by numerous cognitive biases. One class of cognitive bias deals with the ordering of items or interactions: information presented last among a grouping is most salient in memory formation (recency bias), followed by information presented first (primacy bias), followed by information in the middle, collectively known as the serial-position effect. For example, if a human’s last observation of a robot involves a task failure, this will disproportionately negatively alter their perception of the robot’s competence, as well as their trust in the robot moving forward. It is valuable to characterize the effect of these biases within human-robot interactions to inform strategies for risk-aware planning that cultivate appropriate levels of human trust. We conducted a human-subjects study (n=53) testing the influence of the serial-position effect on recalled competence (see overview at https://youtu.be/BgH2zhh1s48). Participants viewed videos of a robot performing the same tasks at the same level of competence, with task order differing by experimental condition (rising competence, falling competence, or failures at the midpoint), asking participants to rate robot competence in between every video as well at the very end of the experiment. We found that while the average between-video rating of robot competence remained stable across conditions, the recalled, post-experiment ratings of competence and trust were significantly lower in the condition with decreasing competence than in either of the other two conditions, suggesting a notable recency bias. We conclude with implications for human-subjects experiment design (i.e., how subjective measures are influenced by ordering effects) and provide design recommendations to minimize them. We further discuss practical applications of these results in creating risk-aware robotic planners capable of trust calibration. Show More

In recent years, advanced model-based and data-driven control methods are unlocking the potential of complex robotics systems, and we can expect this trend to continue at an exponential rate in the near future. However, ensuring safety with these advanced control methods remains a challenge. A well-known tool to make controllers (either Model Predictive Controllers or Reinforcement Learning policies) safe, is the so-called control-invariant set (a.k.a. safe set). Unfortunately, for nonlinear systems, such a set cannot be exactly computed in general. Numerical algorithms exist for computing approximate control-invariant sets, but classic theoretic control methods break down if the set is not exact. This paper presents our recent efforts to address this issue. We present a novel Model Predictive Control scheme that can guarantee recursive feasibility and/or safety under weaker assumptions than classic methods. In particular, recursive feasibility is guaranteed by making the safe-set constraint move backward over the horizon, and assuming that such set satisfies a condition that is weaker than control invariance. Safety is instead guaranteed under an even weaker assumption on the safe set, triggering a safe task-abortion strategy whenever a risk of constraint violation is detected. We evaluated our approach on a simulated robot manipulator, empirically demonstrating that it leads to less constraint violations than state-of-the-art approaches, while retaining reasonable performance in terms of tracking cost, number of completed tasks, and computation time.

Generic re-usable pre-trained image representation encoders have become a standard component of methods for many computer vision tasks. As visual representations for robots however, their utility has been limited, leading to a recent wave of efforts to pre-train robotics-specific image encoders that are better suited to robotic tasks than their generic counterparts. We propose Scene Objects From Transformers, abbreviated as SOFT(•), a wrapper around pre-trained vision transformer (PVT) models that bridges this gap without any further training. Rather than construct representations out of only the final layer activations, SOFT(•) individuates and locates object-like entities from PVT attentions, and describes them with PVT activations, producing an object-centric embedding. Across standard choices of generic pre-trained vision transformers PVT, we demonstrate in each case that policies trained on SOFT(PVT) far outstrip standard PVT representations for manipulation tasks in simulated and real settings, approaching the state-of-the-art robotics-aware representations. Code, appendix and videos: https://sites.google.com/view/robot-soft/

In the field of resource-constrained robots and the need for effective place recognition in multi-robotic systems, this article introduces RecNet, a novel approach that concurrently addresses both challenges. The core of RecNet’s methodology involves a transformative process: it projects 3D point clouds into range images, compresses them using an encoder-decoder framework, and subsequently reconstructs the range image, restoring the original point cloud. Additionally, RecNet utilizes the latent vector extracted from this process for efficient place recognition tasks. This approach not only achieves comparable place recognition results but also maintains a compact representation, suitable for sharing among robots to reconstruct their collective maps. The evaluation of RecNet encompasses an array of metrics, including place recognition performance, the structural similarity of the reconstructed point clouds, and the bandwidth transmission advantages, derived from sharing only the latent vectors. Our proposed approach is assessed using both a publicly available dataset and field experiments1 confirming its efficacy and potential for real-world applications.

Realtime shape estimation of continuum objects and manipulators is essential for developing accurate planning and control paradigms. The existing methods that create dense point clouds from camera images, and/or use distinguishable markers on a deformable body have limitations in realtime tracking of large continuum objects/manipulators. The physical occlusion of markers can often compromise accurate shape estimation. We propose a robust method to estimate the shape of linear deformable objects in realtime using scattered and unordered key points. By utilizing a robust probability-based labeling algorithm, our approach identifies the true order of the detected key points and then reconstructs the shape using piecewise spline interpolation. The approach only relies on knowing the number of the key points and the interval between two neighboring points. We demonstrate the robustness of the method when key points are partially occluded. The proposed method is also integrated into a simulation in Unity for tracking the shape of a cable with a length of 1m and a radius of 5mm. The simulation results show that our proposed approach achieves an average length error of 1.07% over the continuum’s centerline and an average cross-section error of 2.11mm. The real-world experiments of tracking and estimating a heavy-load cable prove that the proposed approach is robust under occlusion and complex entanglement scenarios.

Automation in surgical robotics has the potential to improve patient safety and surgical efficiency, but it is difficult to achieve due to the need for robust perception algorithms. In particular, 6D pose estimation of surgical instruments is critical to enable the automatic execution of surgical maneuvers based on visual feedback. In recent years, supervised deep learning algorithms have shown increasingly better performance at 6D pose estimation tasks; yet, their success depends on the availability of large amounts of annotated data. In household and industrial settings, synthetic data, generated with 3D computer graphics software, has been shown as an alternative to minimize annotation costs of 6D pose datasets. However, this strategy does not translate well to surgical domains as commercial graphics software have limited tools to generate images depicting realistic instrument-tissue interactions. To address these limitations, we propose an improved simulation environment for surgical robotics that enables the automatic generation of large and diverse datasets for 6D pose estimation of surgical instruments. Among the improvements, we developed an automated data generation pipeline and an improved surgical scene. To show the applicability of our system, we generated a dataset of 7.5k images with pose annotations of a surgical needle that was used to evaluate a state-of-the-art pose estimation network. The trained model obtained a mean translational error of 2.59mm on a challenging dataset that presented varying levels of occlusion. These results highlight our pipeline’s success in training and evaluating novel vision algorithms for surgical robotics applications.

Accurate deformable object manipulation (DOM) is essential for achieving autonomy in robotic surgery, where soft tissues are being displaced, stretched, and dissected. Many DOM methods can be powered by simulation, which ensures realistic deformation by adhering to the governing physical constraints and allowing for model prediction and control. However, real soft objects in robotic surgery, such as membranes and soft tissues, have complex, anisotropic physical parameters that a simulation with simple initialization from cameras may not fully capture. To use the simulation techniques in real surgical tasks, the real-to-sim gap needs to be properly compensated. In this work, we propose an online, adaptive parameter tuning approach for simulation optimization that (1) bridges the real-to-sim gap between a physics simulation and observations obtained 3D perceptions through estimating a residual mapping and (2) optimizes its stiffness parameters online. Our method ensures a small residual gap between the simulation and observation and improves the simulation’s predictive capabilities. The effectiveness of the proposed mechanism is evaluated in the manipulation of both a thin-shell and volumetric tissue, representative of most tissue scenarios. This work contributes to the advancement of simulation-based deformable tissue manipulation and holds potential for improving surgical autonomy.

Mobile manipulators have recently gained significant attention in the robotics community due to their superior potential in industrial and service applications. However, the high degree of freedom associated with mobile manipulators poses challenges in achieving real-time whole-body motion planning. To bridge the gap, this paper presents a motion planning method capable of generating high-quality, safe, agile and feasible trajectories for mobile manipulators in real time. First, we present a novel environment-adaptive path searching method, which can generate paths in real-time in various environments by adaptively adjusting searching dimension based on environment complexity. Additionally, we propose a real-time spatial-temporal trajectory optimization method that takes into account the whole-body safety, agility and dynamic feasibility of mobile manipulators. Moreover, task constraints are applied to ensure that the trajectory can fulfill specific task requirements. Simulation and real-world experiments demonstrate that our method is capable of generating whole-body trajectories in real-time in challenging environments. We will release our code to benefit the community.

Existing motion planning approaches for over-actuated unmanned aerial vehicle (UAV) platforms can achieve online planning without considering dynamics. However, in many envisioned application areas such as aerial manipulation, payload delivery, and moving target tracking, it is critical to ensure dynamic consistency in the generated trajectory. The dynamics of these platforms introduce a high nonlinearity, leading to a substantial increase in computational burden. This paper presents an efficient method to plan motions that are consistent with the dynamics of over-actuated UAVs. With a hierarchical control structure, the dimension of the optimization problem is greatly reduced with synthesized wrench commands. Additionally, by exploring the dynamics of over-actuated UAVs, the complex planning process is decoupled into two simpler sub-problems. As a result, the proposed planner can be solved as two small quadratic programmings (QPs) and deployed in real-time. The computational efficiency and dynamic consistency of the proposed method are verified through both simulations and experiments, including comparison with other approaches and dynamic target tracking.

The vague interpretation of myoelectrical signals on the residual limb end makes restoring dexterous hand function in amputees still impossible. Understanding motor control between human motion intention and synaptic inputs to motor neurons also remains a significant challenge. The neural decoding methods of surface EMG signals remains challenging, which limit the application of robot hand in real life. Herein, we propose and substantiate a human-machine interface for motor control that introduces neural information of motor neurons in conjunction with the combination mechanism of muscle contraction. The interface firstly introduces a new concept of motor unit (MU) spike trains, which combines decoupling of the electrical activations on motor neuron axons with extraction of motion patterns from the discharge timings of the motor neuron pools. We realized a real-time implementation of the EMG decomposition algorithm on our developed prosthesis hand control system. The control scheme provides an accurate classification of intuitive hand motions, enabling the amputee to perform versatile finger movements of the prosthesis hand. The concept of motor neuron discharge timings was evaluated through experiments on one amputee participant and six able-bodied participants. The results show that the neuroprosthesis hand control scheme based on MU spike trains has the capacity of generating accurate and intuitive hand movements for amputees in a physical environment.

Controlling the shape of deformable linear objects using robots and constraints provided by environmental fixtures has diverse industrial applications. In order to establish robust contacts with these fixtures, accurate estimation of the contact state is essential for preventing and rectifying potential anomalies. However, this task is challenging due to the small sizes of fixtures, the requirement for real-time performances, and the infinite degrees of freedom of the deformable linear objects. In this paper, we propose a real-time approach for estimating both contact establishment and subsequent changes by leveraging the dependency between the applied and detected contact force on the deformable linear objects. We seamlessly integrate this method into the robot control loop and achieve an adaptive shape control framework which avoids, detects and corrects anomalies automatically. Real-world experiments validate the robustness and effectiveness of our contact estimation approach across various scenarios, significantly increasing the success rate of shape control processes.

In human-robot collaboration, there has been a trade-off relationship between the speed of collaborative robots and the safety of human workers. In our previous paper, we introduced a time-optimal path tracking algorithm designed to maximize speed while ensuring safety for human workers [1]. This algorithm runs in real-time and provides the safe and fastest control input for every cycle with respect to ISO standards [2]. However, true optimality has not been achieved due to inaccurate distance computation resulting from conservative model simplification. To attain true optimality, we require a method that can compute distances 1. at many robot configurations to examine along a trajectory 2. in realtime for online robot control 3. as precisely as possible for optimal control. In this paper, we propose a batched, fast and precise distance checking method based on precomputed link-local SDFs. Our method can check distances for 500 waypoints along a trajectory within less than 1 millisecond using a GPU at runtime, making it suited for time-critical robotic control. Additionally, a neural approximation has been proposed to accelerate preprocessing by a factor of 2. Finally, we experimentally demonstrate that our method can navigate a 6-DoF robot earlier than a geometric-primitives-based distance checker in a dynamic and collaborative environment.

Reliable localization is an essential capability for marine robots navigating in GPS-denied environments. SLAM, commonly used to mitigate dead reckoning errors, still fails in feature-sparse environments or with limited-range sensors. Pose estimation can be improved by incorporating the uncertainty prediction of future poses into the planning process and choosing actions that reduce uncertainty. However, performing belief propagation is computationally costly, especially when operating in large-scale environments. This work proposes a computationally efficient planning under uncertainty framework suitable for large-scale, feature-sparse environments. Our strategy leverages SLAM graph and occupancy map data obtained from a prior exploration phase to create a virtual map, describing the uncertainty of each map cell using a multivariate Gaussian. The virtual map is then used as a cost map in the planning phase, and performing belief propagation at each step is avoided. A receding horizon planning strategy is implemented, managing a goal-reaching and uncertainty-reduction tradeoff. Simulation experiments in a realistic underwater environment validate this approach. Experimental comparisons against a full belief propagation approach and a standard shortest-distance approach are conducted.

Assistive devices, such as exoskeletons and prostheses, have revolutionized the field of rehabilitation and mobility assistance. Efficiently detecting transitions between different activities, such as walking, stair ascending and descending, and sitting, is crucial for ensuring adaptive control and enhancing user experience. We present an approach for real-time transition detection, aimed at optimizing the processing-time performance. By establishing activity-specific threshold values through trained machine learning models, we effectively distinguish motion patterns and we identify transition moments between locomotion modes. This threshold-based method improves real-time embedded processing time performance by up to 11 times compared to machine learning approaches. The efficacy of the developed finite-state machine is validated using data collected from three different measurement systems. Moreover, experiments with healthy participants were conducted on an active pelvis orthosis to validate the robustness and reliability of our approach. The proposed algorithm achieved high accuracy in detecting transitions between activities. These promising results show the robustness and reliability of the method, reinforcing its potential for integration into practical applications.