RSS 2025 论文列表

LiDAR registration is a fundamental task in robotic mapping and localization. A critical component of aligning two point clouds is identifying robust point correspondences using point descriptors, which becomes particularly challenging in scenarios involving domain shifts, seasonal changes, and variations in point cloud structures. These factors substantially impact both handcrafted and learning-based approaches. In this paper, we address these problems by proposing to use DINOv2 features, obtained from surround-view images, as point descriptors. We demonstrate that coupling these descriptors with traditional registration algorithms, such as RANSAC or ICP, facilitates robust 6DoF alignment of LiDAR scans with 3D maps, even when the map was recorded more than a year before. Although conceptually straightforward, our method substantially outperforms more complex baseline techniques. In contrast to previous learning-based point descriptors, our method does not require domain-specific retraining and is agnostic to the point cloud structure, effectively handling both sparse LiDAR scans and dense 3D maps. We show that leveraging the additional camera data enables our method to outperform the best-performing baseline by +24.8 and +17.3 registration recall on the NCLT and Oxford RobotCar datasets. We will publicly release the code of our work upon acceptance of this manuscript (code is available for reviewers in the supplementary material).

Learning-based control approaches like reinforcement learning (RL) have recently produced a slew of impressive results for tasks like quadrotor trajectory tracking and drone racing. Naturally, it is common to demonstrate the advantages of these new controllers against established methods like analytical controllers. We observe, however, that reliably comparing the performance of these very different classes of controllers is more complicated than might appear at first sight. As a case study, we take up the problem of agile tracking of an end-effector for a quadrotor with a fixed-arm. We develop a set of best practices for synthesizing the best RL and Geometric controllers for benchmarking. In the process, we fix widely prevalent RL-favoring biases in prior studies that provide asymmetric access to: (1) the task definition in the form of objective functions, (2) datasets for parameter optimization, and (3) “feed-forward” controller inputs revealing the desired future trajectory. The resulting contributions are threefold: first, our improved robust experimental protocol reveals that the gaps between the two controller classes are much smaller than expected from previously published findings. Geometric control performs on par or better than RL in most practical settings, while RL fares better in transient performance at the expense of steady-state error. Second, our improvements to the experimental protocol for comparing learned and classical controller synthesis approaches are critical: each of the above asymmetries can yield misleading conclusions, and we show evidence that suggests that they indeed have in prior quadrotor studies. Finally, we open-source implementations of Geometric and RL controllers for these aerial vehicles implementing best practices for future development.

The advancement of generalist robotic models capable of executing diverse tasks across varied environments and embodiments has been impeded by the dependence on large-scale, labeled datasets and the inherent heterogeneity of action and observation spaces. To address these challenges, we introduce UniVLA, a framework designed to develop omni-purpose vision-language-action (VLA) policies that facilitate scalable and efficient planning across diverse environments and tasks. Our methodology comprises three pivotal stages: 1) Task-Centric Latent Action Learning, where we derive task-relevant action representations from extensive cross-embodiment videos in an unsupervised manner, utilizing DINOv2 features and language instructions to filter out task-irrelevant dynamics; 2) Latent Action Pretraining, where we train an auto-regressive vision-language model with discretized latent action tokens to enable embodiment-agnostic planning; and 3) Latent Action Decoding, where we translate latent plans into executable behaviors for deployment across diverse and heterogeneous robotic systems. UniVLA achieves state-of-the-art performance on multiple manipulation and navigation benchmarks, surpassing existing VLAs while requiring reduced computational cost. Extensive evaluations underscore the efficiency, scalability, and generalizability of UniVLA, presenting a promising pathway toward the development of next-generation generalist policies.

The behavior of in-home robots must be adaptable to end-users to adequately address individual users’ needs and preferences. Learning from Demonstration (LfD) is a common approach for customizing robot behavior, enabling non-expert users to teach robots how to perform tasks according to their preferences. While LfD allows users to teach robots tasks, it can be difficult for users to specify their individual needs a priori. Therefore, we propose Learning Interpretable Features from Interventions (LIFI), a user-friendly and streamlined method for personalizing robot behavior through interventions. This approach allows users to easily prompt the robot to adapt its behavior by intervening when the robot’s behavior against user expectations. With LIFI, 1) the user intervenes to communicate that the robot is making a mistake, 2) the robot then learns an explanatory feature that describes the failure and 3) uses it to adjust its policy to correct the mistake, aligning with user-specific needs. In a between-subjects evaluation experiment with 48 participants, where the robot attempts household manipulation tasks, we demonstrate that adding features via LIFI improves objective performance and subjective measures, i.e., perceived workload, usability, and trust, compared to a no-feature baseline.

Standing-up control is crucial for humanoid robots, with the potential for integration into current locomotion and loco-manipulation systems. Existing approaches are either limited to simulations that neglect hardware constraints or rely on predefined ground-specific motion trajectories, failing to enable standing-up across diverse postures in the real world. To bridge this gap, we present HoST (

Automatic recovery from falls is a crucial prerequisite before humanoid robots can be reliably deployed. Hand-designing controllers for getting up is difficult because of the varied configurations a humanoid can end up in after a fall and the challenging terrains humanoid robots are expected to operate on. This paper develops a learning framework to produce controllers that enable humanoid robots to get up from varying configurations on varying terrains. Different from previous successful applications of humanoid locomotion learning, the getting-up task involves complex contact patterns, the necessity to accurately model collision geometry, and sparser rewards. We circumvent these challenges through a two-phase approach that follows a curriculum. The first stage focuses on discovering a good get up trajectory under minimal constraints on smoothness or speed / torque limits. The second stage then refines the discovered motions into deployable (

Ensuring safe navigation in complex environments requires accurate real-time traversability assessment and understanding of environmental interactions relative to the robot’s capabilities. Traditional methods, which assume simplified dynamics, often require designing and tuning cost functions to safely guide paths or actions toward the goal. This process is tedious, environment-dependent, and not generalizable. To overcome these issues, we propose a novel learned perceptive Forward Dynamics Model (FDM) that predicts the robot’s future state conditioned on the surrounding geometry and history of proprioceptive measurements, proposing a more scalable, safer, and heuristic-free solution. The FDM is trained on multiple years of simulated navigation experience, including high-risk maneuvers and real-world interactions to incorporate the full system dynamics beyond rigid body simulation. We integrate our perceptive FDM into a zero-shot Model Predictive Path Integral (MPPI) planning framework, leveraging the learned mapping between actions, future states, and failure probability. This allows for optimizing a simplified cost function, eliminating the need for extensive cost-tuning to ensure safety. On the legged robot ANYmal, the proposed perceptive FDM improves the position estimation over competitive baselines, which translates into a significantly higher navigation success rate in rough simulation environments. Moreover, we demonstrate effective sim-to-real transfer and showcase the benefit of training on synthetic and real data. Code and models are made publicly available.

General-purpose humanoid robots are expected to interact intuitively with humans, enabling seamless integration into daily life. Natural language provides the most accessible medium for this purpose. However, translating languages into humanoid whole-body motions remains a significant challenge, primarily due to the gap between linguistic understanding and physical actions. In this work, we present an end-to-end, language-directed policy for real-world humanoid control. Our approach combines reinforcement learning with policy distillation, allowing a single neural network to interpret language commands and execute corresponding physical actions directly. To enhance motion diversity and compositionality, we incorporate a Conditional Variational Autoencoder (CVAE) structure. The resulting policy achieves agile and versatile whole-body behaviors conditioned on language inputs, with smooth transitions between various motions, enabling iterative and adaptable control. We validate the efficacy and generalizability of our method through extensive simulations and real-world experiments, demonstrating robust whole-body control.

State estimation and control are often addressed separately, which can lead to unsafe execution due to sensing noise, execution errors and discrepancies between the planning model and reality. Simultaneous control and trajectory estimation using probabilistic graphical models has been proposed as a unified solution to these challenges. Previous work, however, relies heavily on appropriate Gaussian priors and is limited to holonomic robots with linear time-varying models. The current research extends graphical optimization methods to vehicles with arbitrary dynamical models via Simultaneous Trajectory Estimation and Local Adaptation (STELA). The overall approach first initializes feasible trajectories using a kinodynamic, sampling-based motion planner. Then, STELA simultaneously: (i) estimates the past trajectory based on noisy observations, and (ii) adapts the controls to be executed so as to minimize deviations from the planned, feasible trajectory, while avoiding collisions. The proposed factor graph representation of trajectories in STELA can be applied for any dynamical system given access to first or second-order state update equations, and introduces the duration of execution between two states in the trajectory discretization as another variable to be optimized. These features provide both generalization and flexibility in trajectory following. In addition, and targeting computational efficiency, the proposed strategy allows for the use of incremental updates of the factor graph by using the iSAM algorithm and also introduces a time-window mechanism. This mechanism allows the factor graph to be dynamically updated so as to operate over a limited past history and forward horizon of the planned trajectory. This enables the online update of controls at a minimum of 10Hz. Experiments first demonstrate that STELA achieves at least comparable performance to previous frameworks on idealized vehicles with linear dynamics. More critically, STELA directly applies to and successfully solves trajectory following problems for more complex dynamical models. Beyond generalization, simulations assess STELA’s robustness under varying levels of sensing and execution noise, while ablation studies highlight the importance of different components of the approach. Real-world experiments validate STELA’s practical applicability on a low-cost MuSHR robot, which exhibits high noise and non-trivial dynamics.

This paper tackles the problem of jointly estimating the noise covariance matrix alongside states (parameters such as poses and points) from measurements corrupted by Gaussian noise and, if available, prior information. In such settings, the noise covariance matrix determines the weights assigned to individual measurements in the least squares problem. We show that the joint problem exhibits a convex structure and provide a full characterization of the optimal noise covariance estimate (with analytical solutions) within joint maximum a posteriori and likelihood frameworks and several variants. Leveraging this theoretical result, we propose two novel algorithms that jointly estimate the primary parameters and the noise covariance matrix. Our BCD algorithm can be easily integrated into existing nonlinear least squares solvers, with negligible per-iteration computational overhead. To validate our approach, we conduct extensive experiments across diverse scenarios and offer practical insights into their application in robotics and computer vision estimation problems with a particular focus on SLAM.

Imitation learning has enabled robots to perform complex, long-horizon tasks in challenging dexterous manipulation settings. As new methods are developed, they must be rigorously evaluated and compared against corresponding baselines through repeated evaluation trials. However, policy comparison is fundamentally constrained by a small feasible sample size (e.g., 10 or 50) due to significant human effort and limited inference throughput of policies. This paper proposes a novel statistical framework for rigorously comparing two policies in the small sample size regime. Prior work in statistical policy comparison relies on batch testing, which requires a fixed, pre-determined number of trials and lacks flexibility in adapting the sample size to the observed evaluation data. Furthermore, extending the test with additional trials risks inducing inadvertent p-hacking, undermining statistical assurances. In contrast, our proposed statistical test is

Interruptions, a fundamental component of human communication, can enhance the dynamics and effectiveness of conversations, but only when effectively managed by all parties involved. Despite advancements in robotic systems, state-of-the-art systems still have limited capabilities in handling user-initiated interruptions in real-time. Prior research has primarily focused on post hoc analysis of interruptions. To address this gap, we present a system that detects user-initiated interruptions and manages them in real-time based on the interrupter’s intent (i.e., cooperative agreement, cooperative assistance, cooperative clarification, or disruptive interruption). The system was designed based on interaction patterns identified from human-human interaction data. We integrated our system into an LLM-powered social robot and validated its effectiveness through a timed decision-making task and a contentious discussion task with 21 participants. Our system successfully handled 93.69% (n=104/111) of user-initiated interruptions. We discuss our learnings and their implications for designing interruption-handling behaviors in conversational robots.

In robot teleoperation, humans often issue control signals through an interface that requires physical actuation. This interface-level interaction largely goes unmodeled within the field, yet the robot’s interpretation of an interface-level command can differ from what was intended by the user, as a result of diminished human ability or inadequate mappings from raw interface signals to robot control signals. Interface-aware systems aim to address this limitation in robot teleoperation by explicitly considering the impact of a control interface on user input quality when interpreting interface signals for robot control. This work presents an interface-aware formulation for the direct inference of intended interface-level commands given known interaction characteristics of a control interface using data-driven modeling, allowing for teleoperation assistance without knowledge of the human’s policy. In our specific implementation, we tailor the formulation to model a user’s operation of a sip/puff interface using a network of Gated Recurrent Units, chosen for their ability to model temporal patterns and suitability for data-scarce domains. The resulting model is agnostic to the robot being controlled, which allows for its use in task- and environment-agnostic robot teleoperation assistance. We deploy this model in two variations of assisted eleoperation frameworks using a sip/puff to control a 7-DoF robotic arm, and conduct a human subjects study with spinal cord injured participants to evaluate the efficacy of our method. Our proposed task- and environment-agnostic formulation is effective in reducing collisions during teleoperation, and is preferred by users over teleoperation baselines for ease and intuitiveness of robot operation.

Flying multiple quadrotors in close proximity presents a significant challenge due to complex aerodynamic interactions, particularly downwash effects that are known to destabilize vehicles and degrade performance. Traditionally, multi-quadrotor systems rely on conservative strategies, such as collision avoidance zones around the robot volume, to circumvent this effect. This restricts their capabilities by requiring a large volume for the operation of a multi-quadrotor system, limiting their applicability in dense environments. This work provides a comprehensive, data-driven analysis of the downwash effect, with a focus on characterizing, analyzing, and understanding forces, moments, and velocities in both single and multi-quadrotor configurations. We use measurements of forces and torques to characterize vehicle interactions, and particle image velocimetry (PIV) to quantify the spatial features of the downwash wake for a single quadrotor and an interacting pair of quadrotors. This data can be used to inform physics-based strategies for coordination, leverage downwash for optimized formations, expand the envelope of operation, and improve the robustness of multi-quadrotor control.

We aim to solve the problem of manipulating deformable objects, particularly elastic bands, in real-world scenarios. However, deformable object manipulation (DOM) requires a policy that works on a large state space, due to the unlimited degree of freedom (DoF) of deformable objects. Further, their dense but partial observations (e.g., images or point clouds) may lower the sampling complexity and uncertainty in policy learning. To figure it out, we propose a novel implicit neural-representation (INR) learning for elastic DOMs, called INR-DOM. Our method learns consistent state representations associated with partially observable elastic objects reconstructing a complete and implicit surface represented as a signed distance function. Furthermore, we perform exploratory representation fine-tunning through reinforcement learning (RL) that enables RL algorithms to effectively learn exploitable representations while efficiently obtaining a DOM policy. We perform quantitative and qualitative analysis building three simulated environments and real-world manipulation studies with a Franka Emika Panda arm.

Recent advances in imitation learning, particularly using generative modelling techniques like diffusion, have enabled policies to capture complex multi-modal action distributions. However, these methods often require large datasets and multiple inference steps for action generation, posing challenges in robotics due to the high cost of data collection and limited computational resources. To address this, we introduce IMLE Policy, a novel behaviour cloning approach based on Rejection-Sampling Implicit Maximum Likelihood Estimation (RS-IMLE). IMLE Policy excels in low-data regimes, effectively learning from minimal demonstrations and requiring 38% less data on average to match the performance of baseline methods in learning complex multi-modal behaviours. Its simple generator-based architecture enables single-step action generation, improving inference speed by 97.3% compared to Diffusion Policy, while outperforming single-step Flow Matching. We validate our approach across diverse manipulation tasks in simulated and real-world environments, showcasing its ability to capture complex behaviours under data constraints.

Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a system that integrates data collection and imitation learning from both humans and LocoMan, a quadrupedal robot with multiple manipulation modes. Specifically, we introduce a teleoperation and data collection pipeline, supported by dedicated hardware, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient learning architecture that supports co-training and pretraining with multimodal data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. Experimental results demonstrate that our data collection and training framework significantly improves the efficiency and effectiveness of imitation learning, enabling more versatile quadrupedal manipulation capabilities. Our hardware, data, and code are open-sourced at: https://human2bots.github.io.

Efficient mission planning for cooperative systems involving Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) requires addressing energy constraints, scalability, and coordination challenges between agents. UAVs excel in rapidly covering large areas but are constrained by limited battery life, while UGVs, with their extended operational range and capability to serve as mobile recharging stations, are hindered by slower speeds. This heterogeneity makes coordination between UAVs and UGVs critical for achieving optimal mission outcomes. In this work, we propose a scalable deep reinforcement learning (DRL) framework to address the energy-constrained cooperative routing problem for multi-agent UAV-UGV teams, aiming to visit a set of task points in minimal time with UAVs relying on UGVs for recharging during the mission. The framework incorporates sortie-wise agent switching to efficiently manage multiple agents, by allocating task points and coordinating actions. Using an encoder-decoder transformer architecture, it optimizes routes and recharging rendezvous for the UAV-UGV team in the task scenario. Extensive computational experiments demonstrate the framework’s superior performance over heuristic methods and a DRL baseline, delivering significant improvements in solution quality and runtime efficiency across diverse scenarios. Generalization studies validate its robustness, while dynamic scenario analyses highlight its adaptability to real-time changes with a case study. This work advances UAV-UGV cooperative routing by providing a scalable, efficient, and robust solution for multi-agent mission planning.

For robots to operate in general environments like households, they must be able to perform non-prehensile manipulation actions such as toppling and rolling to manipulate ungraspable objects. However, prior works on non-prehensile manipulation cannot yet generalize across environments with diverse geometries. The main challenge lies in adapting to varying environmental constraints: within a cabinet, the robot must avoid walls and ceilings; to lift objects to the top of a step, the robot must account for the step’s pose and extent. While deep reinforcement learning (RL) has demonstrated impressive success in non-prehensile manipulation, accounting for such variability presents a challenge for the generalist policy, as it must learn diverse strategies for each new combination of constraints. To address this, we propose a modular and reconfigurable architecture that adaptively reconfigures network modules based on task requirements. To capture the geometric variability in environments, we extend the contact-based object representation (CORN) to environment geometries, and propose a procedural algorithm for generating diverse environments to train our agent. Taken together, the resulting policy can zero-shot transfer to novel real-world environments and objects despite training entirely within a simulator. We additionally release a simulation-based benchmark featuring nine digital twins of real-world scenes with 353 objects to facilitate non-prehensile manipulation research in realistic domains.