ICRA 2024 论文列表

The presented work tackles the question of quantifying the pose deviations of robots subject to external disturbance forces. While this question may not be central for large robots perfectly rejecting disturbances through high controller gains, it is an important factor when considering collaborative settings where smaller robots may be deviated from their task because of unmodeled physical interactions. This is all the more true with human-robot collaboration where human capacities may fluctuate over time and have to be compensated by a proper adaptation of the robot control. To move forward in this direction, this work first derives a deviation prediction methodology and exemplifies it using three largely employed control approaches. The proposed prediction method is then validated using simulated and real robot experiments both in single and multiple robots cases. The obtained results constitute a stepping stone towards a quantitative metric for robots adapting their behaviour to human motor fluctuation.

For a service robot, it is crucial to perceive as early as possible that an approaching person intends to interact: in this case, it can proactively enact friendly behaviors that lead to an improved user experience. We solve this perception task with a sequence-to-sequence classifier of a potential user intention to interact, which can be trained in a self-supervised way. Our main contribution is a study of the benefit of features representing the person’s gaze in this context. Extensive experiments on a novel dataset show that the inclusion of gaze cues significantly improves the classifier performance (AUROC increases from 84.5 % to 91.2 %); the distance at which an accurate classification can be achieved improves from 2.4 m to 3.2 m. We also quantify the system’s ability to adapt to new environments without external supervision. Qualitative experiments show practical applications with a waiter robot.

Time-effective and accurate source localization with mobile robots is crucial in safety-critical scenarios, e.g. leakage detection. This becomes particular challenging in realistic cluttered scenarios, i.e. in the presence of complex current flows or wind. Traditional methods often fall short due to simplifications or limited onboard resources.We propose to combine source localization with a Gaussian Markov Random Field (GMRF). This allows to improve source localization hypotheses by building on the GMRF’s concentration and flow field belief that are continuously updated by gathered measurements. We introduce the upstream source proximity (USP) as a natural metric that exploits the joint knowledge represented in the field belief’s concentration and flow field, i.e. predicting sources upstream. As a result, our method yields a computationally efficient source localization and field belief module providing substantially more stable gradients than conventional concentration gradient-based methods.We demonstrate the suitability of our approach in a series of numerical experiments covering complex source location scenarios. With regard to computational requirements, the method achieves update rates of 10Hz on a RaspberryPi4B.

2D top-down maps are commonly used for the navigation and exploration of mobile robots through unknown areas. Typically, the robot builds the navigation maps incrementally from local observations using onboard sensors. Recent works have shown that predicting the structural patterns in the environment through learning-based approaches can greatly enhance task efficiency. While many such works build task-specific networks using limited datasets, we show that the existing foundational vision networks can accomplish the same without any fine-tuning. Specifically, we use Masked Autoencoders, pre-trained on street images, to present novel applications for field-of-view expansion, single-agent topological exploration, and multi-agent exploration for indoor mapping, across different input modalities. Our work motivates the use of foundational vision models for generalized structure prediction-driven applications, especially in the dearth of training data. We share more qualitative results at https://raaslab.org/projects/MIM4Robots.

This paper offers a practical method for certifiably safe operations of an unmanned aerial vehicle (UAV) with limited power and computation, useful for real-time operations where the UAV is exposed to significant disturbances in non-convex free space. We propose a motion planning method based on the Explicit Reference Governor (ERG) framework to ensure the safety of a flying quadrotor UAV. From a small set of experiment data and assumptions on modeling errors, a Lyapunov function is synthesized by which an ERG is constructed to modify the UAV set-points. The method can handle polyhedral obstacles and constraints imposed on the maximum thrust of the UAV and its maximum tilt. We demonstrate the approach with extensive simulations and experiments using a Crazyflie 2.1.

Visual-based robot pose estimation is a fundamental challenge, involving the determination of the camera’s pose with respect to a robot. Conventional methods for camera-to-robot pose calibration rely on fiducial markers to establish keypoint correspondences. However, these approaches exhibit significant variability in accuracy and robustness, particularly in 2D keypoint detection. In this work, we present an end-to-end pose estimation approach that achieves camera-to-robot calibration using monocular images and keypoint information. Our method employs a two-level nested U-shaped architecture, featuring a bottom-level residual U-block to extract richer contextual information from diverse receptive fields to enhance keypoint refinement. By incorporating the perspective-n-point (PnP) algorithm and leveraging 3D robot joint keypoints, we establish correspondence of 3D coordinate points between the robot’s coordinate system and the camera’s coordinate system, facilitating accurate pose estimation. Experimental evaluations encompass real-world and synthetic datasets, demonstrating competitive results across three distinct robot manipulators.

Offline reinforcement learning (RL) aims to optimize a policy, based on pre-collected data, to maximize the cumulative rewards after performing a sequence of actions. Existing approaches learn a value function from historical data and then guide the updating of the policy parameters by maximizing the value function at a single time. Driven by the gap between maximizing the cumulative rewards of RL and the greedy strategy of existing methods, we propose an approach of policy optimization by looking ahead (POLA) to mitigate the gap. Concretely, we optimize the policy on both current and future states where the future states are predicted by a transition model. A trajectory contains numerous actions before the task is done. Performing the best action at each time does not mean an optimal trajectory in the end. We need to allow sub-optimal or negative actions occasionally. But existing methods focus on generating the optimal action at each time according to the maximizing Q-value principle. This motivates our looking ahead approach. Besides, hidden confounding factors may affect the decision making process. To that end, we incorporate the correlations among dimensions of the state into the policy, providing more information about the environment for the policy to make decisions. Empirical results on the Mujoco dataset show the effectiveness of the proposed approach.

Semantic Scene Completion (SSC) aims to jointly generate space occupancies and semantic labels for complex 3D scenes. Most existing SSC models focus on volumetric representations, which are memory-inefficient for large outdoor spaces. Point clouds provide a lightweight alternative but existing benchmarks lack outdoor point cloud scenes with semantic labels. To address this, we introduce PointSSC, the first cooperative vehicle-infrastructure point cloud benchmark for semantic scene completion. These scenes exhibit long-range perception and minimal occlusion. We develop an automated annotation pipeline leveraging Semantic Segment Anything to efficiently assign semantics. To benchmark progress, we propose a LiDAR-based model with a Spatial-Aware Transformer for global and local feature extraction and a Completion and Segmentation Cooperative Module for joint completion and segmentation. PointSSC provides a challenging testbed to drive advances in semantic point cloud completion for real-world navigation. The code and datasets are available at https://github.com/yyxssm/PointSSC.

Surface vibration tactile feedback is capable of conveying various semantic information to humans via handheld electronic devices, such as smartphones, touch panels, and game controllers. However, covering the entire contacting surface of the device with a dense arrangement of actuators can affect its normal use. Determining how to produce desired vibration patterns at any contact point with only a few sparse actuators deployed on the surface of the handheld device remains a significant challenge. In this work, we develop a tactile feedback board in the size of a smartphone with only five actuators, and achieve the precise production of vibration patterns that can focus at any desired position on the board. Specifically, we investigate the vibration characteristics of a single passive coil actuator and construct its vibration pattern model for any position on the feedback board surface. Optimal phase and amplitude modulation, determined using the simulated annealing algorithm, is employed with five actuators in a sparse array. The vibration patterns from all actuators are superimposed linearly to synthetically generate different onboard vibration energy distributions for tactile sensing. Experiments demonstrated that point-wise vibration pattern production on our tactile board achieved an average level of about 0.9 in the Structural Similarity Index Measure (SSIM) evaluation, when compared to the ideal single-point-focused target vibration pattern. Four point-wise patterns focused on the top, bottom, left, and right parts of the tactile board were applied, to guide continuous directional movements without visual assistance, which shows significant implications for machine-assisted cognition based on vibration tactile feedback.

Control barrier functions have become an increasingly popular framework for safe real-time control. In this work, we present a computationally low-cost framework for synthesizing barrier functions over point cloud data for safe vision-based control. We take advantage of surface geometry to locally define and synthesize a quadratic CBF over a point cloud. This CBF is used in a CBF-QP for control and verified in simulation on quadrotors and in hardware on quadrotors and the TurtleBot3. This technique enables safe navigation through unstructured and dynamically changing environments and is shown to be significantly more efficient than current methods.

Visual control policies can encounter significant performance degradation when visual conditions like lighting or camera position differ from those seen during training – often exhibiting sharp declines in capability even for minor differences. In this work, we examine robustness to a suite of these types of visual changes for RGB-D and point cloud based visual control policies. To perform these experiments on both model-free and model-based reinforcement learners, we introduce a novel Point Cloud World Model (PCWM) and point cloud based control policies. Our experiments show that policies that explicitly encode point clouds are significantly more robust than their RGB-D counterparts. Further, we find our proposed PCWM significantly outperforms prior works in terms of sample efficiency during training. Taken together, these results suggest reasoning about the 3D scene through point clouds can improve performance, reduce learning time, and increase robustness for robotic learners. Code: https://github.com/pvskand/pcwm

Low back pain is a widespread issue that affects people worldwide and can lead to serious conditions such as herniated discs, spinal stenosis, or lumbar radiculopathy. Improper posture while lifting heavy weights is a common cause of back pain, especially among laborers. However, current back exoskeletons are often bulky and require electric motors, making them challenging to use and consuming significant power. Some passive exoskeletons don’t require power, but their fixed stiffness constrains normal motion. This paper presents a novel solution: a pneumatic back exoskeleton made of structured fabrics that can adjust stiffness under various air pressures. Additionally, it includes IMU sensors to detect lifting posture and correct it in real time. The exoskeleton’s effectiveness was tested through lifting experiments, demonstrating that it significantly corrects lifting posture, reduces stress on the lumbar spine, and mitigates back muscle stress. This pneumatic back exoskeleton offers a promising solution to prevent low back pain during weight-lifting tasks and provides guidance for future back exoskeleton designs.

Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining. However, while considerable effort has been made to teach the robot the "dos", the "don’ts" received relatively less attention. We argue that, for any practical usage, it is as crucial to teach the robot the "don’ts": conveying explicit instructions about prohibited actions, assessing the robot’s comprehension of these restrictions, and, most importantly, ensuring compliance. Moreover, verifiable safe operation is essential for deployments that satisfy worldwide standards such as ISO 61508, which defines standards for safely deploying robots in industrial factory environments worldwide. Aiming at deploying the LLM agents in a collaborative environment, we propose a queryable safety constraint module based on linear temporal logic (LTL) that simultaneously enables natural language (NL) to temporal constraints encoding, safety violation reasoning and explaining, and unsafe action pruning. To demonstrate the effectiveness of our system, we conducted experiments in VirtualHome environment and on a real robot. The experimental results show that our system strictly adheres to the safety constraints and scales well with complex safety constraints, highlighting its potential for practical utility.

Existing robotic musicians utilize detailed handcrafted instrument models to generate or learn policies for playing because model-free or inaccurate policy rollouts might easily damage or wear out fragile instruments. We introduce an approach to characterize geometric models of chordophones and their audio onset responses directly through audio-tactile exploration with a physical robot arm. Initially, the system refines prior estimates of string positions, provided by kinesthetic teaching or visual estimation, through repeated attempts to pluck individual strings. A subsequent stage implements a Safe Active Exploration paradigm based on Gaussian Processes to explore and characterize the audio onset response of feasible plucking motions while minimizing invalid attempts. The resulting models can be used to actuate an imprecise robotic arm to play sequences of notes with varying loudness on a Chinese Guzheng.

Path planning in a changing environment is a challenging task in robotics, as moving objects impose time-dependent constraints. Recent planning methods primarily focus on the spatial aspects, lacking the capability to directly incorporate time constraints. In this paper, we propose a method that leverages a generative model to decompose a complex planning problem into small manageable ones by incrementally generating subgoals given the current planning context. Then, we take into account the temporal information and use learned time estimators based on different statistic distributions to examine and select the generated subgoal candidates. Experiments show that planning from the current robot state to the selected subgoal can satisfy the given time-dependent constraints while being goal-oriented.

The choices made by autonomous robots in social settings bear consequences for humans and their presumptions of robot behavior. Explanations can serve to alleviate detrimental impacts on humans and amplify their comprehension of robot decisions. We model the process of explanation generation for robot navigation as an automated planning problem considering different possible explanation attributes. Our visual and textual explanations of a robot’s navigation are influenced by the robot’s personality. Moreover, they account for different contextual, environmental, and spatial characteristics. We present the results of a user study demonstrating that users are more satisfied with multimodal than unimodal explanations. Additionally, our findings reveal low user satisfaction with explanations of a robot with extreme personality traits. In conclusion, we deliberate on potential future research directions and the associated constraints. Our work advocates for fostering socially adept and safe autonomous robot navigation.

Mobile manipulators have been employed in many applications that are traditionally performed by either multiple fixed-base robots or a large robotic system. This capability is enabled by the mobility of the mobile base. However, the mobile base also brings redundancy to the system, which makes mobile manipulator motion planning more challenging. In this paper, we tackle the mobile manipulator motion planning problem under the end-effector trajectory continuity constraint in which the end-effector is required to traverse a continuous task-space trajectory (time-parametrized path), such as in mobile printing or spraying applications. Our method decouples the problem into: (1) planning an optimal base trajectory subject to geometric task constraints, end-effector trajectory continuity constraint, collision avoidance, and base velocity constraint; which ensures that (2) a manipulator trajectory is computed subsequently based on the obtained base trajectory. To validate our method, we propose a discrete optimal base trajectory planning algorithm to solve several mobile printing tasks in hardware experiment and simulations.

"Hey, robot. Let’s tidy up the kitchen. By the way, I have back pain today". How can a robotic system devise a shared plan with an appropriate task allocation from this abstract goal and agent condition? Classical AI task planning has been explored for this purpose, but it involves a tedious definition of an inflexible planning problem. Large Language Models (LLMs) have shown promising generalisation capabilities in robotics decision-making through knowledge extraction from Natural Language (NL). However, the translation of NL information into constrained robotics domains remains a challenge. In this paper, we use LLMs as translators between NL information and a structured AI task planning problem, targeting human-robot collaborative plans. The LLM generates information that is encoded in the planning problem, including specific subgoals derived from an NL abstract goal, as well as recommendations for subgoal allocation based on NL agent conditions. The framework, PlanCollabNL, is evaluated for a number of goals and agent conditions, and the results show that correct and executable plans are found in most cases. With this framework, we intend to add flexibility and generalisation to HRC plan generation, eliminating the need for a manual and laborious definition of restricted planning problems and agent models.

Recent results suggest that splitting topological navigation into robot-independent and robot-specific components improves navigation performance by enabling the robot-independent part to be trained with data collected by robots of different types. However, the navigation methods’ performance is still limited by the scarcity of suitable training data and they suffer from poor computational scaling. In this work, we present PlaceNav, subdividing the robot-independent part into navigation-specific and generic computer vision components. We utilize visual place recognition for the subgoal selection of the topological navigation pipeline. This makes subgoal selection more efficient and enables leveraging large-scale datasets from non-robotics sources, increasing training data availability. Bayesian filtering, enabled by place recognition, further improves navigation performance by increasing the temporal consistency of subgoals. Our experimental results verify the design and the new method obtains a 76 % higher success rate in indoor and 23 % higher in outdoor navigation tasks with higher computational efficiency.