ICRA 2024 论文列表

Squirrels exhibit agile leaping between tree branches, often using non-prehensile gripping with compliant and passively adaptive fingers. We aim to test the utility of such gripping in agile robotic maneuvering. In the present study, we first examine the parametric design of a squirrel-inspired underactuated gripper for passive landing on impact. We fix the geometry of the gripper and vary the joint stiffness and contact conditions. We find that stiffer fingers with soft foam pads enlarge the landing sufficiency region. Specifically, friction appears to enlarge horizontal error tolerance, while joint stiffness and pad damping allow for higher impact speeds. Thus, these features should be considered in the design of future agile robot hands and feet that include high impact landings on rods with pose inaccuracy.

While Global Navigation Satellite System (GNSS) is often used to provide global positioning if available, its intermittency and/or inaccuracy calls for fusion with other sensors. In this paper, we develop a novel GNSS-Visual-Inertial Navigation System (GVINS) that fuses visual, inertial, and raw GNSS measurements within the square-root inverse sliding window filtering (SRI-SWF) framework in a tightly coupled fashion, which thus is termed SRI-GVINS. In particular, for the first time, we deeply fuse the GNSS pseudorange, Doppler shift, single-differenced pseudorange, and double-differenced carrier phase measurements, along with the visual-inertial measurements. Inherited from the SRI-SWF, the proposed SRI-GVINS gains significant numerical stability and computational efficiency over the start-of-the-art methods. Additionally, we propose to use a filter to sequentially initialize the reference frame transformation till converges, rather than collecting measurements for batch optimization. We also perform online calibration of GNSS-IMU extrinsic parameters to mitigate the possible extrinsic parameter degradation. The proposed SRI-GVINS is extensively evaluated on our own collected UAV datasets and the results demonstrate that the proposed method is able to suppress VIO drift in real-time and also show the effectiveness of online GNSS-IMU extrinsic calibration. The experimental validation on the public datasets further reveals that the proposed SRI-GVINS outperforms the state-of-the-art methods in terms of both accuracy and efficiency.

This paper presents a reactive navigation method that leverages a Model Predictive Path Integral (MPPI) control enhanced with spline interpolation for the control input sequence and Stein Variational Gradient Descent (SVGD). The MPPI framework addresses a nonlinear optimization problem by determining an optimal sequence of control inputs through a sampling-based approach. The efficacy of MPPI is significantly influenced by the sampling noise. To rapidly identify routes that circumvent large and/or newly detected obstacles, it is essential to employ high levels of sampling noise. However, such high noise levels result in jerky control input sequences, leading to non-smooth trajectories. To mitigate this issue, we propose the integration of spline interpolation within the MPPI process, enabling the generation of smooth control input sequences despite the utilization of substantial sampling noises. Nonetheless, the standard MPPI algorithm struggles in scenarios featuring multiple optimal or near-optimal solutions, such as environments with several viable obstacle avoidance paths, due to its assumption that the distribution over an optimal control input sequence can be closely approximated by a Gaussian distribution. To address this limitation, we extend our method by incorporating SVGD into the MPPI framework with spline interpolation. SVGD, rooted in the optimal transportation algorithm, possesses the unique ability to cluster samples around an optimal solution. Consequently, our approach facilitates robust reactive navigation by swiftly identifying obstacle avoidance paths while maintaining the smoothness of the control input sequences. The efficacy of our proposed method is validated on simulations with a quadrotor, demonstrating superior performance over existing baseline techniques.

Animals possessing spinal columns often exhibit exceptional agility for highly dynamic locomotion. The spine grants the trunk with increased degrees of freedom, thereby endowing diverse postures. This paper presents the development of a robot STRAY for quadrupedal locomotion, featuring a four-degree-of-freedom spine design. Using trajectory based reinforcement learning techniques, STRAY is able to trot and bound dynamically using its spine. Simulation results reveal the positive roles of spinal movement, such as twisting, extension, retraction and rotation, in helping STRAY realize efficient locomotion. Preliminary results from experiments demonstrate that STRAY can achieve a trotting gait of approximately 0.6 m/s and a bounding gait of 0.7 m/s, with desired velocities of 0.8 m/s and 1.0 m/s, respectively. The results also indicate that reinforcement learning is a feasible way to investigate how the spine should be used in dynamic quadrupedal locomotion and achieve more possibilities in the future.

With the proliferation of autonomous vehicles (AVs) comes the need to ensure they abide by safe driving properties. Specifying and monitoring such properties, however, is challenging because of the mismatch between the semantic space over which typical driving properties are asserted (e.g., vehicles, pedestrians, intersections) and the sensed inputs of AVs. Existing efforts either assume for such semantic data to be available or develop bespoke methods for capturing it. Instead, this work introduces a framework that can extract scene graphs (SGs) from sensor inputs to capture the entities related to the AV, and a domain-specific language that enables building propositions over those graphs and composing them through temporal logic. We implemented the framework to monitor for specification violations of 3 top AVs from the CARLA Autonomous Driving Leaderboard, and found that the AVs violated 71% of properties during at least one test. Artifact available at https://github.com/less-lab-uva/SGSM.

The existing internet-scale image and video datasets cover a wide range of everyday objects and tasks, bringing the potential of learning policies that generalize in diverse scenarios. Prior works have explored visual pre-training with different self-supervised objectives. Still, the generalization capabilities of the learned policies and the advantages over well-tuned baselines remain unclear from prior studies. In this work, we present a focused study of the generalization capabilities of the pre-trained visual representations at the categorical level. We identify the key bottleneck in using a frozen pre-trained visual backbone for policy learning and then propose SpawnNet, a novel two-stream architecture that learns to fuse pre-trained multi-layer representations into a separate network to learn a robust policy. Through extensive simulated and real experiments, we show significantly better categorical generalization compared to prior approaches in imitation learning settings. Open-sourced code and videos can be found on our website: https://xingyu-lin.github.io/spawnnet/.

Tracking of plant organs over spatio-temporal sequence of point cloud data is one of the demanding tasks of agricultural robotics for automated plant monitoring and growth analysis. Due to the complex geometry of plants, it is extremely difficult to identify and track the individual organs in different growth stages of plants. In this paper, we present an approach to perform correspondence estimation of different plant organs over a series of spatio-temporal data. The approach is based on two stages. In the first stage we develop a robust skeleton extraction method from unstructured plant point cloud data by adopting Hausdorff distance metric and modified breadth first search algorithm. The proposed skeletonization method is shown to be performing better than state-of-the-art, especially in handling very thin and delicate branches. We also address an overlooked problem of connecting skeleton points in the form of a graph, and demonstrate that different types of plant phenotype parameters can be obtained in a fully automatic manner from the skeleton graph. In the second stage, we exploit the skeleton graphs in developing an algorithm to perform correspondence estimation among the skeleton nodes using a cosine similarity based approach. We demonstrate the effectiveness of the proposed skeletonization technique in tracking different organs of the plant by finding good quality correspondences. Experiments are performed on three datasets on real and synthetic sequence of spatio-temporal plant point cloud data to demonstrate the effectiveness of the proposed method.

Aerial robots have the potential to play a crucial role in assisting humans with complex and dangerous tasks. Nevertheless, the future industry demands innovative solutions to streamline the interaction process between humans and drones to enable seamless collaboration and efficient coworking. In this paper, we present a novel tele-immersive framework that promotes cognitive and physical collaboration between humans and robots through Mixed Reality (MR). This framework incorporates a novel bi-directional spatial awareness and a multi-modal virtual-physical interaction approaches. The former seamlessly integrates the physical and virtual worlds, offering bidirectional egocentric and exocentric environmental representations. The latter, leveraging the proposed spatial representation, further enhances the collaboration combining a robot planning algorithm for obstacle avoidance with a variable admittance control. This allows users to issue commands based on virtual forces while maintaining compatibility with the environment map. We validate the proposed approach by performing several collaborative planning and exploration tasks involving a drone and an user equipped with a MR headset.

We present SpaceHopper, a three-legged, small-scale robot designed for future mobile exploration of asteroids and moons. The robot weighs 5.2 kg and has a body size of 245 mm while using space-qualifiable components. Furthermore, SpaceHopper’s design and controls make it well-adapted for investigating dynamic locomotion modes with extended flight-phases. Instead of gyroscopes or fly-wheels, the system uses its three legs to reorient the body during flight in preparation for landing. We control the leg motion for reorientation using Deep Reinforcement Learning policies. In a simulation of Ceres’ gravity (0.029 g), the robot can reliably jump to commanded positions up to 6 m away. Our real-world experiments show that SpaceHopper can successfully reorient to a safe landing orientation within 9.7 deg inside a rotational gimbal and jump in a counterweight setup in Earth’s gravity. Overall, we consider SpaceHopper an important step towards controlled jumping locomotion in low-gravity environments.

We consider a set of challenging sequential manipulation puzzles, where an agent has to interact with multiple movable objects and navigate narrow passages. Such settings are notoriously difficult for Task-and-Motion Planners, as they require interdependent regrasps and solving hard motion planning problems.In this paper, we propose to search over sequences of easier pick-and-place subproblems, which can lead to the solution of the manipulation puzzle. Our method combines a heuristic-driven forward search of subproblems with an optimization-based Task-and-Motion Planning solver. To guide the search, we introduce heuristics to generate and prioritize useful subgoals. We evaluate our approach on various manually designed and automatically generated scenes, demonstrating the benefits of auxiliary subproblems in sequential manipulation planning.

Hand extension is crucial for stroke survivors with spasticity, where their fingers become rigid and their thumb remains curled within the palm. Due to the underactuated nature of the hand, the dominance of flexor muscles over extensors, and the limited surface area available, developing an extension glove with thumb assistance poses a challenge for researchers. This paper introduces a fully wearable soft hand extension glove based on the X-pouch and strap system, addressing the above challenges. The glove enables adequate finger extension, thumb abduction, and extension for high MAS score patients. Modelling and testing revealed extension torques of up to 2.7 Nm at the MCP joint and 0.67 Nm at the PIP and DIP joints. Performance evaluation, including comparison with existing methods, demonstrated the glove’s superior extension capabilities using a model hand with realistic stiffness. Furthermore, the glove’s effectiveness was confirmed through testing on a stroke patient with MAS = 2, validating its on-body functionality.

Acoustic techniques have been developed as multifunctional tools for various microscale manipulations. In prevalent design paradigms, a position-fixed piezoelectric transducer (PZT) is utilized to generate ultrasound waves. However, the immobility of the PZT restricts the modulation of the acoustic field's position and orientation, consequently diminishing the adaptability and effectiveness of subsequent acoustic micromanipulation tasks. Here, we proposed a miniaturized soft acoustic end-effector and demonstrated acoustic field modulation and microparticle manipulation by adjusting PZT position and orientation. The PZT is mounted on the end of a soft robotic arm that has three individual degrees of freedom and can be deformed in 3D space by inflating or deflating each chamber. Experiments showed that the soft acoustic end-effector can change the traveling direction of microparticles and modulate the location of a standing wave field. Our approach is simple, flexible, and controllable. We envision that the soft acoustic end-effector will facilitate multiscale acoustic manipulation in interdisciplinary applications, especially, for in vivo acoustic therapies.

Navigation through crowded human environments is challenging for social robots. While reinforcement learning has been adopted for its capacity to capture complex interactions, the training process often relies on simulators to replicate realistic crowd behaviors, ensuring cost-efficiency. Existing crowd simulation methods typically rely on either handcrafted rules, which may lead to overly aggressive navigation, or learning from human trajectory demonstrations, which can be challenging to generalize effectively. In this paper, we introduce a data-driven crowd simulation method called SocialGAIL, which leverages Generative Adversarial Imitation Learning (GAIL) to emulate real pedestrian navigation in crowded environments. SocialGAIL utilizes an attention-based graph neural network to encode observations and employs a generator-discriminator architecture to closely mimic pedestrian behavior. We propose a set of metrics to evaluate the faithfulness of crowd simulation. Experimental results demonstrate that SocialGAIL outperforms baseline methods in terms of goal-reaching, intermediate state faithfulness, trajectory faithfulness, and adherence to global trajectory patterns. The code of our approach is available at https://github.com/William-island/SocialGAIL.

Along with the advancement of robot skin technology, there has been notable progress in the development of snake robots featuring body-surface tactile perception. In this study, we proposed a locomotion control framework for snake robots that integrates tactile perception to augment their adaptability to various terrains. Our approach embraces a hierarchical reinforcement learning (HRL) architecture, wherein the high-level orchestrates global navigation strategies while the low-level uses curriculum learning for local navigation maneuvers. Due to the significant computational demands of collision detection in whole-body tactile sensing, the efficiency of the simulator is severely compromised. Thus a distributed training pattern to mitigate the efficiency reduction was adopted. We evaluated the navigation performance of the snake robot in complex large-scale cave exploration with challenging terrains to exhibit improvements in motion efficiency, evidencing the efficacy of tactile perception in terrain-adaptive locomotion.

Model Predictive Control (MPC) has exhibited remarkable capabilities in optimizing objectives and meeting constraints. However, the substantial computational burden associated with solving the Optimal Control Problem (OCP) at each triggering instant introduces significant delays between state sampling and control application. These delays limit the practicality of MPC in resource-constrained systems when engaging in complex tasks. The intuition to address this issue in this paper is that by predicting the successor state, the controller can solve the OCP one time step ahead of time thus avoiding the delay of the next action. To this end, we compute deviations between real and nominal system states, predicting forthcoming real states as initial conditions for the imminent OCP solution. Anticipatory computation stores optimal control based on current nominal states, thus mitigating the delay effects. Additionally, we establish an upper bound for linearization error, effectively linearizing the nonlinear system, reducing OCP complexity, and enhancing response speed. We provide empirical validation through two numerical simulations and corresponding real-world robot tasks, demonstrating significant performance improvements and augmented response speed (up to 90%) resulting from the seamless integration of our proposed approach compared to conventional time-triggered MPC strategies.

In recent years, autonomous driving has garnered significant attention due to its potential for improving road safety through collaborative perception among connected and autonomous vehicles (CAVs). However, time-varying channel variations in vehicular transmission environments demand dynamic allocation of communication resources. Moreover, in the context of collaborative perception, it is important to recognize that not all CAVs contribute valuable data, and some CAV data even have detrimental effects on collaborative perception. In this paper, we introduce SmartCooper, an adaptive collaborative perception framework that incorporates communication optimization and a judger mechanism to facilitate CAV data fusion. Our approach begins with optimizing the connectivity of vehicles while considering communication constraints. We then train a learnable encoder to dynamically adjust the compression ratio based on the channel state information (CSI). Subsequently, we devise a judger mechanism to filter the detrimental image data reconstructed by adaptive decoders. We evaluate the effectiveness of our proposed algorithm on the OpenCOOD platform. Our results demonstrate a substantial reduction in communication costs by 23.10% compared to the non-judger scheme. Additionally, we achieve a significant improvement on the average precision of Intersection over Union (AP@IoU) by 7.15% compared with state-of-the-art schemes.

Learning multi-object dynamics from visual data using unsupervised techniques is challenging due to the need for robust, object representations that can be learned through robot interactions. This paper presents a novel framework with two new architectures: SlotTransport for discovering object representations from RGB images and SlotGNN for predicting their collective dynamics from RGB images and robot interactions. Our SlotTransport architecture is based on slot attention for unsupervised object discovery and uses a feature transport mechanism to maintain temporal alignment in object-centric representations. This enables the discovery of slots that consistently reflect the composition of multi-object scenes. These slots robustly bind to distinct objects, even under heavy occlusion or absence. Our SlotGNN, a novel unsupervised graph-based dynamics model, predicts the future state of multi-object scenes. SlotGNN learns a graph representation of the scene using the discovered slots from SlotTransport and performs relational and spatial reasoning to predict the future appearance of each slot conditioned on robot actions. We demonstrate the effectiveness of SlotTransport in learning object-centric features that accurately encode both visual and positional information. Further, we highlight the accuracy of SlotGNN in downstream robotic tasks, including challenging multi-object rearrangement and long-horizon prediction. Finally, our unsupervised approach proves effective in the real world. With only minimal additional data, our framework robustly predicts slots and their corresponding dynamics in real-world control tasks. Our project webpage: bit.ly/slotgnn.

Cooking robots can enhance the home experience by reducing the burden of daily chores. However, these robots must perform their tasks dexterously and safely in shared human environments, especially when handling dangerous tools such as kitchen knives. This study focuses on enabling a robot to autonomously and safely learn food-cutting tasks. More specifically, our goal is to enable a collaborative robot or industrial robot arm to perform food-slicing tasks by adapting to varying material properties using compliance control. Our approach involves using Reinforcement Learning (RL) to train a robot to compliantly manipulate a knife, by reducing the contact forces exerted by the food items and by the cutting board. However, training the robot in the real world can be inefficient, and dangerous, and result in a lot of food waste. Therefore, we proposed SliceIt!, a framework for safely and efficiently learning robot food-slicing tasks in simulation. Following a real2sim2real approach, our framework consists of collecting a few real food slicing data, calibrating our dual simulation environment (a high-fidelity cutting simulator and a robotic simulator), learning compliant control policies on the calibrated simulation environment, and finally, deploying the policies on the real robot.

Deploying robots in real-world environments, such as households and manufacturing lines, requires generalization across novel task specifications without violating safety constraints. Linear temporal logic (LTL) is a widely used task specification language with a compositional grammar that naturally induces commonalities among tasks while preserving safety guarantees. However, most prior work on reinforcement learning with LTL specifications treats every new task independently, thus requiring large amounts of training data to generalize. We propose LTL-Transfer, a zero-shot transfer algorithm that composes task-agnostic skills learned during training to safely satisfy a wide variety of novel LTL task specifications. Experiments in Minecraft-inspired domains show that after training on only 50 tasks, LTL-Transfer can solve over 90% of 100 challenging unseen tasks and 100% of 300 commonly used novel tasks without violating any safety constraints. We deployed LTL-Transfer at the task-planning level of a quadruped mobile manipulator to demonstrate its zero-shot transfer ability for fetch-and-deliver and navigation tasks.