IROS 2024 论文列表

Trans-media robots, capable of operating across diverse environments, add significant complexity for multi-robot task allocation and planning problems. This paper introduces a novel approach to plan missions for such multi-robot systems, that addresses the associated specific complexities and constraints. It streamlines the overall mission planning process by decomposing it into tractable sub-problems, and addresses the issues of coalition formation, path planning, and task scheduling. It provides mission plans in very little computation time and allows to tackle large missions intractable by global planners, with negligible loss in plan optimality.

We propose a method for solving the strong form of the dynamic Cosserat rod clamped-free boundary value problem (BVP) under frictional contact. The shooting method is employed for BVP solving, and contact response is computed using the penalty method. The proposed method is tailored for accurate and rapid simulation of catheterization-like medical procedures. Our contact scenarios involve complex shapes and real patient geometry, modeled with implicit surfaces as blobby models. The results show that using blobby models for contact detection is orders of magnitude faster than using triangle meshes. This approach provides smoother contact forces, accelerating the convergence of the shooting method. We have made our code available as open source at https://gitlab.inria.fr/rjilani/IROS2024/.

Building a self-model for robots, enabling them to simulate their physical selves and predict future states without direct interaction with the physical world, is crucial for robot motion planning and control. Existing self-modeling methods primarily focus on rigid robots and typically require significant time, effort, and resources to gather training data. In this study, we introduce SoftNeRF, a self-supervised visual self-model designed for soft robots. We use a hybrid neural shape representation based on the Signed Distance Function (SDF) to capture both the geometry and complex nonlinear motion of soft robots. By leveraging differentiable rendering, our method learns a self-model from readily available RGB images, similar to how humans understand their physical state through reflection. To improve training efficiency and model accuracy, we propose an error-guided adaptive sampling strategy. SoftNeRF can serve as a plug-in for various downstream tasks, even when trained with data unrelated to those tasks. We demonstrate SoftNeRF’s ability to support shape prediction and motion planning for robots in both simulated and real-world environments. Furthermore, SoftNeRF excels in detecting and recovering from damage, thereby enhancing machine resilience. Code is available at: https://github.com/irmvlab/soft-nerf.

Differentiable physics simulation provides an avenue to tackle previously intractable challenges through gradient-based optimization, thereby greatly improving the efficiency of solving robotics-related problems. To apply differentiable simulation in diverse robotic manipulation scenarios, a key challenge is to integrate various materials in a unified framework. We present SoftMAC, a differentiable simulation framework that couples soft bodies with articulated rigid bodies and clothes. SoftMAC simulates soft bodies with the continuum-mechanics-based Material Point Method (MPM). We provide a novel forecast-based contact model for MPM, which effectively reduces penetration without introducing other artifacts like unnatural rebound. To couple MPM particles with deformable and non-volumetric clothes meshes, we also propose a penetration tracing algorithm that reconstructs the signed distance field in local area. Diverging from previous works, SoftMAC simulates the complete dynamics of each modality and incorporates them into a cohesive system with an explicit and differentiable coupling mechanism. The feature empowers SoftMAC to handle a broader spectrum of interactions, such as soft bodies serving as manipulators and engaging with underactuated systems. We conducted comprehensive experiments to validate the effectiveness and accuracy of the proposed differentiable pipeline in downstream robotic manipulation applications. Supplementary materials are available on our project website at https://damianliumin.github.io/SoftMAC.

Soft robotic fingers can safely grasp fragile or variable form objects, but their force capacity is limited, especially with less contact area: precision grasps and when objects are smaller or not spherical. Although most research focuses on improving force capacity through mechanical design modifications, optimizing grasping parameters for precision tasks remains crucial. To address this problem, this paper proposes an analytical rotational stability model for soft fingers’ precision grasping, considering grip failure involving slip and dynamic rotational stability. Comprehensive experiments across various objects, grip condition and types of fingers (PneuNet and commercial fingers) are conducted by examining the relationship between grasp parameters and model variables including coulomb friction, bulk stiffness, and dynamic stability. The findings demonstrate the model’s utility in identifying optimal grip parameters that enhance the force capacity of soft fingers without causing dynamic instability. This research contributes to the development of more effective and stable soft robotic fingers for precision grasping tasks.

This works exploits soft constraints in linear temporal logic task planning to enhance the agent’s capability in handling potentially conflicting or even infeasible tasks. Different from most existing works that focus on sticking to the original plan and trying to find a relaxed plan if the workspace does not permit, we augment the soft constraints to represent possible candidate sub-tasks that can be selected to fulfill the global task. Specifically, a hierarchical temporal logic specification is developed to represent LTL tasks with soft constraints and preferences. The hierarchical structure consists of an outer and inner layer, where the outer layer uses co-safe LTL to specify the task-level specifications and the inner layer specifies the low-level task-related atomic propositions via soft constraints. To cope with the hierarchical temporal logic specification, a hierarchical iterative search (HIS) algorithm is developed, which incrementally searches feasible atomic propositions and automaton states, and returns a task plan with minimum cost. Rigorous analysis shows that HIS based planning is feasible (i.e., the generated plan is applicable and satisfactory with respect to the task specification) and optimal (i.e, with minimum cost). Extensive simulation demonstrates the effectiveness of the proposed soft task planning approach.

Mobile robots are being used on a large scale in various crowded situations and become part of our society. The socially acceptable navigation behavior of a mobile robot with individual human consideration is an essential requirement for scalable applications and human acceptance. Deep Reinforcement Learning (DRL) approaches are recently used to learn a robot’s navigation policy and to model the complex interactions between robots and humans. We propose to divide existing DRL-based navigation approaches based on the robot’s exhibited social behavior and distinguish between social collision avoidance with a lack of social behavior and socially aware approaches with explicit predefined social behavior. In addition, we propose a novel socially integrated navigation approach where the robot’s social behavior is adaptive and emerges from the interaction with humans. The formulation of our approach is derived from a sociological definition, which states that social acting is oriented toward the acting of others. The DRL policy is trained in an environment where other agents interact socially integrated and reward the robot’s behavior individually. The simulation results indicate that the proposed socially integrated navigation approach outperforms a socially aware approach in terms of ego navigation performance while significantly reducing the negative impact on all agents within the environment.

Social navigation is a key consideration for integrating robots into human environments. Concurrently, it imposes heightened requisites: tasks must not only be executed succesfully without collisions, but also adhere to principles encompassing comprehensibility, courtesy, social compliance, comprehension, foresight, and scenario compliance. In this paper, we present the incorporation of social norms as a guiding framework for robot navigation within social contexts. We adopt field theory to provide a formal elucidation of the social norms, using Physical-Informed Neural Network (PINN) to predict pedestrian movement under the influence of social norms, respectively, and using Reinforcement Learning (RL) for navigation. We use supervised learning to train the pedestrian velocity field prediction model and reinforcement learning to train the navigation policy. We conduct three parts of experiments: (1) analyzing the spatiotemporal characteristics of the velocity field in the walking pedestrians dataset; (2) evaluating the accuracy of the vector field prediction in the pedestrian dataset; (3) using Gazebo simulation and the PEDSIM library to evaluate the improvement of navigation performance under constraints of social norms. Experiments have confirmed that the pedestrian motion data set indeed satisfies the Gaussian divergence theorem and can be described by the concept of field. The performance of navigation strategies incorporating social rules has been improved to a certain extent.

Navigating a robot among a crowd has received increasing attention from researchers over the last few decades, resulting in the emergence of numerous approaches aimed at addressing the problem of social navigation to date. Our proposed approach couples agent motion prediction and planning to avoid the freezing robot problem while simultaneously capturing multi-agent social interactions by utilizing a state-of-the-art trajectory prediction model i.e., social long short-term memory model (Social-LSTM). Leveraging the output of Social-LSTM for the prediction of future trajectories of pedestrians at each time-step given the robot’s possible future actions, our framework computes the optimal control action using Model Predictive Control (MPC) for the robot to navigate among pedestrians. We demonstrate the effectiveness of our proposed approach in multiple scenarios of simulated social navigation and compare it against several state-of-the-art reinforcement learning-based methods.

Interpolation of rigid body motions, or a general frame motion in Euclidean space, is a recurring topic in robotics. It boils down to generating trajectories in a Lie group, either SE (3) or SO (3) × ℝ3, with given initial and/or terminal values. To this end, spline interpolation schemes were developed where canonical coordinates are represented by cubic splines. They allow for prescribing initial velocity and acceleration only. In many robotic applications, terminal conditions are prescribed, however. In this paper, a novel interpolation scheme is presented that admits prescribing the terminal pose, velocity and acceleration, or the initial condition. As example, the scheme is applied to a rendezvous task of a UAV and describing the deformation of a Cosserat beam as relevant for soft robotics. The presented interpolation scheme can be directly applied to the motion parameterization in terms of (dual) quaternions.

The Vehicle Routing Problem with Drones (VRPD) seeks to optimize the routing paths for both trucks and drones, where the trucks are responsible for delivering parcels to customer locations, and the drones are dispatched from these trucks for parcel delivery, subsequently being retrieved by the trucks. Given the NP-Hard complexity of VRPD, numerous heuristic approaches have been introduced. However, improving solution quality, the definition of which can vary depending on various heuristic approaches, e.g., the total operation time, remain significant challenges. In this paper, we conduct a comprehensive examination of heuristic methods designed for solving VRPD, distilling and standardizing them into core elements. We then develop a novel reinforcement learning (RL) framework that is seamlessly integrated with the heuristic solution components, establishing a set of universal principles for incorporating the RL framework with heuristic strategies in an aim to improve both the solution quality and computation speed, regardless of how the solution quality is defined. This integration has been applied to a state-of-the-art heuristic solution for VRPD, showcasing the substantial benefits of incorporating the RL framework. Our evaluation results demonstrated that the heuristic solution incorporated with our RL framework not only elevated the quality of solutions but also achieved rapid computation speeds, especially when dealing with extensive customer locations.

Mobile robots have become extensively involved in human activities, taking on arduous tasks and providing significant assistance. Robot capabilities have been continuously enhanced, from simple chassis control to path planning and SLAM. Mixed criticality systems enable mobile robots to handle tasks of varying criticality by integrating multiple operating systems, allowing them to accomplish a wide range of tasks. However, besides improving robot computing performance, we should remember that robots are designed to serve humans. Reliability, usability, and affordability are all critical factors for robot design.We introduce SmartKit, a mixed criticality system (MCS) for mobile robots. Leveraging the efficiency in hardware utilization brought by virtualization, SmartKit can execute tasks of different criticality efficiently and securely. This paper will present the software and hardware architecture of SmartKit and provide performance and functionality validation of the robot system.

The multirotor unmanned aerial vehicle (UAV) requires precise control over thrust output when operating in wind-disturbed environments or executing intricate flight missions. Although current commercial force sensors offer high sensitivity and accuracy, they are often heavy and costly. These characteristics restrict their applicability in weight-sensitive and cost-sensitive scenarios, such as thrust measurement in UAVs. To overcome this difficulty, we have developed an embedded barometric force sensor (BFS) that mounts between the UAV’s airframe and motor, allowing direct measurement of the force exerted by the rotor on the UAV’s rigid body. The BFS is designed using low-cost MEMS barometers as tactile force sensors, encased in polyurethane rubber. Subsequently, we established its parameter model and devised a stability improvement strategy to reduce the impact of temperature. Additionally, we designed a structure suitable for mounting the BFS on the UAV to safeguard the rubber module from damage and reconstructed the thrust model to account for the impact of weight and friction on thrust measurement. Finally, we assembled testing platforms to validate the performance of the BFS. Experimental results demonstrate the BFS’s excellent linearity, wide range, adequate bandwidth to respond to UAV thrust variations, and confirm the feasibility of mounting the BFS on the UAV for thrust measurement and force feedback control.

Optical flow estimation is crucial for various applications in vision and robotics. As the difficulty of collecting ground truth optical flow in real-world scenarios, most of the existing methods of learning optical flow still adopt synthetic dataset for supervised training or utilize photometric consistency across temporally adjacent video frames to drive the unsupervised learning, where the former typically has issues of generalizability while the latter usually performs worse than the supervised ones. To tackle such challenges, we propose to leverage the geometric connection between optical flow estimation and stereo matching (based on the similarity upon finding pixel correspondences across images) to unify various real-world depth estimation datasets for generating supervised training data upon optical flow. Specifically, we turn the monocular depth datasets into stereo ones via synthesizing virtual disparity, thus leading to the flows along the horizontal direction; moreover, we introduce virtual camera motion into stereo data to produce additional flows along the vertical direction. Furthermore, we propose applying geometric augmentations on one image of an optical flow pair, encouraging the optical flow estimator to learn from more challenging cases. Lastly, as the optical flow maps under different geometric augmentations actually exhibit distinct characteristics, an auxiliary classifier which trains to identify the type of augmentation from the appearance of the flow map is utilized to further enhance the learning of the optical flow estimator. Our proposed method is general and is not tied to any particular flow estimator, where extensive experiments based on various datasets and optical flow estimation models verify its efficacy and superiority.

We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decision processes. The spectral decomposition brings about representation that can linearly represent the state-action value function induced by any policies, thus can be regarded as skills. The skill representations are transferable across arbitrary tasks with the same transition dynamics. Moreover, to handle the sim-to-real gap in the dynamics, we propose a skill discovery algorithm that learns new skills caused by the sim-to-real gap from real-world data. We promote the discovery of new skills by enforcing orthogonal constraints between the skills to learn and the skills from simulators, and then synthesize the policy using the enlarged skill sets. We demonstrate our methodology by transferring quadrotor controllers from simulators to Crazyflie 2.1 quadrotors. We show that we can learn the skill representations from a single simulator task and transfer these to multiple different real-world tasks including hovering, taking off, landing and trajectory tracking. Our skill discovery approach helps narrow the sim-to-real gap and improve the real-world controller performance by up to 30.2%.

This paper focuses on the acquisition of mapless navigation skills within unknown environments. We introduce the Skill Q-Network (SQN), a novel reinforcement learning method featuring an adaptive skill ensemble mechanism. Unlike existing methods, our model concurrently learns a high-level skill decision process alongside multiple low-level navigation skills, all without the need for prior knowledge. Leveraging a tailored reward function for mapless navigation, the SQN is capable of learning adaptive maneuvers that incorporate both exploration and goal-directed skills, enabling effective navigation in new environments. Our experiments demonstrate that our SQN can effectively navigate complex environments, exhibiting a 40% higher performance compared to baseline models. Without explicit guidance, SQN discovers how to combine low-level skill policies, showcasing both goal-directed navigations to reach destinations and exploration maneuvers to escape from local minimum regions in challenging scenarios. Remarkably, our adaptive skill ensemble method enables zero-shot transfer to out-of-distribution domains, characterized by unseen observations from non-convex obstacles or uneven, subterranean-like environments. The project page is available at https://sites.google.com/view/skill-q-net.

Understanding human actions from body poses is critical for assistive robots sharing space with humans in order to make informed and safe decisions about the next interaction. However, precise temporal localization and annotation of activity sequences is time-consuming and the resulting labels are often noisy. If not effectively addressed, label noise negatively affects the model’s training, resulting in lower recognition quality. Despite its importance, addressing label noise for skeleton-based action recognition has been overlooked so far. In this study, we bridge this gap by implementing a framework that augments well-established skeleton-based human action recognition methods with label-denoising strategies from various research areas to serve as the initial benchmark. Observations reveal that these baselines yield only marginal performance when dealing with sparse skeleton data. Consequently, we introduce a novel methodology, NoiseEraSAR, which integrates global sample selection, co-teaching, and Cross-Modal Mixture-of-Experts (CM-MOE) strategies, aimed at mitigating the adverse impacts of label noise. Our proposed approach demonstrates better performance on the established benchmark, setting new state-of-the-art standards. The source code for this study will be made accessible at https://github.com/xuyizdby/NoiseEraSAR.

In this study, we introduce a deep-learning approach for determining both the 6DoF pose and 3D size of strawberries, aiming to significantly augment robotic harvesting efficiency. Our model was trained on a synthetic strawberry dataset, which is automatically generated within the Ignition Gazebo simulator, with a specific focus on the inherent symmetry exhibited by strawberries. By leveraging domain randomization techniques, the model demonstrated exceptional performance, achieving an 84.77% average precision (AP) of 3D Intersection over Union (IoU) scores on the simulated dataset. Empirical evaluations, conducted by testing our model on real-world datasets, underscored the model’s viability for real-world strawberry harvesting scenarios, even though its training was based on synthetic data. The model also exhibited robust occlusion handling abilities, maintaining accurate detection capabilities even when strawberries were obscured by other strawberries or foliage. Additionally, the model showcased remarkably swift inference speeds, reaching up to 60 frames per second (FPS).

This paper presents a system that picks up protoplasts produced by removing the surrounding cell wall of root cells while preserving their positional information. The fundamental concept of this system involves scanning the root tip over time using a confocal microscopy to measure the positional information of each cell. Then, the protoplast pickup is conducted after switching to a brightfield microscopy to ensure the certainty of pickup. The system measures the position of single protoplasts, adjusts the position of the pipette using a 3-axis micromanipulator, and picks up the target protoplast using a microfluidic pump driven by a piezoelectric actuator. To automate this pickup process, we achieved calibration of the system. The fully automatic 3D calibration of the pipette tip was achieved, allowing 3D micromanipulation under the microscope with an accuracy of 3.1 μm in the XY-plane. Furthermore, by implementing multiple functions such as automatic detection of protoplasts, the process of protoplast pickup has been achieved.