IROS 2024 Papers

This paper presents a generic trajectory planning method for wheeled robots with fixed steering axes while the steering angle of each wheel is constrained. In the existing literatures, All-Wheel-Steering (AWS) robots, incorporating modes such as rotation-free translation maneuvers, in-situ rotational maneuvers, and proportional steering, exhibit inefficient performance due to time-consuming mode switches. This inefficiency arises from wheel rotation constraints and inter-wheel cooperation requirements. The direct application of a holonomic moving strategy can lead to significant slip angles or even structural failure. Additionally, the limited steering range of AWS wheeled robots exacerbates non-linearity characteristics, thereby complicating control processes. To address these challenges, we developed a novel planning method termed Constrained AWS (C-AWS), which integrates second-order discrete search with predictive control techniques. Experimental results demonstrate that our method adeptly generates feasible and smooth trajectories for C-AWS while adhering to steering angle constraints. Code and video can be found at https://github.com/Rex-sys-hk/AWSPlanning.

A typical manipulation task consists of a manipulator equipped with a gripper to grasp and move an object with constraints on the motion of the hand-held object, which may be due to the nature of the task itself or from object-environment contacts. In this paper, we study the problem of computing joint torques and grasping forces for time-optimal motion of an object, while ensuring that the grasp is not lost and any constraints on the motion of the object, either due to dynamics, environment contact, or no-slip requirements, are also satisfied. We present a second-order cone program (SOCP) formulation of the time-optimal trajectory planning problem that considers nonlinear friction cone constraints at the hand-object and object-environment contacts. Since SOCPs are convex optimization problems that can be solved optimally in polynomial time using interior point methods, we can solve the trajectory optimization problem efficiently. We present simulation results on three examples, including a non-prehensile manipulation task, which shows the generality and effectiveness of our approach.

In dynamic open-world environments, agents continually face new challenges due to sudden and unpredictable novelties, hindering Task and Motion Planning (TAMP) in autonomous systems. We introduce a novel TAMP architecture that integrates symbolic planning with reinforcement learning to enable autonomous adaptation in such environments, operating without human guidance. Our approach employs symbolic goal representation within a goal-oriented learning framework, coupled with planner-guided goal identification, effectively managing abrupt changes where traditional reinforcement learning, re-planning, and hybrid methods fall short. Through sequential novelty injections in our experiments, we assess our method’s adaptability to continual learning scenarios. Extensive simulations conducted in a robotics domain corroborate the superiority of our approach, demonstrating faster convergence to higher performance compared to traditional methods. The success of our framework in navigating diverse novelty scenarios within a continuous domain underscores its potential for critical real-world applications.

Haptic devices have shown to be valuable in supplementing surgical training, especially when providing haptic feedback based on user performance metrics such as wrench applied by the user on the tool. However, current 6-axis force/torque sensors are prohibitively expensive. This paper presents the design and calibration of a low-cost, six-axis force/torque sensor specially designed for laparoscopic haptic training applications. The proposed design uses Hall-effect sensors to measure the change in the position of magnets embedded in a silicone layer that results from an applied wrench to the device. Preliminary experimental validation demonstrates that these sensors can achieve an accuracy of 0.45 N and 0.014 Nm, and a theoretical XY range of ±50N, Z range of ±20N, and torque range of ±0.2Nm. This study indicates that the proposed low-cost 6-axis force/torque sensor can accurately measure user force and provide useful feedback during laparoscopic training on a haptic device.

Exploring the limits of quadruped robot agility, particularly in the context of rapid and real-time planning and execution of omnidirectional jump trajectories, presents significant challenges due to the complex dynamics involved, especially when considering significant impulse contacts. This paper introduces a new framework to enable fast, omnidirectional jumping capabilities for quadruped robots. Utilizing minimum jerk technology, the proposed framework efficiently generates jump trajectories that exploit its analytical solutions, ensuring numerical stability and dynamic compatibility with minimal computational resources. The virtual model control is employed to formulate a Quadratic Programming (QP) optimization problem to accurately track the Center of Mass (CoM) trajectories during the jump phase. The whole-body control strategies facilitate precise and compliant landing motion. Moreover, the different jumping phase is triggered by time-schedule. The framework’s efficacy is demonstrated through its implementation on an enhanced version of the open-source Mini Cheetah robot. Omnidirectional jumps-including forward, backward, and other directional-were successfully executed, showcasing the robot’s capability to perform rapid and consecutive jumps with an average trajectory generation and tracking solution time of merely 50 microseconds.

Legged robot proved their capability to cross complex terrain in recent research, yet the autonomy of robots on discrete terrain still needs to be enhanced since it requires a full stack framework. This paper introduces a real-time motion and foothold planning framework tailored for legged robots navigating uneven terrains, such as stepping stones. Our approach addresses the critical challenges of determining feasible global paths and local footholds to enhance autonomous mobility across complex landscapes. By using a sampling-based global path planner integrated with terrain segmentation and the robot’s kinematic model, our framework swiftly generates viable navigation paths. Concurrently, it utilizes a Mixed Integer Programming (MIP) methodology for real-time foothold optimization, ensuring the robot’s stability and safety through dynamic terrain interaction. Finally, an execution layer including Model Predictive Control (MPC) and Whole-Body Control (WBC) generates the robots’ motion. Simulation and real-world experiments demonstrate that our framework improves legged robots’ adaptability on discrete terrains.

Tactile information is the research hotspot of wearable flexible sensors due to its importance and complexity. With the innovation of wearable technology and robotics in healthcare, researchers are increasingly integrating wearable flexible sensors on the front end of robots to reproduce the hand tactile manipulation of human tissues. Therefore, it is hoped to develop a thin-film sensor that can be deployed in a small area to assist robots in surgery and data collection of human tissues. Here we use a one-step injection method to fabricate a novel composite sensor based on liquid metal. By laminating multiple PDMS microfluidic layers, the two parameters of pressure and deformation are measured simultaneously in a decoupled manner. The sensor is small and thin, making it easy to integrate into fingers/robot fingers for assistance. The finger/robot finger exerts pressure on the sensor and the sensor deforms with the material to identify the hardness of the material being touched. Separate performance tests of the two sensors show that the strain and pressure functions are decoupled from each other, and their ratios can identify and classify the hardness of different touched materials (glass, PDMS and silicone). This novel composite sensor we proposed can assist robots in manipulating human tissues during medical surgeries. At the same time, its function in tactile information feedback also has broad applications in medical treatment, rehabilitation and services.

In recent years, research on aerial grasping, manipulation, and transportation of objects has garnered significant attention. These tasks often require UAVs to operate safely close to environments or objects and to efficiently grasp payloads. However, current widely adopted flying platforms pose safety hazards: unprotected high-speed rotating propellers can cause harm to the surroundings. Additionally, the space for carrying payloads on the fuselage is limited, and the restricted position of the payload also hinders efficient grasping. To address these issues, this paper presents a coaxial ducted fan UAV which is equipped with electromagnets mounted externally on the fuselage, enabling safe grasping and transfer of multiple loads in midair without complex additional actuators. It also has the capability to achieve direct human-UAV cargo transfer in the air. The forces acting on the loads during magnetic attachment and their influencing factors were analyzed. An ADRC controller is utilized to counteract disturbances during grasping and achieve attitude control. Finally, flight tests are conducted to verify the UAV’s ability to directly grasp multiple loads from human hands in flight while maintaining attitude tracking.

Point cloud registration refers to the problem of finding the rigid transformation that aligns two given point clouds, and is crucial for many applications in robotics and computer vision. The main insight of this paper is that we can directly optimize the point cloud registration problem without correspondences by utilizing an algorithmically simple, yet computationally complex, semi-exhaustive search approach that is very well-suited for parallelization on modern GPUs. Our proposed algorithm, Direct Semi-Exhaustive Search (DSES), iterates over potential rotation matrices and efficiently computes the inlier-maximizing translation associated with each rotation. It then computes the optimal rigid transformation based on any desired distance metric by directly computing the error associated with each transformation candidate {R,t}. By leveraging the parallelism of modern GPUs, DSES outperforms state-of-the-art methods for partial-to-full point cloud registration on the simulated ModelNet40 benchmark and demonstrates high performance and robustness for pose estimation on a real-world robotics problem (https://youtu.be/q0q2-s2KSuA).

In this paper, we address the problem of estimating the rotational extrinsics, as well as the scale factors of two gyroscopes rigidly mounted on the same device. In particular, we formulate the problem as a least-squares minimization and introduce a direct algorithm that computes the estimated quantities without any iterations, hence avoiding local minima and improving efficiency. Furthermore, we show that the rotational extrinsics are observable while the scale factors can be determined up to a global scale for general configurations of the gyroscopes. To this end, we also study special placements of the gyroscopes where a pair, or all, of their axes are parallel and analyze their impact on the scale factors’ observability. Lastly, we evaluate our algorithm in simulation and real-world experiments to assess its performance as a function of key motion and sensor characteristics.

Predicting accurate observations efficiently from novel views is a key requirement for several robotics applications. Existing shape and surface representations, however, either require expensive ray-tracing operations, e.g., in the case of meshes or signed distance functions (SDFs), or offer only a coarse view, e.g., in the case of quadrics or point clouds. We develop a new representation that captures viewing direction and enables fast novel view synthesis. Our first contribution is a signed directional distance function (SDDF) that extends the SDF definition by measuring distance in a desired viewing direction rather than to the nearest point. As a result, SDDF removes post-processing steps for view synthesis required by SDF, such as surface extraction via marching cubes or rendering via sphere tracing, and allows ray-tracing through a single function call. SDDF also encodes by construction the property that distance decreases linearly along the viewing direction. We show that this enables dimensionality reduction in the function representation and guarantees the prediction accuracy independent of the distance to the surface. Recent advances demonstrate impressive performance of deep neural networks for shape learning, including IGR for SDF, Occupancy Networks for occupancy, AtlasNet for meshes, and NeRF for density. Our second contribution, DeepSDDF, is a deep neural network model for SDDF shape learning. Similar to IGR, we show that DeepSDDF can model whole object categories and interpolate or complete shapes from partial views.

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1% while reducing task completion time by 7.4%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.

Achieving complete area coverage in robotics is an essential aspect for applications such as cleaning and patrolling. While multi-agent frameworks have been implemented to address the challenge of complete coverage, the area coverage performances are hindered by physical constraints and dynamic obstacles that cause inaccessibility to certain areas of the environment. Reconfigurable robots have been adopted to mitigate this issue as the independent alteration of the morphologies during deployments enables overcoming tight spaces to access obstructed areas. Hence, this paper proposes a Multi-Agent Reinforcement Learning (MARL) framework leveraging the Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to enable a team of reconfigurable robots to achieve complete coverage under the presence of dynamic obstacles. The framework is modelled to allow the robots to coordinate and plan their motions effectively while using shape adaptability to access narrow spaces while avoiding dynamic obstacles. Experimental results demonstrated the framework’s ability to be generalised even when scaled up to a different number of agents across larger environments.

Autonomous driving is a highly anticipated approach toward eliminating roadway fatalities. At the same time, the bar for safety is both high and costly to verify. This work considers the role of remotely-located human operators supervising a fleet of autonomous vehicles (AVs) for safety. Such a ‘scalable supervision’ concept was previously proposed to bridge the gap between still-maturing autonomy technology and the pressure to begin commercial offerings of autonomous driving. The present article proposes DISCES, a framework for Data-Informed Safety-Critical Event Simulation, to investigate the practicality of this concept from a dynamic network loading standpoint. With a focus on the safety-critical context of AVs merging into mixed-autonomy traffic, vehicular arrival processes at 1,097 highway merge points are modeled using microscopic traffic reconstruction with historical data from interstates across three California counties. Combined with a queuing theoretic model, these results characterize the dynamic supervision requirements and thereby scalability of the teleoperation approach. Across all scenarios we find reductions in operator requirements greater than 99% as compared to in-vehicle supervisors for the time period analyzed. The work also demonstrates two methods for reducing these empirical supervision requirements: (i) the use of cooperative connected AVs — which are shown to produce an average 3.67 orders-of-magnitude system reliability improvement across the scenarios studied — and (ii) aggregation across larger regions.

Localization is one of the most important tasks for mobile robot operations. Without such capability, a robot may wander toward unsafe states and never complete a desired task. Such capability is even more important in multi-robot system (MRS) operations in which their motion is coordinated based on consensus schemes that leverage information from surrounding neighbors. Thus, in the event of compromised or malfunctioning on-board positioning sensing (e.g., due to cyber attacks or faults) on individual robots, the entire robotic system may be hijacked toward undesired states. In this work, we target this problem by proposing a decentralized framework where: i) robots with loss of localization capabilities detect the anomalous behavior then generate a notification signal within information exchanges to alert neighboring robots, and ii) neighboring robots leverage their mobility to aid in recovery allowing compromised robots to re-localize. Our framework is validated in simulations and lab experiments on proximity-based formations of homogeneous unmanned multi-robot swarms.

In this paper, we introduce a novel convex formulation that seamlessly integrates the Material Point Method (MPM) with articulated rigid body dynamics in frictional contact scenarios. We extend the linear corotational hyperelastic model into the realm of elastoplasticity and include an efficient return mapping algorithm. This approach is particularly effective for MPM simulations involving significant deformation and topology changes, while preserving the convexity of the optimization problem. Our method ensures global convergence, enabling the use of large simulation time steps without compromising robustness. We have validated our approach through rigorous testing and performance evaluations, highlighting its superior capabilities in managing complex simulations relevant to robotics. Compared to previous MPM-based robotic simulators, our method significantly improves the stability of contact resolution — a critical factor in robot manipulation tasks. We make our method available in the open-source robotics toolkit, Drake. The supplemental video is available here.

Topological semantic maps provide a practical solution to enhance indoor navigation for the Partially Sighted or Visually Impaired (PSVI). Segmenting indoor floor plans and extracting boundaries are key to constructing these maps. The existing methods exhibit low accuracy in segmentation. To achieve desired high segmentation accuracy, we introduce a Context-Enhanced Full-Resolution Network (CEFRN) for floor plan segmentation. It is designed to harness the shallow detailed features and inter-category contextual dependencies inherent in floor plans. CEFRN integrates modified residual blocks to capture the low-stage full-resolution features while maintaining its compactness. A position attention module is employed to refine the deep-stage contextual information. We also propose a two-dimensional deep supervision method to merge features from both stages, which significantly boosts the feature representation ability of CEFRN. Finally, a practical topological semantic mapping method for PSVI indoor navigation is introduced. Experimental results demonstrate that CEFRN’s segmentation accuracy well exceeds the state-of-the-art methods’. It can be used to well support accurate topological semantic mapping.

In this paper, a novel approach is proposed for learning robot control in contact-rich tasks such as wiping, by developing Diffusion Contact Model (DCM). Previous methods of learning such tasks relied on impedance control with time-varying stiffness tuning by performing Bayesian optimization by trial-and-error with robots. The proposed approach aims to reduce the cost of robot operation by predicting the robot contact trajectories from the variable stiffness inputs and using neural models. However, contact dynamics are inherently highly nonlinear, and their simulation requires iterative computations such as convex optimization. Moreover, approximating such computations by using finite-layer neural models is difficult. To overcome these limitations, the proposed DCM used the denoising diffusion models that could simulate the complex dynamics via iterative computations, thus improving the prediction accuracy. Stiffness tuning experiments conducted in simulated and real environments showed that the DCM achieved comparable performance to a conventional robot-based optimization method while reducing the number of robot trials.

Offloading software modules to the edge/cloud can enhance a robot’s capabilities by leveraging massive computing power. However, determining which software module should be offloaded and scheduled to which robot/edge/cloud node is a challenging task, particularly for robot fleets with diverse tasks. In this paper, we tackle the software scheduling problem and introduce a taxonomy to categorize software modules and classify their applicability and requirements for offloading. Additionally, by using prior measurements, we model the compute cluster and formalize software scheduling as a multi-objective optimization problem which we tackle with a genetic algorithm. To evaluate our approach with a challenging setup, we build a mobile manipulation task using open-source frameworks and libraries in the Robot Operating System (ROS2) community in simulation as well as a mildly simplified real-world variant. Our evaluation shows significant improvements compared to the built-in scheduler of Kubernetes (K8s) regarding robotic specific metrics such as the rate of missed cycle time in both simulated and real-world experiments.