ICRA 2024 Papers

In this work, we propose the Informed Batch Belief Trees (IBBT) algorithm for motion planning under motion and sensing uncertainties. The original stochastic motion planning problem is divided into a deterministic motion planning problem and a graph search problem. First, we solve the deterministic planning problem using Rapidly-exploring Random Graph (RRG) to construct a nominal trajectory graph. Then, an informed cost-to-go heuristic for the original problem is computed based on the nominal trajectory graph. Finally, we grow a belief tree by searching the graph using the proposed heuristic. IBBT interleaves batch state sampling, nominal trajectory graph construction, heuristic computing, and searching over the graph to find belief space motion plans. IBBT is an anytime, incremental algorithm. With an increasing number of batches of samples added to the graph, the algorithm finds improved plans. IBBT is efficient by reusing results between sequential iterations. The belief tree search is an ordered search guided by an informed heuristic. We test IBBT in different planning environments. Our numerical investigation confirms that IBBT finds non-trivial motion plans and is faster compared with previous similar methods.

Models with fewer parameters are necessary for the neural control of memory-limited, performant robots. Finding these smaller neural network architectures can be time-consuming. We propose HyperPPO, an on-policy reinforcement learning algorithm that utilizes graph hypernetworks to estimate the weights of multiple neural architectures simultaneously. Our method estimates weights for networks that are much smaller than those in common-use networks yet encode highly performant policies. We obtain multiple trained policies at the same time while maintaining sample efficiency and provide the user the choice of picking a network architecture that satisfies their computational constraints. We show that our method scales well - more training resources produce faster convergence to higher-performing architectures. We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2.1 quadrotor. Website: https://sites.google.com/usc.edu/hyperppo

A human foot with high degrees of freedom (DOF) that has multi-DOF toe joints and a two-DOF ankle provides multiple benefits, such as increased stride length and walking speed, impact mitigation, and enhanced balancing. However, creating such mechanisms for legged robots has been challenging due to increased complexity, heavy weight, and vulnerability to impact. In this paper, a novel leg and toe mechanism inspired by human biomechanics, featuring a one-DOF knee joint, two-DOF ankle joint, and one-DOF toe joint, is developed. All actuators are located at the proximal part of the thigh frame to minimize the distal mass. High payload timing belts and unique linkage mechanisms are utilized in the transmission to achieve high backdrivability and high joint stiffness. Actuation torques are intentionally coupled inspired by human anatomy, enduring the high propulsive force to the ground for dynamic movements, such as jumping. The implemented leg and toe mechanisms weigh 8.16 kg, and the height from the ground to the hip center is 786 mm. The proposed mechanism has been proven to be effective through force test and distance jump experiments.

Hydrodynamic interactions are crucial for understanding fish movement, particularly within the realm of robotic applications. Traditionally, many studies have favoured simplified travelling-wavy undulations derived from observed real fish kinematics. This approach often neglects higher-order undulations, thereby missing the subtleties of authentic fish movements. In this study, we utilised Computational Fluid Dynamics (CFD) to investigate the implications of using real fish kinematics in hydrodynamic interactions among schooling fish. We analysed two scenarios: one driven by real fish kinematics in spatiotemporal formations, and the other by travelling-wavy undulations inferred from the same real fish kinematics. Our results highlight the advantages of using real fish body kinematics for a more accurate representation of hydrodynamics in fish swimming. In contrast, the idealised travelling-wavy undulations tend to apply excessive force, displacing real fish more than expected. Additionally, the vortices and corresponding flow fields generated by real fish kinematics were found to be more stable than those arising from simplified travelling-wavy undulations. Our study underscores the significance of integrating real fish kinematics into robotic fish design and hydrodynamic studies in schooling fish.

For robotic transtibial prosthesis control, the global tibia kinematics can be used to monitor gait cycle progression and command smooth and continuous actuation. In this work, these global tibia kinematics define a phase variable impedance controller (PVIC), which is implemented as the nonvolitional base controller within a hybrid volitional control framework (PVI-HVC). The gait progression estimation and biomechanic performance of one able-bodied individual walking on a robotic ankle prosthesis via a bypass adapter are compared for three control schemes: benchmark passive controller, PVIC, and PVI-HVC. The different actuation of each had a direct effect on the global tibia kinematics, but the average deviation between the estimated and ground truth gait percentages were 1.6%, 1.8%, and 2.1%, respectively, for each controller. Both PVIC and PVI-HVC produced good agreement with able-bodied kinematic and kinetic references. As designed, PVI-HVC results were similar to those of PVIC when the user used low volitional intent, but yielded higher peak plantarflexion, peak torque, and peak power when the user commanded high volitional input in late stance. This additional torque and power also allowed the user to volitionally and continuously achieve activities beyond level walking, such as ascending ramps, avoiding obstacles, standing on tip-toes, and tapping the foot. In this way, PVI-HVC offers the kinetic and kinematic performance of the PVIC during level ground walking, along with the freedom to volitionally pursue alternative activities.

This paper presents a 6-DOF hybrid robot for percutaneous needle intervention procedures. The new robot combines the advantages of both serial robots and parallel robots, featuring compactness, high accuracy, and small footprint while overcoming the problems of the high cost of serial robots and the small workspace and singularity issue of parallel robots. Besides, by analyzing the workspace of the robot, the equation is derived between the structure parameter and workspace to adjust the parameters of the robot to satisfy different working scenes. According to the experiment, the accuracy of the robot is related to the position, distance, and insertion angle. The result shows that the performance is better when working near the center workspace and away from the servos and the average error of the robot is 1.39mm. The phantom experiment of lumbar puncture validates its feasibility.

There is a lack of cleaning robots dedicated to the scrubbing of contaminated surfaces. Contaminated surfaces in domestic and industrial settings typically require manual scrubbing which can be costly or hazardous. There is growing demand for automated sanitization systems in hospitals, food-processing plants, and other settings where cleanliness of surfaces is important. To address the opportunity to automate the scrubbing of surfaces, this work focuses on the use of series elastic actuators which can apply consistent trajectories of scrubbing force. Consistent force during scrubbing increases the rate of removal for a contaminant. An elastic robot which has rigid links and low-stiffness joints can perform friction-based cleaning of surfaces with complex geometries while maintaining consistent scrubbing force. This study uses a hybrid force-position control scheme and a low-cost elastic robot to perform scrubbing. This study observes the relationship between joint stiffness in the robot and the disturbance rejection for forcebased control during scrubbing.

This paper presents the hardware design and development of Hyblock, a modular hydraulic robot for heavy-duty application such as construction. The robot is equipped with a simple docking mechanism called a C-type expansion dowel and a novel hydraulic circuit MHSB that matches the modular structure. In this paper, we first report on the design of the robot hardware including the dowel and hydraulic circuit, then present preliminary experiments on pressure-based torque control and docking control using proximal magnetic sensors. Next, we propose a framework for dynamic reconfiguration and task-space motion control built on the concept of dowel connectors. Simulation results demonstrate that a collective modular robot achieves desired motion tasks while keeping all normal contact forces of the connectors being lower-bound. The results are also explained in the supplementary video.

Transferring human motion skills to humanoid robots remains a significant challenge. In this study, we introduce a Wasserstein adversarial imitation learning system, allowing humanoid robots to replicate natural whole-body locomotion patterns and execute seamless transitions by mimicking human motions. First, we present a unified primitive-skeleton motion retargeting to mitigate morphological differences between arbitrary human demonstrators and humanoid robots. An adversarial critic component is integrated with Reinforcement Learning (RL) to guide the control policy to produce behaviors aligned with the data distribution of mixed reference motions. Additionally, we employ a specific Integral Probabilistic Metric (IPM), namely the Wasserstein-1 distance with a novel soft boundary constraint to stabilize the training process and prevent model collapse. Our system is evaluated on a full-sized humanoid JAXON in the simulator. The resulting control policy demonstrates a wide range of locomotion patterns, including standing, push-recovery, squat walking, humanlike straight-leg walking, and dynamic running. Notably, even in the absence of transition motions in the demonstration dataset, the robot showcases an emerging ability to transit naturally between distinct locomotion patterns as desired speed changes.

In this paper, we present a novel system for Human-Robot Interactive Creation of Artworks (HRICA). Different from previous robot painters, HRICA allows a human user and a robot to alternately draw strokes on a canvas, to collaboratively create a portrait drawing through frequent interactions. The key is to enable the robot to understand human intentions, during the interactive creation process. We here formulate this as a mask-free image inpainting problem, and propose a novel method to estimate the complete version of a portrait drawing, after the human user has drawn some initial strokes. In this way, the robot can select some complementary strokes and draw them on the canvas. To train and evaluate our inpainting method, we construct a novel large-scale portrait drawing dataset, CelebLine, which composes of high-quality portrait line-drawings, with dense labels of both 2D semantic parsing masks and 3D depth maps. Finally, we develop a human-robot interactive drawing system with low-cost hardware, user-friendly interface, and interesting creation experience. Experiments show that our robot can stably cooperate with human users to create diverse styles of portrait drawings. In addition, our portrait drawing inpainting method significantly outperforms previous advanced methods. The code and dataset have been released at: https://github.com/fei-aiart/HRICA.

Deep reinforcement learning (RL) has shown promising results in robot motion planning with first attempts in human-robot collaboration (HRC). However, a fair comparison of RL approaches in HRC under the constraint of guaranteed safety is yet to be made. We, therefore, present human-robot gym, a benchmark suite for safe RL in HRC. We provide challenging, realistic HRC tasks in a modular simulation framework. Most importantly, human-robot gym is the first benchmark suite that includes a safety shield to provably guarantee human safety. This bridges a critical gap between theoretic RL research and its real-world deployment. Our evaluation of six tasks led to three key results: (a) the diverse nature of the tasks offered by human-robot gym creates a challenging benchmark for state-of-the-art RL methods, (b) by leveraging expert knowledge in form of an action imitation reward, the RL agent can outperform the expert, and (c) our agents negligibly overfit to training data.

This paper addresses human-robot collaborative (HRC) precision assembly that complements natural human ability and the strength of an autonomous robot system. Our approach enables both flexibility and efficiency of tight-clearance assembly of various complex-shaped parts in the presence of uncertainty without requiring assembly skills and knowledge of robotics from the human operator. We demonstrated the effectiveness of our approach in a variety of experiments and comparisons with other HRC assembly approaches.

Preference-based optimizing methods have shown their advantages and potential in exploring individual, comfortable, and effective control strategies and assistance parameters of exoskeletons during locomotion. Research indicates that compared with naive wearers, knowledgeable wearers with abundant exoskeleton assistance experience have obvious advantages in speeding up the parameters exploration process and improving the assistant performance. However, there is no existing method that could utilize the human-exoskeleton locomotion interaction experience (HELIE) to assist naive wearers during the exploration process. In this work, we propose a novel preference-based human-exoskeleton locomotion interaction experience transfer (LIET) framework, which could speed up the exploration of human-preferred parameters and acquire more satisfying results for naive wearers via the HELIE acquired from knowledgeable wearers. In addition, based on the proposed LIET framework, we establish the mathematical expression of the HELIE transfer during exoskeleton assistance. This will promote the research that concerns utilizing HELIE for exoskeleton control parameters optimizations in the future. Finally, experiments demonstrate the proposed LIET framework could speed up the exploration process and acquire more satisfying optimized results for naive wearers.

Current methods of deploying robots that operate in dynamic, uncertain environments, such as Uncrewed Aerial Systems in search & rescue missions, require nearly continuous human supervision for vehicle guidance and operation. These methods do not consider high-level mission context resulting in cumbersome manual operation or inefficient exhaustive search patterns. We present a human-centered autonomous frame-work that infers geospatial mission context through dynamic feature sets, which then guides a probabilistic target search planner. Operators provide a set of diverse inputs, including priority definition, spatial semantic information about ad-hoc geographical areas, and reference waypoints, which are probabilistically fused with geographical database information and condensed into a geospatial distribution representing an operator’s preferences over an area. An online, POMDP-based planner, optimized for target searching, is augmented with this reward map to generate an operator-constrained policy. Our results, simulated based on input from five professional rescuers, display effective task mental model alignment, 18% more victim finds, and 15 times more efficient guidance plans then current operational methods.

Reinforcement Learning (RL) has been widely used to create generalizable autonomous vehicles. However, they rely on fixed reward functions that struggle to balance values like safety and efficiency. How can autonomous vehicles balance different driving objectives and human values in a constantly changing environment? To bridge this gap, we propose an adaptive reward function that utilizes visual attention maps to detect pedestrians in the driving scene and dynamically switch between prioritizing safety or efficiency depending on the current observation. The visual attention map is used to provide spatial attention to the RL agent to boost the training efficiency of the pipeline. We evaluate the pipeline against variants of an occluded pedestrian crossing scenario in the CARLA Urban Driving simulator. Specifically, the proposed pipeline is compared against a modular setup that combines the well-established object detection model, YOLO, with a Proximal Policy Optimization (PPO) agent. The results indicate that the proposed approach can compete with the modular setup while yielding greater training efficiency. The trajectories collected with the approach confirm the effectiveness of the proposed adaptive reward function.

While surgical robots, such as the da Vinci Surgical System, have become prevalent in minimally invasive surgery, they are predominantly used by the human operator to directly teleoperate the tools. This paper aims to analyse the different methods of human robot shared control in the surgical domain. We propose a reinforcement learning algorithm, transverse generative adversarial imitation learning (tGAIL), which is employed to train the robot from the expert’s demonstration and show competitive generalization ability compared to inverse reinforcement learning and conventional GAIL. We then propose a priority-changing shared control method to effectively combine the surgeon and robot’s strengths by dynamically adjusting control priority based on the deviation distance. We show that using this method in a supervision framework boosts the performance of the human operator when completing the peg transfer task. By learning from the expert and collaborating with the human during the task, the intelligent agent can help to reduce operation time by 31.7% and the human input by 60.5% compared to direct teleoperation.

Shared electric micromobility has surged to a popular model of urban transportation due to its efficiency in short-distance trips and environmentally friendly characteristics compared to traditional automobiles. However, managing thousands of shared electric micromobility vehicles including rebalancing and charging to meet users’ travel demands still has been a challenge. Existing methods generally ignore human preferences in vehicle selection and assume all nearby vehicles have an equal chance of being selected, which is unrealistic based on our findings. To address this problem, we design PERCEIVE, a human preference-aware rebalancing and charging framework for shared electric micromobility vehicles. Specifically, we model human preferences in vehicle selection based on vehicle usage history and current status (e.g., energy level) and incorporate the vehicle selection model into a robust adversarial reinforcement learning framework. We further utilize conformal prediction to quantify human preference uncertainty and fuse it with the reinforcement learning framework. We evaluate our framework using two months of real-world electric micromobility operation data in a city. Experimental results show that our method achieves a performance gain of at least 4.02% in the net revenue and offers more robust performance in worst-case scenarios compared to state-of-the-art baselines.

In the burgeoning field of autonomous vehicles (AVs), trajectory prediction remains a formidable challenge, especially in mixed autonomy environments. Traditional approaches often rely on computational methods such as time-series analysis. Our research diverges significantly by adopting an interdisciplinary approach that integrates principles of human cognition and observational behavior into trajectory prediction models for AVs. We introduce a novel “adaptive visual sector” mechanism that mimics the dynamic allocation of attention human drivers exhibit based on factors like spatial orientation, proximity, and driving speed. Additionally, we develop a “dynamic traffic graph” using Convolutional Neural Networks (CNN) and Graph Attention Networks (GAT) to capture spatio-temporal dependencies among agents. Benchmark tests on the NGSIM, HighD, and MoCAD datasets reveal that our model (GAVA) outperforms state-of-the-art baselines by at least 15.2%, 19.4%, and 12.0%, respectively. Our findings underscore the potential of leveraging human cognition principles to enhance the proficiency and adaptability of trajectory prediction algorithms in AVs.

Control barrier functions (CBFs) have become popular as a safety filter to guarantee the safety of nonlinear dynamical systems for arbitrary inputs. However, it is difficult to construct functions that satisfy the CBF constraints for high relative degree systems with input constraints. To address these challenges, recent work has explored learning CBFs using neural networks via neural CBFs (NCBFs). However, such methods face difficulties when scaling to higher dimensional systems under input constraints. In this work, we first identify challenges that NCBFs face during training. Next, to address these challenges, we propose policy neural CBFs (PNCBFs), a method of constructing CBFs by learning the value function of a nominal policy, and show that the value function of the maximum-over-time cost is a CBF. We demonstrate the effectiveness of our method in simulation on a variety of systems ranging from toy linear systems to a jet aircraft with a 16-dimensional state space. Finally, we validate our approach on a two-agent quadcopter system on hardware under tight input constraints.