IROS 2023 Papers

Differential Dynamic Programming (DDP) is an efficient computational tool for solving nonlinear optimal control problems. It was originally designed as a single shooting method and thus is sensitive to the initial guess supplied. This work considers the extension of DDP to multiple shooting (MS), improving its robustness to initial guesses. A novel derivation is proposed that accounts for the defect between shooting segments during the DDP backward pass, while still maintaining quadratic convergence locally. The derivation enables unifying multiple previous MS algorithms, and opens the door to many smaller algorithmic improvements. A penalty method is introduced to strategically control the step size, further improving the convergence performance. An adaptive merit function and a more reliable acceptance condition are employed for globalization. The effects of these improvements are benchmarked for trajectory optimization with a quadrotor, an acrobot, and a manipulator. MS-DDP is also demonstrated for use in Model Predictive Control (MPC) for dynamic jumping with a quadruped robot, showing its benefits over a single shooting approach.

Powered lower-limb prostheses have the potential to improve amputee mobility by closely imitating the biomechanical function of the missing biological leg. To accomplish this goal, powered prostheses need controllers that can seamlessly adapt to the ambulation activity intended by the user. Most powered prosthesis control architectures address this issue by switching between specific controllers for each activity. This approach requires online classification of the intended ambulation activity. Unfortunately, any misclassification can cause the prosthesis to perform a different movement than the user expects, increasing the likelihood of falls and injuries. Therefore, classification approaches require near-perfect accuracy to be used safely in real life. In this paper, we propose a unified controller for powered knee prostheses which allows for walking, stair ascent, and stair descent without the need for explicit activity classification. Experiments with one individual with an above-knee amputation show that the proposed controller enables seamless transitions between activities. Moreover, transition between activities is possible while leading with either the sound-side or the prosthesis. A controller with these characteristics has the potential to improve amputee mobility.

Multi-Agent Reinforcement Learning (MARL) has become a promising solution for constructing a multi-agent autonomous driving system (MADS) in complex and dense scenarios. But most methods consider agents acting selfishly, which leads to conflict behaviors. Some existing works incorporate the concept of social value orientation (SVO) to promote coordination, but they lack the knowledge of other agents' SVOs, resulting in conservative maneuvers. In this paper, we aim to tackle the mentioned problem by enabling the agents to understand other agents' SVOs. To accomplish this, we propose a two-stage system framework. Firstly, we train a policy by allowing the agents to share their ground truth SVOs to establish a coordinated traffic flow. Secondly, we develop a recognition network that estimates agents' SVOs and integrates it with the policy trained in the first stage. Experiments demonstrate that our developed method significantly improves the performance of the driving policy in MADS compared to two state-of-the-art MARL algorithms.

Tactile sensors based on optical waveguides are highly sensitive to pressure, possess good chemical inertness and electromagnetic resistance, and are unaffected by temperature changes in the surrounding environment. Researchers have developed various waveguide structures with multi-level cores to simultaneously measure tactile forces and positions. However, these designs result in thicker waveguides and reduced sensitivity in the lower levels. This study introduces a two-dimensional reticular core optical waveguide for tactile force and positioning sensing, where vertical waveguides intersect each other. The reticular core reduces waveguide thickness and simplifies fabrication processes. The simulation investigates the characteristics of light propagation and geometric parameters. Experimental results confirm the proposed reticular waveguide's force-sensing capability, with an average sensitivity of 0.36 dB/N. Compared to the split-level structure, the reticular waveguide demonstrates more consistent sensitivities along the two shear directions. Utilizing a deep neural network, the spatial resolution achieves approximately 0.72 mm along the X-axis and 1.14 mm along the Y-axis, outperforming the split-level structure.

Is mobile robot navigation a solved problem? We asked this question to 14 professional robot software engineers who work with navigation stacks of mobile, wheeled robots on a daily basis. They unanimously report that it remains challenging to ensure the performance of their mobile robots. We find that the method of choice to verify a robot's performance is to expose it to different environments under varying conditions. Unfortunately, these field tests are costly to set up and often too risky to execute. Therefore, robot software engineers want to replicate real-world environments in simulated sceneries (i) to test their navigation stack or parts of it prior to deployment; and (ii) to reproduce erroneous behaviour observed in the real world. Motivated by these insights, we have developed a domain-specific language and associated tooling which enables engineers (i) to specify sceneries of indoor environments; and (ii) to automatically generate variants thereof that resemble changes in the real world. We demonstrate how our approach enables the simulation-based replication of real-world environments and allows us to recreate erroneous robot behaviour as reported in the interviews. While performing simulation-based tests, we discovered an 8-year dormant bug in the ROS navigation stack.

This paper presents the design, characterization, and testing of a steerable needle robot for minimally invasive neurosurgery. The robot consists of a rigid outer tube and two telescopic tendon-driven steerable tubes. Through the rotation, translation, and bending of individual tubes, this telescopic tendon-driven needle robot can perform dexterous motion and follow the path of the tip. We presented the design of the needle robot and its actuation system, modeling of the robotic kinematics, characterization of the robot motion, results of the open-loop kinematic control, and demonstration of the follow-the-leader motion. The position error of the robot tip is 0.92 mm, and follow-the-leader motion error is 1.1 mm. Due to its small footprint and unique motion ability, the robot has the potential to be manipulated inside human brain and used for minimally invasive neurosurgery.

Real-time magnetic resonance imaging (MRI) in-terventions are significantly impacted by material compatibility problems and size constraints in the MRI bore. Limited physi-cian access to patients within the bore of the MRI necessitates iterative positioning and imaging, which prolongs the duration of the procedure and increases patient risk. We present a passive MR-safe haptic teleoperation device for prostate needle biopsy inside an MRI machine. The device uses a low-friction hydrostatic transmission based on paired rolling diaphragm actuators, linear and rotary. The device has two degrees of freedom, allowing needle insertion and rotation. The robot produces negligible MR imaging artifacts, has effective positioning tracking, and can reliably detect needle punctures in the clinically relevant range. We describe the design components, system transparency, and perform a needle insertion test.

Unmanned aerial vehicles (UAVs) have revolutionised various industries, such as agriculture, remote sensing, and infrastructure inspection. To explore new designs and improve UAV flight performance, roboticists are seeking inspiration from nature. In this paper, we present a bioinspired tailsitter UAV utilizing shape-morphing wings with aerofoil-shaped artificial feathers. The design of the UAV is inspired by the shape and motion of bird wings, which can change their shape and span to adapt to different flight conditions. The pigeon's wing skeletal structure serves as the basis for the design, and the wing was developed to be fully tendon-driven employing a single motor for each side. The wings can contract and extend, resulting in a contraction ratio of 49% of the extended wing span. In hovering flight mode, the wing contraction shows a 42% decrease in drag for improved wind disturbance rejection. Wind tunnel testing characterises the wing's aerodynamic performance, revealing significant deflection at high angles of attack due to the articulated skeletal structure. The wings demonstrate low power consumption, averaging only 5.1 W during morphing in experiments. Finally, we demonstrate the wing's robustness through outdoor flight experiments. The research findings provide insights into the potential of bioinspired designs for tailsitter UAVs and offer a promising avenue for future research in this field.

One can estimate the velocity and acceleration of robot manipulators by utilizing nonlinear observers. This involves combining inertial measurement units (IMUs) with the motor encoders of the robot through a model-based sensor fusion technique. This approach is lightweight, versatile (suitable for a wide range of trajectories and applications), and straightforward to implement. In order to further improve the estimation accuracy while running the system, we propose to adapt the noise information in this paper. This would automatically reduce the system vulnerability to imperfect modelings and sensor changes. Moreover, viable strategies to maintain the system stability are introduced. Finally, we thoroughly evaluate the overall framework with a seven DoF robot manipulator whose links are equipped with IMUs.

Over the last decade, a plethora of soft robotic devices have been proposed for the execution of complex grasping and dexterous manipulation tasks. Tasks requiring such increased dexterity are typically executed using fully-actuated, rigid end-effectors equipped with sophisticated sensing and controlled with complex control laws. The new class of soft robotic devices offers an alternative to the traditional end-effectors and facilitates the development of robotic grasping and manipulation solutions that are lightweight, safe to interact with, affordable, and easy to use and control. Within the class of soft robotic grippers and hands, promising recent developments were made in ultra-affordable, even disposable mechanisms based on origami and kirigami structures. This paper proposes a new kirigami-inspired robotic gripper geometry employing compression-based actuation. The compression actuation fundamentally differentiates this new design class from previous kirigami grippers, resulting in more compact robotic grippers with superior grasping capabilities. In particular, we investigate how the shapes of the internal cuts of the kirigami geometries can affect the gripper performance in terms of force exertion and grasping capabilities. A series of experiments are conducted to understand better the working principles behind this new type of kirigami grippers and experimentally validate their efficacy in the execution of complex, everyday life tasks. Further demonstrations of the gripper's capabilities include the pick-and-placing of human hair, egg yolk, and even liquids.

This paper proposes a smart handheld textural sensing medical device with complementary Machine Learning (ML) algorithms to enable on-site Colorectal Cancer (CRC) polyp diagnosis and pathology of excised tumors. The proposed unique handheld edge device benefits from a unique tactile sensing module and a dual-stage machine learning algorithms (composed of a dilated residual network and a t-SNE engine) for polyp type and stiffness characterization. Solely utilizing the occlusion-free, illumination-resilient textural images captured by the proposed tactile sensor, the framework is able to sensitively and reliably identify the type and stage of CRC polyps by classifying their texture and stiffness, respectively. Moreover, the proposed handheld medical edge device benefits from internet connectivity for enabling remote digital pathology (boosting the diagnosis in operating rooms and promoting accessibility and equity in medical diagnosis).

This paper introduces a novel, small form-factor, aerial vehicle research platform for agile object detection, classification, tracking, and interaction tasks. General-purpose hardware components were designed to augment a given aerial vehicle and enable it to perform safe and reliable grasping. These components include a custom collision tolerant cage and low-cost Gripper Extension Package, which we call GREP, for object grasping. Small vehicles enable applications in highly constrained environments, but are often limited by computational resources. This work evaluates the challenges of pick-and-place tasks, with entirely onboard computation of object pose and visual odometry based state estimation on a small platform, and demonstrates experiments with enough accuracy to reliably grasp objects. In a total of 70 trials across challenging cases such as cluttered environments, obstructed targets, and multiple instances of the same target, we demonstrated successfully grasping the target in 93 % of trials. Both the hardware component designs and software framework are released as open-source, since our intention is to enable easy reproduction and application on a wide range of small vehicles.

Soft fluidic actuators are becoming popular for their backdrivability, potential for high power density, and their support for power supply through flexible tubes. Control and design of such actuators requires serviceable models that describe how they relate fluid pressure and flow to mechanical force and motion. We present a simple 2-port model of a bellows actuator that accounts for the relationships among fluid and mechanical variables imposed by the kinematics of the deforming bellows structure and accounts for elastic energy stored in the actuator's thermoplastic material structure. Elastic energy storage due to axial deformation is captured by revolving a differential strip whose linear elastic behavior is a nonlinear function of the actuator length. The model is evaluated through experiments in which either actuator length and pressure or force and pressure are imposed. The model has an error of 9.8% of the force range explored and yields insight into the effects of geometry changes. The resulting model can be used for model-based control or actuator design across the full operating range and can be exercised under either imposed force or imposed actuator length.

The rigid and straight nature of conventional surgical drills and screwdrivers makes it difficult to access the posterior mandible for fracture reduction without the creation of facial incisions. To assist transoral mandibular fracture reduction in hard-to-reach areas, we propose a shared-control dexterous robotic system. The end effector of this system is an articulated drilling/screwing tool to provide distal dexterity. This system uses an admittance-control-based approach to provide precision and stability during shared-control hole-drilling processes. A cadaver study showed the efficacy of the proposed system to assist plate fixation in the reduction of mandibular fractures. The proposed articulated surgical tool was capable of drilling holes in and driving screws into the mandible of a cadaver head. In addition, the shared-control robotic system ensured that the drill moved along its axial direction, leading to stable and precise hole drilling.

This paper presents a novel integration of a shared autonomous mobile humanoid robot for remote nursing assistance. The proposed nursing robot has a motorized versatile supporting structure to allow flexible integration of the system components, autonomously adjust its mobile manipulation workspace and improve its reachability and manipulability to operate in a cluttered environment. The robot also provides a novel integration of robot autonomy to reduce the human effort to coordinate the motorized chest and arm motion, control the precise manipulation of objects and camera viewpoint, and handle complex collision avoidance in human-guided gross manipulation. Moreover, we developed an open-source virtual testbed that integrates ROS- and Unity-based robot simulation and benchmark mobile manipulation nursing tasks and scenarios in a realistic simulation of a hospital environment. The virtual testbed supports various contemporary gaming and AR/VR interfaces to control the virtual human and robots, and provides autonomy for navigation, manipulation, and remote active perception assistance. We conducted a user study (N=9) to validate that the versatile supporting structure and shared autonomy of the physical testbed can effectively reduce the human effort to control unstructured manipulation, and improve the robot's reachability and manipulability. In addition, we conducted a pilot study (N=8) to test the usability of the virtual testbed and collect feedback from representative users.

Modern exoskeletons are typically developed to optimize for a single, physiological objective, the “gold standard” of which is a reduction of the wearer's metabolic rate. However, recent research suggests that these changes in metabolic rate are not yet perceivable on average. To address this gap, this study explores a novel economic value metric to quantify the value of exoskeleton assistance. The overarching goal of this work is the development of a perceptible metric that leverages the user experience to quantify exoskeleton success. We use the Vickrey second-price auction to obtain the monetary compensation needed for participants to continue walking for consecutive two-minute bouts. Comparing the participant's bidding trends when wearing and not wearing an exoskeleton captures the economic value of the experience, termed Marginal Value (MV). To reduce the logistical burden of the auction, we simulated human participants (robo-bidders) to compete alongside real participants. This work presents a sensitivity analysis to understand how the number and bidding behavior of the robo-bidders affects our economic value metric, MV. We found that MV was not significantly affected by the number of robo-bidders or their bidding behavior (i.e. effort rate). The bidding behavior of the human participants was affected by the robo-bidder effort rate, indicating that there is interplay in the bidding dynamics among the auction participants, but these changes do not significantly affect the marginal value. This study tentatively validates the current approach in generating our proposed metric for exoskeleton success, paving the way for economic value to be further explored as a holistic, personalized metric for the development of lower-limb exoskeletons.

It is challenging to realize the safe navigation of mobile robots in crowds. Most of the previous studies may lead to unsafe robot navigation in crowds, as safety guarantee is lacked. To solve this problem, we devise a safety filter (SF) that enables realization of safe robot navigation in crowds, and provides safety guarantees by verifying whether the optimal action recommended by an unsafe method is safe and, if not, corrects the action. The three main processes performed by the SF applied to given robot are (1) construction of the safe state constraints of the robot using a safe set; (2) construction of the safe action constraints of the robot based on discrete-time generalized velocity obstacles (DGVOs); and (3) determination of a feasible solution of the SF design problem, or, if none can be found, replacement of the above hard constraints with heuristic soft constraints. We used the SF with a reaction-based method and three learning-based methods in simulation experiments of random and non-random crowds, and the results showed that the SF decreases the collision rates and danger rates and thereby increases the success rates of these methods. We also deployed the SF with three learning-based methods on an mr1000 robot in real-world experiments, and the results showed that the SF enabled the robot using learning-based methods to navigate to its goal without colliding with humans.

Most land-air amphibious UAVs feature a four-wheel design that limits their adaptability in narrow and uneven spaces. This study proposes the rotor flywheel as a new land-air design that integrates a one-wheel structure and eight-rotor wings for more flexible motion. The dynamics model is then conducted with the Kane method, finding two power-saving self-balance state while rolling. Its controller design highlights the multi-input decoupling approach utilizing feedback, along with the dynamic model-based component to enable efficient control of its intricate operations. Results of simulations and experimental tests have validated the stability and adaptability of the mode-switching and rolling of the robot in ground motion.

Electric vehicles (EVs) play critical roles in autonomous mobility-on-demand (AMoD) systems, but their unique charging patterns increase the model uncertainties in AMoD systems (e.g. state transition probability). Since there usually exists a mismatch between the training and test/true environments, incorporating model uncertainty into system design is of critical importance in real-world applications. However, model uncertainties have not been considered explicitly in EV AMoD system rebalancing by existing literature yet, and the coexistence of model uncertainties and constraints that the decision should satisfy makes the problem even more challenging. In this work, we design a robust and constrained multi-agent reinforcement learning (MARL) framework with state transition kernel uncertainty for EV AMoD systems. We then propose a robust and constrained MARL algorithm (ROCOMA) with robust natural policy gradients (RNPG) that trains a robust EV rebalancing policy to balance the supply-demand ratio and the charging utilization rate across the city under model uncertainty. Experiments show that the ROCOMA can learn an effective and robust rebalancing policy. It outperforms non-robust MARL methods in the presence of model uncertainties. It increases the system fairness by 19.6% and decreases the rebalancing costs by 75.8%.