IROS 2024 Papers

Performing human-like motion is crucial for service humanoid robots. Real-time motion retargeting allows clear observation of the robot’s pose and provides instant feedback during human demonstrator actions. This paper presents an optimization-based real-time anthropomorphic motion retargeting framework for transferring human arm motion to a robot arm. The framework is generic, applicable to both spherical-rotational-spherical (SRS) and non-SRS robot arms. We introduce the normalized normal vector of the arm plane as an anthropomorphic criterion within our framework. The method is validated on a service humanoid robot, with both static and dynamic evaluations. The statistical analysis show that our method maintains strong anthropomorphic features while ensuring accurate wrist pose tracking.

For the cascaded planning and control modules implemented for robot navigation, the frequency gap between the planner and controller has received limited attention. In this study, we introduce a novel B-spline parameterized optimization-based planner (BSPOP) designed to address the frequency gap challenge with limited onboard computational power in robots. The proposed planner generates continuous-time control inputs for low-level controllers running at arbitrary frequencies to track. Furthermore, when considering the convex control action sets, BSPOP uses the convex hull property to automatically constrain the continuous-time control inputs within the convex set. Consequently, compared with the discrete-time optimization-based planners, BSPOP reduces the number of decision variables and inequality constraints, which improves computational efficiency as a byproduct. Simulation results demonstrate that our approach can achieve a comparable planning performance to the high-frequency baseline optimization-based planners while demanding less computational power. Both simulation and experiment results show that the proposed method performs better in planning compared with baseline planners in the same frequency.

In minimally invasive pain management procedures, precise needle positioning is paramount for effective treatment and patient safety. Traditional techniques like the loss-of-resistance (LOR) method may be insufficient, especially in patients with narrowed epidural spaces. The use of imaging tools such as C-arms carries risks due to radiation exposure for medical professionals. A new system for detecting the epidural space based on optical interferometry is proposed to tackle this issue. Prior research has focused on force measurement systems to identify tissue puncture or rupture. Although mechanical sensors have been utilized, they add bulk and complexity to systems. Optical sensors like Fiber Bragg grating (FBG) and Fabry-Pérot interferometer (FPI) offer stable, high-resolution measurements suitable for complex biological tissues. This study aims to develop a sensor and needle system for epidural injections, incorporating quantitative metrics for validation. An optical interferometer-based force measurement sensor was integrated into a commercial epidural needle, and calibration was performed to establish a correlation between system output and actual force. The system employs a graphical user interface (GUI) to identify puncture points based on abrupt force decreases. A user study involving interventionalists assessed the system’s performance by measuring invasive depth and success rates. The user study demonstrated that the proposed sensorized system could detect the puncture with an average success rate of 72.63 %. This study represents a significant advancement toward safer and more precise epidural procedures, addressing critical clinical considerations for practical applications.

In robot-assisted retinal surgery, a Remote Center of Motion (Rcm) allows the surgical instrument to rotate around a distal fixed point without any lateral translations. The Rcm point should be perfectly aligned inside the trocar. Otherwise, unexpected tool translations at the expected remote center will enlarge the force applied to the trocar and consequently result in post-operative complications. Due to the narrow size of the trocar and the lack of real-time detection equipment, the Rcm point is hard to be perfectly located inside the trocar. Even if the Rcm is perfectly aligned, the movement of the tissue around the eyeball could make it inappropriate again. In this paper, inspired by the control strategy of surgeons, an online Rcm adjusting strategy is proposed. Instead of only using one fixed Rcm point, to restrict the force between the surgical tool and the trocar, the proposed strategy adjusts the position of the Rcm point during the motion. The results show our approach significantly reduces the force between the robot end-effector and surgical port by 64.2%. In addition, the results also demonstrate that our approach complies the Rcm trajectories without deforming or spoiling the working space, which is significantly important for obeying surgeon’s instructions in practice.

In indoor environments where the Global Navigation Satellite System (GNSS) isn’t available, the infrastructure-based LiDAR-camera joint array can provide high-precision localization for mobile robots, such as Autonomous Valet Parking (AVP). The primary challenge in employing the infrastructure-based LiDAR-camera joint array is the extrinsic calibration between the LiDAR and the camera. Moreover, to handle interference deviation caused by vibrations or inadequate mounting stiffness during operation, the calibration’s extrinsic parameters must be automatically updated online, presenting higher demands for infrastructure-based LiDAR-camera extrinsic calibration. This paper proposes an infrastructure LiDAR-camera online automatic calibration method based on prior knowledge of cross-modal target registration. This method requires no manual targets and initial pose guesses and can achieve extrinsic calibration. The object-prior model based on a lightweight object detection algorithm can rapidly detect scenes favorable for extrinsic calibration in sub-images of camera images. This creates favorable conditions for the registration of cross-modal networks and poses optimization of the LiDAR camera. Additionally, because a lightweight algorithm is used, the process does not compromise efficiency or consume excessive computational resources. Experimental results demonstrate that the proposed calibration method is suitable for calibrating infrastructure-based LiDAR-camera, with comparable accuracy and the ability to perform online calibration. Comparative experiments also show that the object-prior model can indeed select better scenes for LiDAR-camera extrinsic calibration, thus improving the accuracy and stability of extrinsic calibration to some extent.

In response to torso-oriented magnetic localization tasks that require the system to have interactivity and flexibility with guaranteed accuracy, a novel bio-inspired magnetic sensor array configuration is proposed in this paper. Precisely, the ideas of the natural characteristics of octopus flexible tentacles and the "wrap" morphology are integrated into the design of the magnetic localization system based on the sensor array method. It is worth mentioning that such a design enhances the interactivity and flexibility of the localization system compared to the general planar sensor array strategy. Apart from the concept introduction, the geometry analysis of the proposed configuration is presented based on the constant curvature model. Besides, the magnetic localization algorithm for the system is presented by constructing a magnetic tracking optimization function. Eventually, the proposed concept and developed algorithm are examined in the sensor-array-simulation environment to manifest their effectiveness and applicability. The experimental results indicate that the octopus-inspired-configuration sensor array achieves a mean accuracy at a centimeter-level in our cases, and has better accuracy with a mean value of ē as 0.0178 m and {\overline {SQR} _{ave{\text{ }}}}{\overline {SQR} _{ave{\text{ }}}} as 0.0883 for the center interest space compared to general planar configuration one. Moreover, the effect of the configuration error is analyzed. These results verify the feasibility and superiority of the proposed concept and hold significant practical significance in addressing the challenge associated with magnetic localization tasks toward the clinical application scenarios.

Visual-Inertial Odometry (VIO) has been widely used by autonomous drones as an onboard navigation method. However, it suffers from drifts especially in scenarios where the environments have few texture features such as an empty room with solid color walls. Optical flow sensors are another type of onboard sensor used by drones that face downward and measure the velocity by detecting changes in pixels between consecutive images, which don’t introduce accumulative error. In this work, we present an efficient tight-coupled estimator to improve the accuracy of VIO by fusing the measurements of a downward-facing optical flow sensor into the VIO framework consistently. We further analyze the observability of the estimators and prove that there are four unobservable directions in the ideal case and then we utilize OC-EKF to maintain the consistency of the estimator. Furthermore, we extend an adaptive weighting algorithm to the proposed method, which can better adapt to the scenes where feature tracking is less accurate. Finally, both simulation and real-world experiments demonstrate the feasibility of the proposed method.

Tumors affecting abdominal organs rank among the deadliest malignancies. In this context, Magnetic Resonance Imaging (MRI) serves as an effective diagnostic tool with a strong potential to support image-guided minimally invasive interventions for treating these tumours, offering an ionizing-radiation-free medical modality. MRI provides exceptional soft tissue contrast and multi-angle imaging, enabling accurate intraoperative localisation of target tumours within these vital organs. Nevertheless, MRI-guided minimally invasive interventions still encounter significant challenges due to the strong magnetic field environment and the narrow and deep bore of MRI machines. This paper proposes a novel MR safe 5-degrees-of-freedom (DoFs) parallel table-mounted double-arch needle insertion robot with a scissor-folding mechanism (SFM) for abdominal interventions. The proposed robot is designed to fit a standard 70-cm MRI bore. Initial evaluation experiments indicate mean errors of 3.14 mm for the proposed robotic arch and 2.23 mm for the full needle insertion robot, respectively. Additionally, preliminary testing of the system in an MRI environment resulted in unaltered MRI imaging output, with negligible artefacts associated with the presence of the robot within the bore.

To ensure safe cooperative driving in mixed traffic with both manned and unmanned vehicles, it is crucial to understand and model the driving styles of human drivers. This paper explores how to develop accurate recognition of driving style and use that for the prediction of vehicle motion, which enables better performance in cooperative driving. A simulation testbed that consists of a driving simulator and a copilot is first introduced for the purpose of data collection and testing. A Long Short-Term Memory (LSTM)-based network that models human driving styles and predicts driving acceleration is developed. Standalone tests are conducted to examine the model performance in the simulation testbed. Finally, the model is evaluated in a series of merging experiments that involves 5 vehicles.

Cooking robots have long been desired by the commercial market, while the technical challenge is still significant. A major difficulty comes from the demand of perceiving and handling liquid with different properties. This paper presents a robot system that mixes batter and makes pancakes out of it, where understanding and handling the viscous liquid is an essential component. The system integrates Haptic Sensing and control algorithms to autonomously stir flour and water to achieve the desired batter uniformity, estimate the batter’s properties such as the water-flour ratio and liquid level, as well as perform precise manipulations to pour the batter into any specified shape. Experimental results show the system’s capability to always produce batter of desired uniformity, estimate water-flour ratio and liquid level precisely, and accurately pour it into complex shapes. This research showcases the potential for robots to assist in kitchens and step towards commercial culinary automation.

Autonomous takeoff of flapping-wing robots (FWRs) is crucial for accelerating response speed and reducing costs in executing tasks. Jumping-aided takeoff is an effective method adopted by birds. However, the limited power density of motors poses challenges in achieving this type of takeoff for FWRs. In this study, we introduce a ground-based FWR ejecting system that utilizes a symmetric slider-crank mechanism (S-SCM) to store energy in spring. This stored energy is then converted into the takeoff speed of the FWR. The dynamic model of each working stage is established, and the design parameters are optimized according to simulation results. The prototype of the FWR ejecting system is fabricated for experimental validations. The results indicate that the system can provide a takeoff speed of 4 m/s for the 270 g FWR. Notably, the system is deployable on rough terrains and only adds a 3.2 g payload to the robot. Our work advances the autonomous takeoff of FWRs, promoting the application of such robots.

In mobile robotics, effectively covering known terrains is essential. While grid-based methods surpass exact cell decomposition in path length and multi-robot scalability, they face challenges in irregular areas. Here we develop a model for shortening coverage paths in arbitrary environments using grid-based methods, which redefines the path optimization problem as finding the largest Hamiltonian sub-graph of a given grid graph. Additionally, we present a Hamiltonian cycle expansion strategy to simplify the resolution process and propose a low-repetitive coverage path planner based on the strategy. Our path planner enables the quick finding of an efficient full coverage path in any region. Simulation results show that our algorithm consistently produces efficient coverage paths across diverse settings and demonstrates its adaptability in multi-robot systems.

Versatile, autonomous robotic boats can offer excellent environmental inspection and monitoring solutions for remote, dangerous, hard to reach, or access protected water bodies. This paper introduces such a platform in the form of an autonomous, cost-effective, waterjet-powered robotic trimaran. Motivated by the need for an efficient aquatic monitoring, particularly in Aotearoa - New Zealand’s diverse environments, the trimaran provides an efficient, low-cost, and easy to replicate alternative to resource-intensive research vessels. The proposed platform, costs $600-1,500 USD to develop (depending on the sensing system configuration), weighs under 5 kg, and excels in bathymetry and water quality testing. The trimaran can reach speeds of up to 2 m/s offering obstacle avoidance of natural features, such as rocks. Utilizing off-the-shelf components and 3D printing technology, the proposed platform offers excellent reproducibility and robustness while operating in shallow waters with its jet propulsion system. The paper presents in detail the design characteristics, the sensing system employed, testing results focusing on bathymetry, and highlights the ability of vessel and the potential for future research and data collection.

This study introduces an agile robotic penguin featuring a pair of 2-degrees of freedom (DoF) wing mechanisms. Each wing can independently control flapping and feathering motions with two in-house submersible geared servomotors through a differential gear mechanism. Notably, our mechanism allows unrestricted feathering beyond 360°. Since feathering directly changes the wing’s angle of attack (AoA), the hydrodynamic forces can be significantly adjusted to achieve agile maneuvers. This paper demonstrates various maneuvers, including rapid acceleration, hard braking, rolling, pitching, and yawing, achieved solely by changing the feathering motion of both wings. Our robotic penguin reached a maximum forward speed of 1.8 m/s, comparable to the foraging speed of real penguins. The average roll, pitch, and yaw rates were 363°/s, 75°/s, and 92°/s, respectively. This robot serves as a model for the biological study of maneuverability in real penguins and the engineering exploration of bioinspired agile underwater robots.

This study presents the development and validation of an innovative hand exoskeleton designed for the re-habilitation of patients with Complex Regional Pain Syndrome (CRPS), a condition frequently arising post-injury or surgeries. The prototype is tailored for the hand, a region commonly affected by CRPS, and is notable for its adaptability and a comprehensive sensor system for monitoring individual joint movements. Reliable sensor performance was defined through precise force measurements and stability over time, showing minimal drift. These features enable personalized rehabilitation and objective progress tracking, addressing limitations in traditional physiotherapy such as availability, cost, and time constraints. The contributions of this work lie in its innovative design and the potential for robotic systems to improve therapeutic outcomes in CRPS rehabilitation.

This work proposes a novel and provably correct method for three-dimensional optimal motion planning in complex environments. Our approach models the 3D motion planning problem by solving streamlines of the potential fluid flow, filling a gap in traditional motion planning techniques by guaranteeing a closed-loop, smooth and natural-looking navigation solution. Special emphasis is given to an inherent challenge of artificial potential field (APF) methods, namely establishing proofs of safety and stability over the entire optimization process. A model-based actor-critic reinforcement learning algorithm is introduced to approximate the optimal solution to the Hamilton-Jacobi-Bellman equation and update the controller parameters in a deterministic manner. Through a series of ROS-Gazebo software-in-the-loop simulations the proposed methodology demonstrates robustness and outperforms widely used methods such as the RRT∗, highlighting its contribution to the field of 3D optimal motion planning.

The torso of quadruped mammals serves as a robust foundation for their bodies, enabling a diverse array of agile and intricate movements. However, current quadruped robots cannot match the extensive range of motion exhibited by biological torsos while also serving a load-bearing role. Therefore, this paper presents an active bionic torso that emulates the quadrupedal animal's spinal column and associated muscular structure. This innovative torso can mimic animal torsos movements, including flexing, extending, lateral bending, and axial rotation. A thorough analysis of the torso's kinetic, workspace, and structural dynamics has been conducted. This proposed torso boasts a considerable load-bearing capacity and can support a load that exceeds its weight tenfold. The passive spring incorporated into the bionic torso emulates the intervertebral discs' shock-absorbing and load-bearing functions. Additionally, this paper documents the development of a quadrupedal robot fitted with the proposed bionic torso, demonstrating the torso's mobility in a real-world application.

Connection radius in asymptotically optimal motion planning algorithms is of interest to both understand the theoretical properties of these algorithms, as well as to ensure practical performance by estimating lower bounds. The smaller the connection radius, the sparser the data structures constructed using them, which makes the associated algorithms computationally more efficient. The original radii for both roadmap and tree variants were reported to be asymptotically shrinking functions of n. A recent amendment to the original arguments for trees demonstrated that the radius has to be larger for tree-based variants (RRT*). A practical problem in the newly proposed radius is the persistence of hard-to-estimate or large-valued parameters (like optimal path cost) within the connection radius function. In this short paper, a new perspective is presented of approaching the proof of asymptotic optimality of RRT* from a minimal variant of RRT* that only includes tree additions within connection neighborhoods. The work provides an alternative connection radius that gets rid of unwieldy parameters, presents insights that holds promise in studying the problem and using the result.

We propose a method for depth estimation under different illumination conditions, i.e., day and night time. As photometry is uninformative in regions under low-illumination, we tackle the problem through a multi-sensor fusion approach, where we take as input an additional synchronized sparse point cloud (i.e., from a LiDAR) projected onto the image plane as a sparse depth map, along with a camera image. The crux of our method lies in the use of the abundantly available synthetic data to first approximate the 3D scene structure by learning a mapping from sparse to (coarse) dense depth maps along with their predictive uncertainty – we term this, SpaDe. In poorly illuminated regions where photometric intensities do not afford the inference of local shape, the coarse approximation of scene depth serves as a prior; the uncertainty map is then used with the image to guide refinement through an uncertainty-driven residual learning (URL) scheme. The resulting depth completion network leverages complementary strengths from both modalities – depth is sparse but insensitive to illumination and in metric scale, and image is dense but sensitive with scale ambiguity. SpaDe can be used in a plug-and-play fashion, which allows for 24% improvement when augmented onto existing methods to preprocess sparse depth. We demonstrate URL on the nuScenes dataset where we improve over all baselines by an average 12.39% in all-day scenarios, 12.02% when tested specifically for daytime, and 14.95% for nighttime scenes. Code available at : https://github.com/ezhovv/all-day-depth