IROS 2023 论文列表

This paper presents a novel, low-cost proprioceptive sensing solution for legged robots with point feet to achieve accurate low-drift long-term position and velocity estimation. In addition to conventional sensors, including one body Inertial Measurement Unit (IMU) and joint encoders, we attach an additional IMU to each calf link of the robot just above the foot. An extended Kalman filter is used to fuse data from all sensors to estimate the robot's body and foot positions in the world frame. Using the additional IMUs, the filter is able to reliably determine foot contact modes and detect foot slips without tactile or pressure-based foot contact sensors. This sensing solution is validated in various hardware experiments, which confirm that it can reduce position drift by nearly an order of magnitude compared to conventional approaches with only a very modest increase in hardware and computational costs.

Over the past few years, there has been a great deal of research on navigation tasks in indoor environments using deep reinforcement learning agents. Most of these tasks use only visual information in the form of first-person images to navigate to a single goal. More recently, tasks that simultaneously use visual and auditory information to navigate to the sound source and even navigation tasks with multiple goals instead of one have been proposed. However, there has been no proposal for a generalized navigation task combining these two types of tasks and using both visual and auditory information in a situation where multiple sound sources are goals. In this paper, we propose a new framework for this generalized task: multi-goal audio-visual navigation. We first define the task in detail, and then we investigate the difficulty of the multi-goal audio-visual navigation task relative to the current navigation tasks by conducting experiments in various situations. The research shows that multi-goal audio-visual navigation has the difficulty of the implicit need to separate the sources of sound. Next, to mitigate the difficulties in this new task, we propose a method named sound direction map (SDM), which dynamically localizes multiple sound sources in a learning-based manner while making use of past memories. Experimental results show that the use of SDM significantly improves the performance of multiple baseline methods, regardless of the number of goals.

The adaptability of soft robots makes them ideal candidates to maneuver through unstructured environments. However, locomotion challenges arise due to complexities in modeling the body mechanics, actuation, and robot-environment dynamics. These factors contribute to the gap between their potential and actual autonomous field deployment. A closed-loop path planning framework for soft robot locomotion is critical to close the real-world realization gap. This paper presents a generic path planning framework applied to TerreSoRo (Tetra-Limb Terrestrial Soft Robot) with pose feedback. It employs a gait-based, lattice trajectory planner to facilitate navigation in the presence of obstacles. The locomotion gaits are synthesized using a data-driven optimization approach that allows for learning from the environment. The trajectory planner employs a greedy breadth-first search strategy to obtain a collision-free trajectory. The synthesized trajectory is a sequence of rotate-then-translate gait pairs. The control architecture integrates high-level and low-level controllers with real-time localization (using an overhead webcam). Terre-SoRo successfully navigates environments with obstacles where path re-planning is performed. To best of our knowledge, this is the first instance of real-time, closed-loop path planning of a non-pneumatic soft robot.

Manipulating deformable objects, such as ropes (1D), fabrics (2D), and bags (3D), poses a significant challenge in robotics research due to their high degree of freedom in physical state and nonlinear dynamics. Compared with single-dimensional deformable objects, multi-dimensional object manipulation suffers from the difficulty in recognizing the characteristics of the object correctly and making an accurate action decision on the deformable object of various dimensions. Some methods are proposed to use neural networks to rearrange deformable objects in all dimensions, but their approaches are not accurate in predicting the motion of the robot as they just consider the equivariance in the picking objects. To address this problem, we present a novel Transporter Network encoded and decoded with equivariance to generalize to different picking and placing positions. Additionally, we propose an equivariant goal-conditioned model to enable the robot to manipulate deformable objects into flexible configurations without relying on artificially marked visual anchors for the target position. Finally, experiments conducted in both Deformable-Ravens and the real world demonstrate that our equivariant models are more sample efficient than the traditional Transporter Network. The video is available at https://youtu.be/5_q5ff9c9FU.

The emergence of harvesting robotics offers a promising solution to the issue of limited agricultural labor resources and the increasing demand for fruits. Despite notable advancements in the field of harvesting robotics, the utilization of such technology in orchards is still limited. The key challenge for harvesting robots is to improve the operational efficiency. Taking into account inner-arm conflicts, couplings of DoFs, and the dynamic tasks, we propose a task planning strategy for a harvesting robot with four arms in this paper. The proposed method employs a Markov game framework to formulate the four-arm robotic harvesting task, which avoids the computational complexity of solving an NP-hard scheduling problem. Furthermore, a multi-agent reinforcement learning (MARL) structure with a fully centralized collaboration protocol is used to train a MARL-based task planning network. Several simulations and orchard experiments are conducted to validate the effectiveness of the proposed method for a multi-arm harvesting robot in comparison with the existing method.

Search and rescue applications often need multiple agents to complete a set of conflicting tasks. This paper studies a Multi-Agent Multi-Objective Ergodic Search (MA-MO-ES) approach to this problem where each objective or task is to cover a domain subject to an information map. The goal is to allocate coverage tasks to agents so that all maps are explored ergodically. The combinatorial nature of task allocation makes it computationally expensive to solve for optimal allocation using brute force. Apart from a large number of possible allocations, computing the cost of a task allocation is itself an expensive planning problem. To mitigate the computational challenge, we present a branch and bound-based algorithm with pruning techniques that reduce the number of allocations to be searched to find optimal coverage task allocation. We also present an approach to leverage the similarity between information maps to further reduce computation. Extensive testing on 147 randomly generated test cases shows an order of magnitude improvement in runtime compared to an exhaustive brute force approach.

Consider a Multi-Agent Collective Construction (MACC) problem that aims to generate a plan for fictitious cubic robots to build a three-dimensional structure comprised of cubic blocks. These cubic robots can carry one cubic block at a time; robots may move left, right, forwards, backward, or climb up or down one block. To construct structures taller than one cube, the robots must build supporting scaffolding made of blocks and remove the scaffolding once the structure is built. Prior works sought to create a planner that considered the structure as one monolithic assembly, which becomes intractable for larger workspaces and complex structures. To this end, we present a decomposition algorithm that breaks the structure into substructures that can be planned for independently. We use Mixed Integer Linear Programming (MILP) to plan for each of these substructures and then aggregate the solutions to construct the entire structure. Extensive testing on 200 randomly generated structures shows an order of magnitude improvement in the solution computation time compared to an MILP approach without decomposition. Finally, we leverage the independence between substructures to detect which substructures can be built in parallel.

While motion classification architectures have improved in accuracy and robustness in recent years, computationally expensive approaches and sophisticated hardware dependencies limit their real-world applicability. To overcome these challenges, we have designed a lightweight, realtime architecture for classifying motions of the arm & hand using features derived from motor unit action potentials within surface Electromyographic (sEMG) signals, rather than which provide direct interrogation of underlying muscle activation patterns. We tested the architecture on 6 motions performed dynamically across a range of muscle contraction intensities achieving median classification accuracies ranging from 91.3% to 93.3 % and an average processing time of approximately 40 ms across three different classifiers. Taken together, our findings demonstrate potential robustness of motor unit based neural interfaces for motion classification tasks.

Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multi-modality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions. https://sites.google.com/view/mp-diffusion

In semiconductor manufacturing, lithography machines are becoming more and more sophisticated system of systems. As an example, a TWINSCAN wafer scanner machine is composed of a wafer, and reticle handlers, reticle, optics, and two wafer chains or systems. In previous studies, we covered the interactions between the reticle, optics, and wafer chains during the step-and-scan cycle. In this study, we focus on the interaction between the additional wafer chain responsible for aligning the wafer substrate and taking its height map during the measurement cycle, and the other chains that are active during the step-and-scan cycle. Working in parallel to increase machine throughput, the inertial forces associated motion of the two cycles induce vibration that may propagate throughout the chains in the machine if no appropriate measures are taken. In this investigation, we look at the reference trajectories responsible for steering the chains throughout the two cycles, and propose two reference trajectory orchestrations that factor in the machine design, geometry, mass distribution, and functions. Theoretically, these orchestrations lead to suppressing the induced vibration without sacrificing the machine throughput while keeping the involved control loops intact.

Ultrasound (US) imaging is widely used for diagnosing and monitoring arterial diseases, mainly due to the advantages of being non-invasive, radiation-free, and real-time. In order to provide additional information to assist clinicians in diagnosis, the tubular structures are often segmented from US images. To improve the artery segmentation accuracy and stability during scans, this work presents a novel pulsation-assisted segmentation neural network (PAS-NN) by explicitly taking advantage of the cardiac-induced motions. Motion magnification techniques are employed to amplify the subtle motion within the frequency band of interest to extract the pulsation signals from sequential US images. The extracted real-time pulsation information can help to locate the arteries on cross-section US images; therefore, we explicitly integrated the pulsation into the proposed PAS-NN as attention guidance. Notably, a robotic arm is necessary to provide stable movement during US imaging since magnifying the target motions from the US images captured along a scan path is not manually feasible due to the hand tremor. To validate the proposed robotic US system for imaging arteries, experiments are carried out on volunteers' carotid and radial arteries. The results demonstrated that the PAS-NN could achieve comparable results as state-of-the-art on carotid and can effectively improve the segmentation performance for small vessels (radial artery). The code11Code: https://qithub.com/dianveHuanq/RobPMEPASNN and demonstration video22Video: https://youtu.belc9AM042_lUQ can be publicly accessed.

2D forward-looking sonar is a crucial sensor for underwater robotic perception. A well-known problem in this field is estimating missing information in the elevation direction during sonar imaging. There are demands to estimate 3D information per image for 3D mapping and robot navigation during fly-through missions. Recent learning-based methods have demonstrated their strengths, but there are still drawbacks. Supervised learning methods have achieved high-quality results but may require further efforts to acquire 3D ground-truth labels. The existing self-supervised method requires pretraining using synthetic images with 3D supervision. This study aims to realize stable self-supervised learning of elevation angle estimation without pretraining using synthetic images. Failures during self-supervised learning may be caused by motion degeneracy problems. We first analyze the motion field of 2D forward-looking sonar, which is related to the main supervision signal. We utilize a modern learning framework and prove that if the training dataset is built with effective motions, the network can be trained in a self-supervised manner without the knowledge of synthetic data. Both simulation and real experiments validate the proposed method.

Animals' anatomies have control systems combined with multi-motors and high-bandwidth sensors. Their complicated mechanisms give them high maneuverability with sufficient inertial stabilization performance during walking, jumping, and flying. From the point of aerial locomotion, flying insects use abdomen reflexes to stabilize their head positions. Articulation of the thoracic-abdominal joint contributes to the reorientation of their bodies over the law of conservation of angular momentum. Since acceleration is a fundamental component of maneuverability, increasing the acceleration without destabilizing the body is achieved with additional appendages such as the tail and abdomen. Highly actuated abdominal muscles are an essential feature of these natural flyers, conspicuously missing from the current aerial vehicles regarding maneuverability. This study proposes a bio-inspired aerial vehicle morphology with an actively controlled abdomen-like appendage. We aim to investigate the advantages and disadvantages of the abdomen-like appendage mounted on a bi-rotor aerial vehicle by constructing the dynamical model and designing optimization-based controllers; Linear Quadratic Regulator (LQR), Model Predictive Control (MPC), and Adaptive Model Predictive Control (A-MPC). We complete our analysis with a motion planning algorithm by combining the sampling-based neighborhood graph approach with the A-MPC strategy. We demonstrate through simulation experiments that the appendage improves the stability and maneuverability of aerial vehicles and the resulting motion planning structure with A-MPC ensures that the state and input constraints are not violated.

Symbolic task planning for robots is computationally challenging due to the combinatorial complexity of the possible action space. This fact is amplified if there are several sub-goals to be achieved due to the increased length of the action sequences. In this work, we propose a multi-goal symbolic task planner for deterministic decision processes based on Monte Carlo Tree Search. We augment the algorithm by prioritized node expansion which prioritizes nodes that already have fulfilled some sub-goals. Due to its linear complexity in the number of sub-goals, our algorithm is able to identify symbolic action sequences of 145 elements to reach the desired goal state with up to 48 sub-goals while the search tree is limited to under 6500 nodes. We use action reduction based on a kinematic reachability criterion to further ease computational complexity. We combine our algorithm with object localization and motion planning and apply it to a real-robot demonstration with two manipulators in an industrial bearing inspection setting.

Miniature soft robots present great potential in delicate manipulations due to their gentle force, complaint structures, and flexible motions. Easy control and fast response make pneumatic actuation a prevalent method for driving soft robotics. In addition, sequential motions are also crucial for enhancing the grasping and moving abilities of soft robots. How-ever, existing miniature pneumatic soft robots are limited to one-dimensional geometries and simple motions due to the difficulties in designing and fabricating intricate small airways in miniature pneumatic soft robots, which restricts them from more versatile deformations. Here, we employ intricate monolithic microchannels embedded into miniature soft robots' mon-olithic bodies for sequential motions. After verifying the effects of the channel diameter, strain-limiting layer, and elastic modulus of the robot's body on the bending behaviors of the ID soft robots, we fabricated a soft flower robot capable of sequential and simultaneous 3D-to-3D shape morphing through five individual microchannels and a soft carnivorous plant robot containing 2D interconnected microchannels capable of sequential enclosed grasping through a single inlet.

We present a modular framework designed to enable a robot hand-arm system to learn how to catch flying objects, a task that requires fast, reactive, and accurately-timed robot motions. Our framework consists of five core modules: (i) an object state estimator that learns object trajectory prediction, (ii) a catching pose quality network that learns to score and rank object poses for catching, (iii) a reaching control policy trained to move the robot hand to pre-catch poses, (iv) a grasping control policy trained to perform soft catching motions for safe and robust grasping, and (v) a gating network trained to synthesize the actions given by the reaching and grasping policy. The former two modules are trained via supervised learning and the latter three use deep reinforcement learning in a simulated environment. We conduct extensive evaluations of our framework in simulation for each module and the integrated system, to demonstrate high success rates of in-flight catching and robustness to perturbations and sensory noise. Whilst only simple cylindrical and spherical objects are used for training, the integrated system shows successful generalization to a variety of household objects that are not used in training.

Multi-flagellated bacteria utilize the hydrodynamic interaction between their filamentary tails, known as flagella, to swim and change their swimming direction in low Reynolds number flow. Simplified hydrodynamics model, like Resistive Force Theories (RFT), lacks the capability to capture the dynamics of certain interactions known as bundling and tumbling. However, for the development of efficient and steerable robots inspired by bacteria, it becomes crucial to exploit this interaction. In this paper, we present the construction of a macroscopic bio-inspired robot featuring two rigid flagella arranged as right-handed helices, along with a cylindrical head. By rotating the flagella in opposite directions, the robot's body can reorient itself through repeatable and controllable tumbling. To accurately model this bi-flagellated mechanism in low Reynolds flow, we employ a coupling of rigid body dynamics and the method of Regularized Stokeslet Segments (RSS). Unlike RFT, RSS takes into account the hydrodynamic interaction between distant filamentary structures. Furthermore, we delve into the exploration of the parameter space in terms of the flagellum geometry to optimize the propulsion and torque of the system. To achieve the desired reorientation of the robot, we propose a tumble control scheme that involves modulating the rotation direction and speed of the two flagella. The scheme enhance the steerability by enabling the robot to attain the desired heading angle with high accuracy. Notably, the overall scheme boasts a simplified design and control as it only requires two control inputs. With our macroscopic framework serving as a foundation, we envision the eventual miniaturization of this technology to construct mobile and controllable micro-scale bacterial robots.

This work introduces a new design paradigm for robotic legs. Our concept extends upon classical series elastic actuation and directly integrates the series compliance into the structure of the leg. This integration reduces the mechanical design complexity and can potentially reduce the overall weight of the leg. In this paper, we introduce a prototype leg with a continuously compliant shank and derive and analyze a non-linear beam model that is used to predict its contact forces. The model is validated in two static experiments: one on the isolated shank and one on the full leg. It shows good agreement with measurements. In our validation, we also studied the influence of the model discretization, showing that about 10 nodes are sufficent. Furthermore, we introduce the concept of a force workspace: the range of forces that can be created by controlling the joint angles of the leg. Due to the coupling of nonlinear deformations and nonlinearities in the kinematics, this workspace is non-trivial. In particular, we demonstrate that it is bounded by a force-singularity in which the force/joint-angle relationship cannot be inverted. The results presented in this work can be applied in the development of state-estimators, set-point filters and controllers, and they can inform the future design of suitable geometries of compliant elements.

In recent years, Action Detection has become an active research topic in various fields such as human-robot interaction and assistive robots. Most of the previous methods in this field focus on temporally processing the action representation, without considering the dependencies among the action classes. However, actions that occur in a video are constantly related, and this correlation could offer effective clues for detection tasks. In this work, we propose to exploit the information of related action classes with the help of a graph neural network in conjunction with temporal modeling. We introduce the attention-based temporal class module (ATC), which models the inherent action dependencies on the graph and learns action-specific features among temporal dimensions with a dual-branch attention mechanism. Further, we present the Graph-based Class-level Attention Network (GCAN), which is built upon ATC modules with increasing temporal receptive fields to handle actions instances in complex untrimmed videos. Our network is evaluated on two challenging benchmark datasets with dense annotations: Charades and MultiTHUMOS. Experimental results show that our approach demonstrates highly competitive results with a significantly reduced model complexity.