ICRA 2023 论文列表

This paper presents a new design of aerial vehicles with tetrahedral geometry. We call this design the TetraQuad. The TetraQuad is a fractal modular aerial robot. A characteristic of fractals is that they have a geometric shape that can be assembled to generate the same geometry on a larger scale. Therefore multiple TetraQuad modules can be assembled to produce a larger scaled tetrahedral shaped aerial vehicle. The advantage is to have modular aerial robots that assemble in the vertical direction; this increases the rigidity of the structure, as well as reduces the wake interaction of the elevated propellers in the assembly. This work presents a design and analysis of the TetraQuad module as well as assemblies of multiple modules. A modular controller strategy is discussed. The functionality of the controller is illustrated using simulations. We validate our design with experimental flight tests.

Over the last two decades, the robotics community witnessed the emergence of various motion representations that have been used extensively, particularly in behavorial cloning, to compactly encode and generalize skills. Among these, probabilistic approaches have earned a relevant place, owing to their encoding of variations, correlations and adaptability to new task conditions. Modulating such primitives, however, is often cumbersome due to the need for parameter re-optimization which frequently entails computationally costly operations. In this paper we derive a non-parametric movement primitive formulation that contains a null space projector. We show that such formulation allows for fast and efficient motion generation and adaptation with computational complexity O(n2) without involving matrix inversions, whose complexity is O(n3). This is achieved by using the null space to track secondary targets, with a precision determined by the training dataset. Using a 2D example associated with time input we show that our non-parametric solution compares favourably with a state-of-the-art parametric approach. For demonstrated skills with high-dimensional inputs we show that it permits on-the-fly adaptation as well.

Flapping wing micro aerial vehicles face challenges in sensing and reacting to disturbances like wind gusts. This work introduces a new microscale bio-inspired digital strain sensor to detect these perturbations. The sensor is designed to change logic states when a specified strain threshold has been reached. The sensors are 3D printed on a flexible Mylar wing using two-photon polymerization. Three digital sensors with varying strain thresholds demonstrate differences in activation timing due to different design parameters. The sensors are tested at the 25 Hz flapping frequency of a hawkmoth, an insect with comparable wing size. A perturbation was added to the flapping wing by subjecting it to a 3 m/s wind gust. A single digital sensor is able to identify the wind disturbance by comparing the time of the first strain threshold crossing. A separate approach looks at the change in sensor ‘on’-time for each flap cycle and provides a clear indication of the wind disturbance.

This paper proposes a new framework for the computed torque method (CTM) of robot manipulators without velocity measurements. We first introduce the Luenberger-observer-based CTM with only position measurements. We then clarify that the external disturbance affects not only the tracking performances with respect to the plant but also the estimation accuracies relevant to the state observer. To address this problem, we establish a new architecture for the so-called dual-loop control scheme, by which both the tracking performances and estimation accuracies can be simultaneously improved, in contrast to its existing structure. A guideline for taking control parameters corresponding to the proposed control structure is also provided with respect to the stabilization of the overall closed-loop systems. Finally, simulation and experimental results are provided to demonstrate the validity and practical feasibility of the developed structure.

Gaze estimation provides insight into a person's intent and engagement level, which is helpful in collaborative human-robot applications. With significant advancements in deep learning architectures, appearance-based gaze estimation has gained much attention. Appearance-based methods have shown significant improvement in gaze accuracy and, unlike traditional approaches, they function well in environments where there are no constraints. We present another convolution-based gaze estimation approach to further reduce the angular error. For estimating gaze under extreme conditions such as head variations and distances, full-face images have been shown to be efficient, so we rely on full-face and pay more attention to necessary features. With the proposed architecture, we achieve an accuracy of 3.75° on the MPIIFaceGaze dataset and 3.96° on the ETH-XGaze open-source dataset. In addition, we test eye gaze tracking in real-time robotic applications, such as attention detection, and pick-and-place.

This paper proposes a new methodology to develop a time-varying group formation tracking scheme for a class of multi-agent systems (e.g. different types of multi-robot systems) utilising Negative Imaginary (NI) theory. It offers a two-loop control scheme in which the inner loop deploys an appropriate feedback linearising control law to transform the nonlinear dynamics of each agent into a double integrator system, while the outer loop applies an NI-based time-varying group formation control protocol on the linearised agents. This approach offers greater flexibility in choosing a controller, easy implementation and tuning, reduces the overall complexity of the scheme, and uses only output feedback (hence reduced sensing requirements) to achieve formation control in contrast to the existing formation control schemes. The paper has also provided lab-based experimental validation results to demonstrate the feasibility and usefulness of the proposed scheme. Two experiments were conducted on a group of small-scale quadcopters connected via a network to test the time-varying group formation tracking performance.

Redundant manipulators offer a continuum of joint configurations which satisfy a specific end-effector pose, an advantage when operating within confined spaces. This, how-ever, challenges a controller to select a single goal configuration from a wide range when path planning. This paper outlines the use of the MySQL database management system for systematic goal selection during redundant manipulator path planning in confined spaces. We outline a sampling method to envelope all configurations of a redundant manipulator and utilise it to generate a complete database of configurations. We demonstrate the application of this method to generate a large data-set of (1 billion) manipulator configurations for a KUKA LBR iiwa 14 equipped with a Robotiq 2F-85 gripper. With this database, the controller systematically selects goal configurations during 50 path planning scenarios within the confined space of a glovebox. We compare this to an iterative method using existing kinematic solvers to select goal configurations as a baseline. The database method achieves a 100% success rate in 42% of the scenarios attempted. In comparison, the baseline method achieves >50% success rate in just 6% of the scenarios attempted. Our proposed method also produces repeatable paths, which are similar in length and link swept area for each attempt of the same scenario, whereas the baseline method generates a different path in every attempt.

Modeling dynamics is often the first step to making a vehicle autonomous. While on-road autonomous vehicles have been extensively studied, off-road vehicles pose many challenging modeling problems. An off-road vehicle encounters highly complex and difficult-to-model terrain/vehicle interactions, as well as having complex vehicle dynamics of its own. These complexities can create challenges for effective high-speed control and planning. In this paper, we introduce a framework for multistep dynamics prediction that explicitly handles the accumulation of modeling error and remains scalable for sampling-based controllers. Our method uses a specially-initialized Long Short-Term Memory (LSTM) over a limited time horizon as the learned component in a hybrid model to predict the dynamics of a 4-person seating all-terrain vehicle (Polaris S4 1000 RZR) in two distinct environments. By only having the LSTM predict over a fixed time horizon, we negate the need for long term stability that is often a challenge when training recurrent neural networks. Our framework is flexible as it only requires odometry information for labels. Through extensive experimentation, we show that our method is able to predict millions of possible trajectories in real-time, with a time horizon of five seconds in challenging off road driving scenarios.

In-hand manipulation is challenging for a multi-finger robotic hand due to its high degrees of freedom and complex interaction with the object. To enable in-hand manipulation, existing deep reinforcement learning-based approaches mainly focus on training a single robot-structure-specific policy through the centralized learning mechanism, lacking adaptability to changes like robot malfunction. To solve this limitation, this work treats each finger as an individual agent and trains multiple agents to control their assigned fingers to complete the in-hand manipulation task cooperatively. We propose the Multi-Agent Global-Observation Critic and Local-Observation Actor (MAGCLA) method, where the critic can observe all agents' actions globally, and the actor only locally observes its neighbors' actions. Besides, conventional individual experience replay may cause unstable cooperation due to the asynchronous performance increment of each agent, which is critical for in-hand manipulation tasks. To solve this issue, we propose the Synchronized Hindsight Experience Replay (SHER) method to synchronize and efficiently reuse the replayed experience across all agents. The methods are evaluated in two in-hand manipulation tasks on the Shadow dexterous hand. The results show that SHER helps MAGCLA achieve comparable learning efficiency to a single policy, and the MAGCLA approach is more generalizable in different tasks. The trained policies have higher adaptability in the robot malfunction test compared to the baseline multi-agent and single-agent approaches.

This paper implements a vision-based moving target tracking system of quadrotors with visual-inertial localization in GNSS-denied indoor environments. We use the visual-inertial odometry to estimate the states of the UAV by minimizing visual and inertial residuals, and estimate the states of the target with extended Kalman Filter from visual detection. This research formulates the target tracking problem as optimization-based trajectory generation where a weighted sum cost function jointly penalizes the tracking error, the control cost of the trajectory and the trajectory length, while enforcing the safety and feasibility constraints. We present a strategy that represents the trajectory as piecewise Bézier curves using Bernstein polynomial basis. Due to the special properties of Bézier curves, the position of the entire trajectory and its derivatives can be directly bounded within the safe spaces, thus this facilitating the dynamics of the quadrotor. The proposed strategy can generate smooth and collision-free tracking trajectories and is time and space efficient. We conduct simulations and real-world experiments to validate the effectiveness of our system.

Human gait is quite complex, especially when considering the irregular and uncertain environments that humans are able to walk in. While unperturbed gait in a controlled environment is understood to a large degree, gait in more unique environments, such as asymmetric compliant terrain, is not understood to the same degree. In this study, we build upon a neuromuscular gait model and extend it to allow for walking on unilaterally compliant (soft) surfaces. This model is then compared to and verified by experimental human data. The model can successfully walk with step length trends similar to human data. Additionally, the model shows similar behaviors with respect to kinematics and muscle activity. We believe this work contributes significantly to a better understanding of the control of human gait and could lead to model-informed, patient-specific rehabilitation strategies that can advance the field of rehabilitation robotics, as well as the development of bio-inspired controllers for bipedal robots that would be able to traverse through dynamic and complaint terrains.

In conjunction with huge recent progress in cam-era and computer vision technology, camera-based sensors have increasingly shown considerable promise in relation to tactile sensing. In comparison to competing technologies (be they resistive, capacitive or magnetic based), they offer super-high-resolution, while suffering from fewer wiring problems. The human tactile system is composed of various types of mechanoreceptors, each able to perceive and process distinct information such as force, pressure, texture, etc. Camera-based tactile sensors such as GelSight mainly focus on high-resolution geometric sensing on a flat surface, and their force measurement capabilities are limited by the hysteresis and non-linearity of the silicone material. In this paper, we present a miniaturised dome-shaped camera-based tactile sensor that allows accurate force and tactile sensing in a single coherent system. The key novelty of the sensor design is as follows. First, we demonstrate how to build a smooth silicone hemispheric sensing medium with uniform markers on its curved surface. Second, we enhance the illumination of the rounded silicone with diffused LEDs. Third, we construct a force-sensitive mechanical structure in a compact form factor with usage of springs to accurately perceive forces. Our multi-modal sensor is able to acquire tactile information from multi-axis forces, local force distribution, and contact geometry, all in real-time. We apply an end-to-end deep learning method to process all the information.

Endotracheal intubation is a mandatory competency for most medical staff. This procedure involves opening the entrance of the patient's upper windpipe using a laryngoscope and then inserting a tube into the windpipe to supply Oxygen to the patient. This time critical intervention requires careful control of the force vector on the tongue to lift it parallel to the jaw than to push the jaw to open the mouth. However, traditional intubation training methods in which novices practice intubation on prostheses lack haptic feedback to improve force control. We designed a sensorised intubation training phantom that can provide trainees with vibrotactile feedback reflecting the laryngoscope's force on the tongue. The critical component of this phantom is a silicon rubber tongue embedded with magnets and hall effect sensors. We calibrated the hall effect sensor readings to predict the force vector exerted on the tongue with errors less than 0.5 N in the lifting and pushing directions. We conducted a controlled experiment, mainly comparing the training results between participants with and without haptic feedback. Results show a statistically significant drop in the undesired forces due to haptic feedback, and the skill is retained when tested after 24 hours without haptic feedback.

Person re-identification plays a key role in applications where a mobile robot needs to track its users over a long period of time, even if they are partially unobserved for some time, in order to follow them or be available on demand. In this context, deep-learning-based real-time feature extraction on a mobile robot is often performed on special-purpose devices whose computational resources are shared for multiple tasks. Therefore, the inference speed has to be taken into account. In contrast, person re-identification is often improved by architectural changes that come at the cost of significantly slowing down inference. Attention blocks are one such example. We will show that some well-performing attention blocks used in the state of the art are subject to inference costs that are far too high to justify their use for mobile robotic applications. As a consequence, we propose an attention block that only slightly affects the inference speed while keeping up with much deeper networks or more complex attention blocks in terms of re-identification accuracy. We perform extensive neural architecture search to derive rules at which locations this attention block should be integrated into the architecture in order to achieve the best trade-off between speed and accuracy. Finally, we confirm that the best performing configuration on a re-identification benchmark also performs well on an indoor robotic dataset.

Collision evaluation is of essential importance in various applications. However, existing methods are either cumbersome to calculate or not exact. Therefore, considering the cost of implementation, most whole-body planning works, which require evaluating collision between robots and environments, struggle to tradeoff between accuracy and computationally efficiency. In this paper, we propose a zero-gap whole-body collision evaluation that can be formulated as a low-dimensional linear programming. This evaluation can be solved analytically in linear complexity. Moreover, the method provides gradient efficiently, making it accessible to optimization-based applications. Additionally, this method provides support for obstacles represented by either points or hyperplanes. Experiments on the widely used aerial and car-like robots validate the versatility and practicality of our method.

A novel kinematically redundant 6+1-degree-of-freedom (dof) spatial hybrid parallel robot is proposed. Each of the two legs of the robot has a fully parallel structure to minimize the moving inertia by mounting actuators on the base. The kinematic model of each leg and overall robot architecture is developed based on the constraint conditions of the robot geometry. The singularity analysis of legs 1 and 2 reveals that their serial and parallel singularities can be avoided by properly dimensioning the robot and sacrificing the edge of the workspace. In addition, it is shown that the type II (parallel) singularities can be completely avoided, resulting in a large orientational workspace. The gripping mechanism is then introduced which is operated by the redundant degree of freedom of the robot. A CAD model of the robot and a computer animation are provided to demonstrate the positioning and orientation of the robot and the gripping function.

We focus on the task of language-conditioned grasping in clutter, in which a robot is supposed to grasp the target object based on a language instruction. Previous works separately conduct visual grounding to localize the target object, and generate a grasp for that object. However, these works require object labels or visual attributes for grounding, which calls for handcrafted rules in planner and restricts the range of language instructions. In this paper, we propose to jointly model vision, language and action with object-centric representation. Our method is applicable under more flexible language instructions, and not limited by visual grounding error. Besides, by utilizing the powerful priors from the pre-trained multi-modal model and grasp model, sample efficiency is effectively improved and the sim2real problem is relived without additional data for transfer. A series of experiments carried out in simulation and real world indicate that our method can achieve better task success rate by less times of motion under more flexible language instructions. Moreover, our method is capable of generalizing better to scenarios with unseen objects and language instructions.

Dexterity is demanded for an endoscopic tool to handle complicated procedures in neurosurgery, e.g., removing diseased tissue from inside the deep brain along a tortuous path. Current robotic tools are either rigid or lack wristed motion ability at the tip, leading to limited usage in minimally invasive procedures. In this paper, a hybrid steerable robot with a magnetic wristed forceps is proposed to provide enhanced dexterity for endoscopic epilepsy surgery. A set of three precurved Nitinol tubes with concentric deployment, called a concentric tube robot (CTR), serves as a 6 degrees-of-freedom (DoF) robotic positioner. The magnetic wristed forceps is composed of a rotational wrist joint, and forceps at the tip, both of which are actuated remotely by magnetic fields. The magnetic wrist and forceps provide an extra rotational DoF and a gripping DoF on top of the CTR, respectively. The magnetic wrist and gripper are designed to have a hollow channel along their common axis, inside which a soft tube is deployed as a second functional tool for irrigation or suction. An electromagnetic navigation system (eMNS) with 8 coils is used to create the quasi-static magnetic fields. Experimental characterization of the robot kinematics is performed and the results show the mean motion error of CTR is 2.8 mm. The workspace is also analyzed and results indicate that the proposed hybrid robot has a significantly larger reachable area compared to the one of the CTR alone. Mock epilepsy procedures are performed on a brain phantom to validate the feasibility of the hybrid robot for neurosurgery applications.

We present a new framework for implementing real-time embedded safety-critical controllers which utilizes hybrid computing to address the issue of limited computational resources, a problem that is particularly prevalent in microrobotics. In our approach, the nominal stabilizing control algorithm is implemented digitally while the safety-critical quadratic program is solved via a dedicated analog resistor array. We apply this hybrid computing architecture to a simulated collision avoidance task for a micro-aerial vehicle and show the benefit relative to a purely-digital implementation. By leveraging analog quadratic programming on the Crazyflie 2.1 micro quadrotor, a reduction in overall processing time from 8.9 ms to 0.6 ms is estimated for this computationally-limited system. We further display the viability of our proposed safety-critical control framework through real-time flight demonstrations, utilizing a novel prototype analog circuit tethered to the Crazyflie. The flight results confirm the functionality of the control structure and prototype circuit while highlighting the overall capabilities of hybrid computing.