ICRA 2022 Papers

Drop mitigation is an important function of micro aerial vehicles (MAVs) that are used for internal inspections of enclosed and cluttered structures (height: 5–10 m). The mechanism also allows continuous operation of drones, prevents the downtime required for maintenance and repair and also provides a safer environment for workers who are working below the drones. Some solutions include parachutes, auto-rotors, and active auto-rotors. However, these are inapplicable to MAV s with a protector shell because of their size and payload. We herein propose a new rapid response drop mitigation method for MA V s involving active autorotation with bendable wings and shells. Active autorotation enables faster deceleration during dropping motion as compared to parachutes or passive autorotation. Bendable wings can ensure optimal flight performance and enable sufficient deceleration during falling motion. This newly proposed fixture was shown to reduce the impact impulse by 32.2 % and horizontal oscillation by 34.4 %. From our flight tests conducted at heights of 5 m and 10m in both outdoor and indoor environments, the measured impact impulse for both profiles were attained at 0.93 N s. The minimum impact impulse that can cause harm to the human eye, which is the most vulnerable part, is 2 N s. Thus, this mechanism was successfully proven to provide a greater safety buffer based on the drop test conducted. We envision this mechanism to provide greater safety to drone operating environments in relevant fields involving indoor and urban drone flights. Furthermore, it can reduce damage to drones and structures and avoid injuries arising from drone crashes.

Macrophage immunotherapy is a promising clinical approach to treat cancer. However, low targeting efficiency severely limits the immunotherapeutic effect of macrophages. Here, we report a unique macrophage robot that can target and kill cancer cells using a combination of external acoustic and magnetic fields. First, the inactive macrophages (Mø) are magnetized by endocytosis of the \gamma\gamma-Fe2O3 nanoparticles (FeNPs). Then, the magnetized M⊘can be moved towards the capillary wall under the influence of an acoustic radiation force generated from a lead zirconate titanate piezoelectric (PZT) transducer. Finally, the magnetized cells rotate forward under the action of alternating magnetic fields (AMF). During the process of magnetizing macrophages, FeNPs activate the anti-tumor immune activity of macrophages (M1) to induce cancer cell death. Overall, the present study highlights a novel cell robot that can target and kill cancer cells. Considering that the nanoparticles, macrophages, magnetic fields, and ultrasound technology have all been FDA approved for clinical settings, our targeted delivery system has tremendous clinical translational potential.

Multi-perspective cameras are quickly gaining importance in many applications such as smart vehicles and virtual or augmented reality. However, a large system size or absence of overlap in neighbouring fields-of-view often complicate their calibration. We present a novel solution which relies on the availability of an external motion capture system. Our core contribution consists of an extension to the hand-eye calibration problem which jointly solves multi-eye-to-base problems in closed form. We furthermore demonstrate its equivalence to the multi-eye-in-hand problem. The practical validity of our approach is supported by our experiments, indicating that the method is highly efficient and accurate, and outperforms existing closed-form alternatives.

For model-free deep reinforcement learning of quadruped locomotion, the initialization of robot configurations is crucial for data efficiency and robustness. This work focuses on algorithmic improvements of data efficiency and robustness simultaneously through automatic discovery of initial states, which is achieved by our proposed K-Access algorithm based on accessibility metrics. Specifically, we formulated accessibility metrics to measure the difficulty of transitions between two arbitrary states, and proposed a novel K-Access algorithm for state-space clustering that automatically discovers the centroids of the static-pose clusters based on the accessibility metrics. By using the discovered centroidal static poses as the initial states, we can improve data efficiency by reducing redundant explorations, and enhance the robustness by more effective explorations from the centroids to sampled poses. Focusing on fall recovery as a very hard set of locomotion skills, we validated our method extensively using an 8-DoF quadrupedal robot Bittle. Compared to the baselines, the learning curve of our method converges much faster, requiring only 60% of training episodes. With our method, the robot can successfully recover to standing poses within 3 seconds in 99.4% of the test cases. Moreover, the method can generalize to other difficult skills successfully, such as backflipping.

Semantic segmentation is a core ability required by autonomous agents, as being able to distinguish which parts of the scene belong to which object class is crucial for navigation and interaction with the environment. Approaches which use only one time-step of data cannot distinguish between moving objects nor can they benefit from temporal integration. In this work, we extend a backbone LatticeNet to process temporal point cloud data. Additionally, we take inspiration from optical flow methods and propose a new module called Abstract Flow which allows the network to match parts of the scene with similar abstract features and gather the information temporally. We obtain state-of-the-art results on the SemanticKITTI dataset that contains LiDAR scans from real urban environments. We share the PyTorch implementation of TemporalLatticeNet at https://github.com/AIS-Bonn/temporal_latticenet.

The occupancy grid map is a critical component of autonomous positioning and navigation in the mobile robotic system, as many other systems' performance depends heavily on it. To guarantee the quality of the occupancy grid maps, researchers previously had to perform tedious manual recognition for a long time. This work focuses on automatic abnormal occupancy grid map recognition using the residual neural network with novel attention mechanism modules. We propose an effective channel and spatial Residual Squeeze-and-Excitation (csRSE) attention module, which contains a residual block for producing hierarchical features, followed by both channel SE (cSE) block and spatial SE (sSE) block for the sufficient information extraction along the channel and spatial pathways. To further summarize the occupancy grid map characteristics and experiments with our csRSE attention modules, we constructed a dataset called occupancy grid map dataset (OGMD) for our experiments. On this OGMD test dataset, we tested a few variants of our proposed structure and compared them with other attention mechanisms. Our experimental results show that the proposed attention network can infer the abnormal map with state-of-the-art (SOTA) accuracy of 96.23% for abnormal occupancy grid map recognition.

Building assistive interfaces for controlling robots through arbitrary, high-dimensional, noisy inputs (e.g., webcam images of eye gaze) can be challenging, especially when it involves inferring the user's desired action in the absence of a natural ‘default’ interface. Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem, and enables the interface to adapt to individual users. However, this approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse. We propose a hierarchical solution that learns efficiently from sparse user feedback: we use offline pre-training to acquire a latent embedding space of useful, high-level robot behaviors, which, in turn, enables the system to focus on using online user feedback to learn a mapping from user inputs to desired high-level behaviors. The key insight is that access to a pre-trained policy enables the system to learn more from sparse rewards than a naïve RL algorithm: using the pre-trained policy, the system can make use of successful task executions to relabel, in hindsight, what the user actually meant to do during unsuccessful executions. We evaluate our method primarily through a user study with 12 participants who perform tasks in three simulated robotic manipulation domains using a webcam and their eye gaze: flipping light switches, opening a shelf door to reach objects inside, and rotating a valve. The results show that our method successfully learns to map 128-dimensional gaze features to 7-dimensional joint torques from sparse rewards in under 10 minutes of online training, and seamlessly helps users who employ different gaze strategies, while adapting to distributional shift in webcam inputs, tasks, and environments

Automated laboratory experiments have the potential to propel new discoveries, while increasing reproducibility and improving scientists' safety when handling dangerous materials. However, many automated laboratory workflows have not fully leveraged the remarkable advancements in robotics and digital lab equipment. As a result, most robotic systems used in the labs are programmed specifically for a single experiment, often relying on proprietary architectures or using unconventional hardware. In this work, we tackle this problem by proposing a novel robotic system architecture specifically designed with and for chemists, which allows the scientist to easily reconfigure their setup for new experiments. Specifically, the system's strength is its ability to combine together heterogeneous robotic platforms with standard laboratory equipment to create different experimental setups. Finally, we show how the architecture can be used for specific laboratory experiments through case studies such as solubility screening and crystallisation.

This paper is focused on safe mapless navigation of mobile robots in unknown and possibly complex environments containing both internal and dynamic obstacles. We present a novel modular approach that combines the strengths of artificial potential functions (APF) with deep reinforcement learning. Differing from related work, the robot learns how to adjust the two input parameters of the APF controller as necessary through soft actor-critic algorithm. Environmental complexity measures are introduced in order to ensure that the robot's training covers a range of learning scenarios that vary in regard to maneuvering difficulty. Our experimental results show that differing from the classical navigation methods and end-to-end models, the robot can navigate successfully on its own even in complex scenarios with moving entities without requiring any maps.

Heuristic search-based motion planning algorithms typically discretise the search space in order to solve the shortest path problem. Their performance is closely related to this discretisation. A fine discretisation allows for better approximations of the continuous search space, but makes the search for a solution more computationally costly. A coarser resolution might allow the algorithms to find solutions quickly at the expense of quality. For large state spaces, it can be beneficial to search for solutions across multiple resolutions even though defining the discretisations is challenging. The recently proposed algorithm Multi-Resolution A* (MRA*) searches over multiple resolutions. It traverses large areas of obstacle-free space and escapes local minima at a coarse resolution. It can also navigate so-called narrow passageways at a finer resolution. In this work, we develop AMRA*, an anytime version of MRA*, AMRA* tries to find a solution quickly using the coarse resolution as much as possible. It then refines the solution by relying on the fine resolution to discover better paths that may not have been available at the coarse resolution. In addition to being anytime, AMRA* can also leverage information sharing between multiple heuristics. We prove that AMRA* is complete and optimal (in-the-limit of time) with respect to the finest resolution. We show its performance on 2D grid navigation and 4D kinodynamic planning problems.

In this paper, we develop a novel adaptive motion imitation algorithm (AMI) for robotic systems. Although AMI can be used in a variety of human-robot interaction scenarios, we are particularly interested in robotic rehabilitation where the robot plays the role of demonstrating and practicing challenging motion physiotherapy. During therapy, the robot first demonstrates a reference trajectory to the patient that needs to be repeated during practice and then adapts its motion to a cyclic speed and amplitude based on the patient's abilities. Using this algorithm, the robotic system learns an upper-body motion of the human user and performs a unique, similar, and easier motion based on the learned trajectory from the user. Adaptation in the AMI is based on deep reinforcement learning with deep deterministic policy gradient implemented in the Robot Operating System (ROS) environment. Experimental data collected from 11 users during upper body human-robot imitation sessions with social robot Zeno was used to show that the algorithm can learn reference elbow joint trajectories of the user in an off-line manner after just a few cycles. Finally, we also implemented the algorithm online using the Baxter robot to demonstrate its learning and playback performance.

In this work, we propose A2DIO, a novel hybrid neural network model with a set of carefully designed attention mechanisms for pose invariant inertial odometry. The key idea is to extract both local and global features from the window of IMU measurements for velocity prediction. A2DIO leverages the convolutional neural network (CNN) to capture the sectional features and long-short term memory (LSTM) recurrent neural network to extract long-range dependencies. In both CNN and LSTM modules, attention mechanisms are designed and embedded for better model representation. Specifically, in the CNN attention block, the convolved features are refined along both channel and spatial dimensions, respectively. For the LSTM module, softmax scoring is applied to update the weights of the hidden states along the temporal axis. We evaluate A2DIO on the benchmark with the largest and most natural IMU data, RoNIN. Extensive ablation experiments demonstrate the effectiveness of our A2DIO model. Compared with the state of the art, the 50th percentile accuracy of A2DIO is 18.21 % higher and the 90th percentile accuracy is 21.15 % higher for all the phone holders not appeared in the training set.

This paper is concerned with the problem of estimating (interpolating and smoothing) the shape (pose and the six modes of deformation) of a slender flexible body from multiple camera measurements. This problem is important in both biology, where slender, soft, and elastic structures are ubiquitously encountered across species, and in engineering, particularly in the area of soft robotics. The proposed mathematical formulation for shape estimation is physics-informed, based on the use of the special Cosserat rod theory whose equations encode slender body mechanics in the presence of bending, shearing, twisting and stretching. The approach is used to derive numerical algorithms which are experimentally demonstrated for fiber reinforced and cable-driven soft robot arms. These experimental demonstrations show that the methodology is accurate (<5 mm error, three times less than the arm diameter) and robust to noise and uncertainties.

Connection mechanisms are crucial in reconfigurable robots. In this work, we present a novel approach, based on the self-healing property of a hydrogel synthesized by our group, which allows us to easily attach and detach robotic modules using water as the only trigger element. Our connection mechanism does not need external energy to work and it is reversible and soft, being useful for soft modular robots. Tensile, fatigue and adhesion tests are presented to demonstrate the mechanical performance of our mechanism. Two modular soft robots, manipulator and snake, are featured to show the functionality of our approach.

In this work, a robot control method is proposed for dragging an object by applying top contact forces under unknown friction and object dynamics. This is a non-prehensile manipulation of an object that can enhance the grasping capabilities of a robotic manipulator in a plethora of grasping scenarios. In the proposed method, an initializing controller generates reference contact force trajectories until a desired contact motion status is achieved that enables dragging the object without slippage of the robot tip. Based on these forces, a sufficient Virtual Friction Cone (VFC) is calculated which allows proper position control of the object with no further slippage of the robotic tip. The proposed method is validated via simulations, where contact forces are simulated with the elasto-plastic friction model, and experiments with a KUKA robot dragging a variety of objects with different dynamics and surface friction.

The field of physical human-robot interaction has dramatically evolved in the last decades. As a result, the robotic system's requirements have become more challenging, including personalized behavior for different tasks and users. Various machine learning techniques have been proposed to give the robot such adaptability features. This paper proposes a model-based evolutionary optimization algorithm to tune the apparent impedance of a wrist rehabilitation device. We used passivity to define boundaries for the possible controller outcomes, limiting the shared autonomy of the robot and ensuring the coupled system stability. The experiment consists of a hardware-in-the-loop optimization and a one-degree-of-freedom robot used for wrist rehabilitation. Experimental results showed that the proposed technique could generate customized passive impedance controllers for three subjects. Furthermore, when compared with a constant impedance controller, the method suggested decreased in 20% the root mean square of interaction torques while maintaining stability during optimization.

Beetles can walk smoothly on the meshed surface without slipping or getting stuck in the meshed surface due to its stiffness-variable tarsi and expandable hooks on the tip of tarsi. In this study, we find that beetles bend and open their claws proactively to walk freely. Inspired by the mechanism, we designed a centimeter-scale climbing robot, equipping an artificial claw to open and bend in the same cyclic manner as the natural beetles. The robot can climb freely on the mesh surface of 30° without being stuck at a speed of 26.18 mm/s (0.3 body length per second), and the speed was 37.5 mm/s on the 55-degree rough slop. This is the first demonstration of a centimeter-scale robot that can climb on the mesh surface.

In both teleoperation in real space and exploration in virtual space, ‘passive’ and ‘active’ haptic feedback can help to improve the performance of the task, especially in object handover and exploring. However, the current wearable haptic devices are hard to display continuous omnidirectional motion feedback simultaneously, which makes it not yet achieved. In this study, we thus propose a cutaneous haptic device, which enables continuous omnidirectional motion feedback for exhibiting ‘active’ and ‘passive’ haptic feedback. By applying small smart actuators (i.e., piezo actuators), the device can obtain contact force and be wearable. By arranging the closed loop with a plain-woven structure, our device makes continuous omnidirectional motion feedback possible. Our 35 g device can generate 0.94 N contact force and 0.5 N shear force. The passive and active haptic evaluations also proved its haptic capability. In conclusion, our proposed device with ‘active’ and ‘passive’ haptic feedback can provide continuous omnidirectional motion making it possible to be used for precise teleoperation.

This paper introduces an intelligent system which composes music following the users' instructions. Current auto-matic music generation models are lack of stability. Meanwhile, they cannot satisfy the preference of different people. To overcome these challenges, we train a Transformer-based neural network to generate short music segments using a dataset. A user can compose music pieces by interacting with a well-trained generator. Our system collects the user's feedback during the interactions, and fine-tunes the neural network to optimize the generator. After a large number of interactions, our system can learn the musical taste of the user and customize a personal automatic music composer for him or her. Our work enhances the application value of generative models significantly, which enables people to compose music with the assistance of artificial intelligence.