IROS 2022 Papers

While automated research vehicles are already populating the roads, their commercial availability at scale is still to come. Presumably, one of the key challenges is to derive behaviors that are safe and comfortable but at the same time not overcautious, despite considerable uncertainties. These uncertainties stem from imperfect perception, occlusions and limited sensor range, but also from the unknown future behavior of other traffic participants. A holistic uncertainty treatment, for example in a general POMDP formulation, often induces a strong limitation on the action space due to the need for real-time capability. Further, related approaches often do not account for the need for verifiable safety, including traffic rule compliance. The proposed approach is targeted towards scenarios with clear precedence. It is based on an MDP with an action-independent belief (AIB-MDP): We assume that the future belief over the trajectories of other traffic participants is independent of the ego vehicle's behavior. Thus, the future belief can be predicted and simplified in an upstream module, independent of motion planning. This modularization facilitates subsequent ego motion planning in a continuous action space despite the thorough uncertainty consideration. The improved performance compared to state-of-the-art is demonstrated in three example scenarios.

Motion estimation approaches typically employ sensor fusion techniques, such as the Kalman Filter, to handle individual sensor failures. More recently, deep learning-based fusion approaches have been proposed, increasing the performance and requiring less model-specific implementations. However, current deep fusion approaches often assume that sensors are synchronised, which is not always practical, especially for low-cost hardware. To address this limitation, in this work, we propose AFT-VO, a novel transformer-based sensor fusion architecture to estimate VO from multiple sensors. Our framework combines predictions from asynchronous multi-view cameras and accounts for the time discrepancies of measurements coming from different sources. Our approach first employs a Mixture Density Network (MDN) to estimate the probability distributions of the 6-DoF poses for every camera in the system. Then a novel transformer-based fusion module, AFT-VO, is introduced, which combines these asynchronous pose estimations, along with their confidences. More specifically, we introduce Discretiser and Source Encoding techniques which enable the fusion of multi-source asynchronous signals. We evaluate our approach on the popular nuScenes and KITTI datasets. Our experiments demonstrate that multi-view fusion for VO estimation provides robust and accurate trajectories, outperforming the state of the art in both challenging weather and lighting conditions.

Communication between robots and the server is a major problem for cloud robotic systems. In this paper, we address the problem caused by data loss during such communications and propose an efficient buffering algorithm, called AFR, to solve the problem. We model the problem into an optimization problem to maximize the received Quantity of Information (QoI). Our AFR algorithm is formally proved to achieve near-optimal QoI, which has a lower bound that is a constant multiple of the unrealizable optimal QoI. We implement our AFR algorithm in ROS without changing the API for the applications. Our experiments on two cloud robot applications show that our AFR algorithm can efficiently and effectively reduce the impact of data loss. For the remote mapping application, the RMSE caused by data loss can be reduced by about 20%. For the remote tracking application, the probability of tracking failure caused by data loss can be reduced from about 40 %-60 % to under 10%. Meanwhile, our AFR algorithm introduces time overhead of under 10 microseconds.

ACEFusion is the first 3D reconstruction system able to capture the geometry and semantics of dynamic scenes using an RGB-D camera in real-time on a robotic computing platform. Harnessing the hardware accelerators of an Nvidia Jetson AGX Xavier, the system uses heterogeneous computing to achieve 30 FPS under a 30W power budget. Using a data-parallel design, we perform most image computation on the dedicated hardware accelerators, freeing the general purpose cores and GPU to process 3D geometry. To further increase efficiency, we employ a hybrid geometry representation with octrees for static-semantic reconstruction and surfels for dynamic reconstruction. ACEFusion achieves competitive results on standard benchmarks while efficiently performing a more complex overall task than existing SLAM techniques. Figure. 1 shows the output of our system on a dynamic sequence.

Multi-agent path finding in dynamic environments is of great academic and practical value for multi-robot systems in the real world. To improve the effectiveness and efficiency of the learning process during path planning in dynamic environments, we introduce an algorithm called Attention and BicNet based Multi-agent path planning with effective reinforcement (AB-Mapper) under the actor-critic reinforcement learning framework. In this framework, on one hand, we design an actor-network that can utilize the BicNet with communication function to achieve the intra-team coordination. On the other hand, we propose a critic network that can selectively allocate attention weights to surrounding agents. This attention mechanism allows an individual agent to automatically learn a better evaluation of actions by considering the behaviours of its surrounding agents. Compared with the SOTA method Mapper in crowded environments with dynamic obstacles, our AB-Mapper is more effective (90.27±0.06% vs. 61.65±13.90% in terms of mean success rate) in solving the general multi-agent path finding problem.

We present an accessible robot interface for telemanipulation (A-RIFT), which preserves the haptic channel partially in a zero-additional-cost interface by visual substitution of force feedback (VSFF). This work explores a gap in the literature, resulting from the focus on performance improvements in telerobotics at increasing interface costs. Unlike most telemanipulation interfaces for high-degree-of-freedom robotic systems, this one requires minimal training and can be run in a web browser under high latency conditions, using an Internet connected computer with the user's own mouse and keyboard. To evaluate the performance of the system, we ran a controlled user study (N=12) to test how different distances (local vs. remote) and VSFF (on vs. off) affect the system's usability. As expected, participants in remote conditions performed worse than those in closer proximity. Despite several participants claiming that the visual display of force feedback did not help them, our analysis of their task performance showed that operators in remote condition actually performed statistically significantly better with the visual force feedback display than without it. These results indicate a promising new interface design direction for low-cost telemanipulation.

This paper presents a wearable device for finger assessment that can identify finger joint impedance parameters through harmonic oscillation perturbations. This device is designed to help assess motor impairments related to hypertonic soft-tissue changes, that can arise from a number of conditions such as stroke. By measuring the ratio of the applied torque and resulting velocities, the impedance values for any bending direction of a metacarpophalangeal (MCP) joint can be estimated. The ability of this device to effectively estimate finger parameters was tested in experiments with six participants. The experimental result was validated through comparison to prior works on finger impedance estimation. The user experience of the presented system was also analysed, indicating that the device design is comfortable and acceptable for participants.

This paper presents a unified approach to the design of Model Predictive Controllers (MPC), custom-tailored for path following by Automated Guided Vehicles (AGVs). The approach can be applied in a unified manner to several relevant AGV kinematic configurations, including tricycle, differential, and double steer-drive. By leveraging Linear Parameter Varying (LPV) MPC, it provides maximum maneuverability and industrial-grade positioning accuracy. We incorporate state-of-the-art optimized velocity planning, to maximize vehicle utilization. Experimental validation is performed on three different kinematic configurations, including a real forklift with tricycle configuration, using industrially-relevant positioning maneuvers.

Logistics warehouses face the challenge of fulfilling large bulk pick orders limit in a given time, as the information of logistics orders is different and timeliness. Therefore, in automated warehouses, it is imperative to improve the efficiency and intelligence of order picking by robotic systems. However, the existing automated guided vehicle (AGV) system has only a few fixed functions (such as order sorting and transportation, etc.), which cannot be changed in time according to actual needs. Meanwhile, the scheduling algorithm has only mass heuristics results and a few approximate algorithm results. In this paper, we build a new AGV system, including two kinds of shelves and four kinds of stations, where the system can add new station types according to the actual situation. We establish the equivalent relationship between the order group picking task in this system and the multi-stage hybrid flow shop scheduling problem, without considering the order group transfer process between stations. Furthermore, we propose a polynomial time approximation scheme (PTAS) for the scheduling problem in this system which has been proved to be strongly NP-hard [13].

Insects can locomote readily in challenging environments, such as over steep inclines and across obstacle-laden terrains, which still frustrate robots of similar size. In this work, inspired by the passive compliant properties of insect limbs, we use the insect-scale Harvard Ambulatory Microrobot and multilayer microfabrication techniques as platform to study the ability of passive mechanisms to improve open-loop running in rough terrains. We tested the performance of different limb designs in vivo, exploring how the magnitude, directionality, and distribution of compliance incorporated into the leg impacted robot performance. Limbs were evaluated on both a featureless substrate and an increasingly-adversarial 3D-printed terrain designed to mimic natural environments. We tested the limbs using a trotting gait in the quasi-static (2 Hz) and body dynamics (25 Hz) stride frequency regimes. Performance was reported as bodylengths traveled per gait cycle on the featureless substrate, and as the largest feature height the robot was able to overcome in the terrain. The work presented here provides design principles for a passive limb that expands the terrain accessible to small robot; we find a limb with a single asymmetrical joint is able to improve quasi-static terrain traversal by 203% relative to a rigid limb.

We consider the problem of controlling a bilateral leader-follower robotic surgical set-up to allow kinesthetic haptic feedback to the user when the instrument approaches a forbidden area like sensitive organs arteries or veins that should be protected from injuries during surgery. The leader is a haptic device while the follower is a general purpose manipulator holding an elongated tool with an articulated instrument that should be manipulated through an entry port. We propose a control framework that is proved passive, incorporating a target admittance model for the follower that is designed in a way to impose a remote center of motion (RCM) while being subject to repulsive forces generated by properly designed artificial potentials associated with forbidden areas. Simulation and experimental results utilizing a virtual intraoperative environment provided as a point cloud of a kidney and its surrounding vessels characterized as forbidden areas, validate and demonstrate the performance of the proposed control scheme.

Among primates, the prehensile nature of the hand is vital for greater adaptability and a secure grip over the substrate/branches, particularly for arm-swinging motion or brachiation. Though various brachiation mechanisms that are mechanically equivalent to underactuated pendulum models are reported in the literature, not much attention has been given to the hand design that facilitates both locomotion and within-hand manipulation. In this paper, we propose a new robotic gripper design, equipped with shape conformable active gripping surfaces that can act as an active or passive joint and adapt to substrates with different shapes and sizes. A floating base serial chain, named GraspMaM, equipped with two such grippers, increases the versatility by performing a range of locomotion and manipulation modes without using dedicated systems. The unique gripper design allows the robot to estimate the passive joint state while arm-swinging and exhibits a dual relationship between manipulation and locomotion. We report the design details of the multimodal gripper and how it can be adapted for the brachiation motion assuming it as an articulated suspended pendulum model. Further, the system parameters of the physical prototype are estimated, and experimental results for the brachiation mode are discussed to validate and show the effectiveness of the proposed design.

Flapping wing insects benefit from a compliant thorax that provides elastic energy exchange and resiliency to wing collisions. In this paper, we present a flapping wing robot that uses an underactuated compliant transmission inspired by the insect thorax. We developed a novel fabrication method that combines carbon fiber (CF) laminate and soft robotics fabrication techniques for transmission construction. The transmission design is optimized to achieve desired wingstroke requirements and to allow for independent motion of each wing. We validate these design choices in bench-top tests measuring transmission compliance and kinematics. We integrate the transmission with laminate wings and two types of actuation, demonstrating elastic energy exchange and limited lift-off capabilities Lastly, we tested collision mitigation through flapping wing experiments that obstructed the motion of a wing. These experiments demonstrate that an underactuated compliant, transmission can provide resilience and robustness to flapping wing robots.

Quadrupedal manipulators require to be compliant when dealing with external forces during autonomous manipulation, tele-operation or physical human-robot interaction. This paper presents a whole-body controller that allows for the implementation of a Cartesian impedance control to coordinate tracking performance and desired compliance for the robot base and manipulator arm. The controller is formulated through an optimization problem using Quadratic Programming (QP) to impose a desired behavior for the system while satisfying friction cone constraints, unilateral force constraints, joint and torque limits. The presented strategy decouples the arm and the base of the platform, enforcing the behavior of a linear double-mass spring damper system, and allows to independently tune their inertia, stiffness and damping properties. The control architecture is validated through an extensive simulation study using the 90kg HyQ robot equipped with a 7-DoF manipulator arm. Simulation results show the impedance rendering performance when external forces are applied at the arm's end-effector. The paper presents results for full stance condition (all legs on the ground) and, for the first time, also shows how the impedance rendering is affected by the contact conditions during a dynamic gait.

We create a virtual 2D tactile array for soft pneumatic actuators using embedded audio components. We detect contact-specific changes in sound modulation to infer tactile information. We evaluate different sound representations and learning methods to detect even small contact variations. We demonstrate the acoustic tactile sensor array by the example of a PneuFlex actuator and use a Braille display to individually control the contact of 29 x 4 pins with the actuator's 90 x 10 mm palmar surface. Evaluating the spatial resolution, the acoustic sensor localizes edges in x- and y-direction with a root-mean-square regression error of 1.67 mm and 0.0 mm, respectively. Even light contacts of a single Braille pin with a lifting force of 0.17 N are measured with high accuracy. Finally, we demonstrate the sensor's sensitivity to complex contact shapes by successfully reading the 26 letters of the Braille alphabet from a single display cell with a classification rate of 88 %.

We present a versatile framework for the computational co-design of legged robots and dynamic maneuvers. Current state-of-the-art approaches are typically based on random sampling or concurrent optimization. We propose a novel bilevel optimization approach that exploits the derivatives of the motion planning sub-problem (i.e., the lower level). These motion-planning derivatives allow us to incorporate arbitrary design constraints and costs in an general-purpose nonlinear program (i.e., the upper level). Our approach allows for the use of any differentiable motion planner in the lower level and also allows for an upper level that captures arbitrary design constraints and costs. It efficiently optimizes the robot's morphology, payload distribution and actuator parameters while considering its full dynamics, joint limits and physical constraints such as friction cones. We demonstrate these capabilities by designing quadruped robots that jump and trot. We show that our method is able to design a more energy-efficient Solo robot for these tasks.

To ensure real-time response to passengers, existing solutions to the vehicle dispatch problem typically optimize dispatch policies using small batch windows and ignore the spatial-temporal dynamics over the long-term horizon. In this paper, we focus on improving the long-term performance of ride-sharing services and propose a deep reinforcement learning based approach for the ride-sharing dispatch problem. In particular, this work includes: (1) an offline policy evaluation (OPE) based method to learn a value function that indicates the expected reward of a vehicle reaching a particular state; (2) an online learning procedure to update the offline trained value function to capture the real-time dynamics during the operation; (3) an efficient online dispatch method that optimizes the matching policy by considering both past and future influences. Extensive simulations are conducted based on New York City taxi data, and show that the proposed solution further increases the service rate compared to the state-of-the-art farsighted ride-sharing dispatch approach.

Fluidically actuated soft robots have promising capabilities such as inherent compliance and user safety. The control of soft robots needs to properly handle nonlinear actuation dynamics, motion constraints, workspace limitations, and variable shape stiffness, so having a unique algorithm for all these issues would be extremely beneficial. In this work, we adapt Model Predictive Control (MPC), popular for rigid robots, to a soft robotic arm called SoPrA. We address the challenges that current control methods are facing, by proposing a frame-work that handles these in a modular manner. While previous work focused on Joint-Space formulations, we show through simulation and experimental results that Task-Space MPC can be successfully implemented for dynamic soft robotic control. We provide a way to couple the Piece-wise Constant Curvature and Augmented Rigid Body Model assumptions with internal and external constraints and actuation dynamics, delivering an algorithm that unites these aspects and optimizes over them. We believe that a MPC implementation based on our approach could be the way to address most of model-based soft robotics control issues within a unified and modular framework, while allowing to include improvements that usually belong to other control domains such as machine learning techniques.

We investigate the problem of planning stable grasps for object manipulations using an 18-DOF robotic hand with four fingers. The main challenge here is the high-dimensional search space, and we address this problem using a novel two-stage learning process. In the first stage, we train an autoregressive network called the hand-pose-generator, which learns to generate a distribution of valid 6D poses of the palm for a given volumetric object representation. In the second stage, we employ a network that regresses 12D finger joint configurations and a scalar grasp quality from given object representations and palm poses. To train our networks, we use synthetic training data generated by a novel grasp planning algorithm, which also proceeds stage-wise: first the palm pose, then the finger positions. Here, we devise a Bayesian Optimization scheme for the palm pose and a physics-based grasp pose metric to rate stable grasps. In experiments on the YCB benchmark data set, we show a grasp success rate of over 83%, as well as qualitative results grasping unknown objects on a real robot system.