IROS 2021 Papers

When legged robots impact their environment, they undergo large changes in their velocities in a small amount of time. Measuring and applying feedback to these velocities is challenging, and is further complicated due to uncertainty in the impact model and impact timing. This work proposes a general framework for adapting feedback control during impact by projecting the control objectives to a subspace that is invariant to the impact event. The resultant controller is robust to uncertainties in the impact event while maintaining maximum control authority over the impact invariant subspace. We demonstrate the utility of the projection on a walking controller for a planar five-link-biped and on a jumping controller for a compliant 3D bipedal robot, Cassie. The effectiveness of our method is shown to translate well on hardware.

Recent Imitation Learning (IL) techniques focus on adversarial imitation learning algorithms to learn from a fixed set of expert demonstrations. While these approaches are theoretically sound, they suffer from a number of problems such as poor sample efficiency, poor stability, and a host of issues that Generative Adversarial Networks (GANs) suffer from. In this paper we introduce a generalization of Behavior Cloning (BC) that is applicable in any IL setting. Our algorithm first approximates behavior cloning loss using a neural network and then uses that loss network to generate a loss signal which is minimized using standard supervised learning. We call the resulting algorithm family Approximated Behavior Cloning (ABC), introduce variants for each IL setting, and demonstrate an order of magnitude improvement in sample efficiency and increased stability in standard imitation learning environments.

Many of today’s robot perception systems aim at accomplishing perception tasks that are too simplistic and too hard. They are too simplistic because they do not require the perception systems to provide all the information needed to accomplish manipulation tasks. Typically the perception results do not include information about the part structure of objects, articulation mechanisms and other attributes needed for adapting manipulation behavior. On the other hand, the perception problems stated are also too hard because — unlike humans— the perception systems cannot leverage the expectations about what they will see to their full potential. Therefore, we investigate a variation of robot perception tasks suitable for robots accomplishing everyday manipulation tasks, such as household robots or a robot in a retail store. In such settings it is reasonable to assume that robots know most objects and have detailed models of them. We propose a perception system that maintains its beliefs about its environment as a scene graph with physics simulation and visual rendering. When detecting objects, the perception system retrieves the model of the object and places it at the corresponding place in a VR-based environment model. The physics simulation ensures that object detections that are physically not possible are rejected and scenes can be rendered to generate expectations at the image level. The result is a perception system that can provide useful information for manipulation tasks.

Imaging techniques are established aids in surgical procedures and have become indispensable in modern medicine. Combined with robot-assisted systems, significantly higher precision can already be achieved today and work steps can be simplified. We hypothesize that algorithms can make the operations of the future safer and easier. The paper introduces an approach for the control of a robotic manipulator that, based on standardized imaging techniques, monitors and adapts externally given movement commands. The physician controls the system, benefits from the robot’s precision, and movements will be intervened in if unwanted contact with the environment would occur. This is important, e.g., when operating near sensitive tissue, nerves or bone that must not be injured. Objects from the image data are considered in the control scheme and may have arbitrary shape and size. Furthermore, since the approach can be in principle adapted to any shape it is shown how robot operations in confined spaces, that are challenging to achieve using an external controller, can be simplified by the introduced approach. The online evaluation takes place in Cartesian space and is then transformed into joint space using the inverse kinematics. The approach is implemented using GAZEBO and a simulation study is performed for the 6-DOF industrial manipulator Stäubli TX2-60.

Motion planning is a largely solved problem for robot arms with joint state feedback, but remains an area of research for sensorless manipulators such as toy robot arms and heavy equipment such as excavators and cranes. A promising approach to this problem is deep learning, which employs a pre-trained convolutional neural network to identify manipulator links and estimate joint states from a monocular camera video feed. Whereas manual labeling of training image sets is tedious and non-transferable, a simulation environment can automatically generate labeled training image sets of any size. The issue is the gap between simulated and real-world images. This paper solves this problem by implementing a Generative Adversarial Network. The complete joint state estimation pipeline is implemented and tested in hardware experiments to validate our proposed approach.

Many state-of-art robotics applications require fast and efficient motion planning algorithms. Existing motion planning methods become less effective as the dimensionality of the robot and its workspace increases, especially the computational cost of collision detection routines. In this work, we present a framework to address the cost of expensive primitive operations in sampling-based motion planning. This framework determines the validity of a sample robot configuration through a novel combination of a Contractive AutoEncoder (CAE), which captures an occupancy grids representation of the robot's workspace, and a Multilayer Perceptron (MLP), which efficiently predicts the collision state of the robot using the output from the CAE. We evaluate our framework on multiple planning problems with a variety of robots in 2D and 3D workspaces. The results show that (1) the framework is computationally efficient in all investigated problems, and (2) the framework generalizes well to new workspaces.

This paper addresses the problem of identifying whether/how a black-box autonomous system has regressed in performance when compared to previous versions. The approach analyzes performance datasets (typically gathered through simulation-based testing) and automatically extracts test parameter clusters of predicted performance regression. First, surrogate modeling with quantile random forests is used to predict regions of performance regression with high confidence. The predicted regression landscape is then clustered in both the output space and input space to produce groupings of test conditions ranked by performance regression severity. This approach is analyzed using randomized test functions as well as through a case study to detect performance regression in autonomous surface vessel software.

Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications. The assistance system should not only achieve high accuracy under normal conditions, but obtain robust perception against extreme situations. However, traffic accidents that involve object collisions, deformations, overturns, etc., yet unseen in most training sets, will largely harm the performance of existing semantic segmentation models. To tackle this issue, we present a rarely addressed task regarding semantic segmentation in accidental scenarios, along with an accident dataset DADA-seg. It contains 313 various accident sequences with 40 frames each, of which the time windows are located before and during a traffic accident. Every 11th frame is manually annotated for benchmarking the segmentation performance. Furthermore, we propose a novel event-based multi-modal segmentation architecture ISSAFE. Our experiments indicate that event-based data can provide complementary information to stabilize semantic segmentation under adverse conditions by preserving fine-grain motion of fast-moving foreground (crash objects) in accidents. Our approach achieves +8.2% mIoU performance gain on the proposed evaluation set, exceeding more than 10 state-of-the-art segmentation methods. The proposed ISSAFE architecture is demonstrated to be consistently effective for models learned on multiple source databases including Cityscapes, KITTI-360, BDD and ApolloScape.

Teaching robots how to apply forces according to our preferences is still an open challenge that has to be tackled from multiple engineering perspectives. This paper studies how to learn variable impedance policies where both the Cartesian stiffness and the attractor can be learned from human demonstrations and corrections with a user-friendly interface. The presented framework, named ILoSA, uses Gaussian Processes for policy learning, identifying regions of uncertainty and allowing interactive corrections, stiffness modulation and active disturbance rejection. The experimental evaluation of the framework is carried out on a Franka-Emika Panda in four separate cases with unique force interaction properties: 1) pulling a plug wherein a sudden force discontinuity occurs upon successful removal of the plug, 2) pushing a box where a sustained force is required to keep the robot in motion, 3) wiping a whiteboard in which the force is applied perpendicular to the direction of movement, and 4) inserting a plug to verify the usability for precision-critical tasks in an experimental validation performed with non-expert users.

The implementation of stable locomotion on humanoid robots is a difficult task. This is complicated by the fact that there is no uniform method for analyzing a robot and its control architecture and for calculating indicators to quantify performance of flat ground walking. Moreover, there is no widely accepted indicator do distinct between a stable and unstable state of the robot. We propose the Increased Step Size Stability Assessment (I3SA) as a testing protocol and standardized procedure for data collection and evaluation of stability for locomotion on flat terrain. We apply this test to the humanoid robot REEM-C. The biped must cover a set distance of four meters with predefined step sizes. The initial step size is defined as 20% of the total leg length of the robot. After three successful trials, the step size is continuously increased until REEM-C’s last successful trial at 40% of its total leg length, leading to REEM-C’s I3SA rating of 40. The recorded data are evaluated using metrics known from the literature, such as the capture point, foot placement estimator, and the angular momentum acting at the center of mass. We illustrate the experimental setup, data collection and processing, the calculation of performance indicators for several step sizes and the trial which resulted in a fall of the robot. The trend towards decreasing stability with increased step size and the available key assumptions are reported.

Over the years, many motion planning algorithms have been proposed. It is often unclear which algorithm might be best suited for a particular class of problems. The problem is compounded by the fact that algorithm performance can be highly dependent on parameter settings. This paper shows that hyperparameter optimization is an effective tool in both algorithm selection and parameter tuning over a given set of motion planning problems. We present different loss functions for optimization that capture different notions of optimality. The approach is evaluated on a broad range of scenes using two different manipulators, a Fetch and a Baxter. We show that optimized planning algorithm performance significantly improves upon baseline performance and generalizes broadly in the sense that performance improvements carry over to problems that are very different from the ones considered during optimization.

Realizing the potential of active lower-limb pros-theses to increase user mobility and efficiency requires safe, reliable, stable, and intuitive control strategies. The two prevailing classes of lower-limb prosthesis control can be categorized as volitional and non-volitional. Volitional control strategies (VCs) directly sense the user’s intentions, but this generally intuitive approach can be quite demanding, leading to user fatigue and device misactivation. Non-volitional control strategies (NVCs) sense the state of the system instead, often taking advantage of the gait’s cyclic nature to produce robust and reliable outputs. NVCs, however, do not give the user freedom to realize non-standard movements. This paper introduces a Hybrid Volitional Control (HVC) approach that operates across the entire gait cycle and seeks to balance the reliability, safety, and low demand of NVCs with the freedom and intuitive control of VCs. Through simulations of 2.5° ramp descent, level ground walking, and 5° ramp ascent, HVC shows an opportunity to reduce the torque error from an able-bodied reference in all cases compared to two NVCs (as much as 94%). Similarly, HVC shows an opportunity to reduce volitional demands compared to a pure VC (as much as 91%). Volitional thresholding is considered for users to ambulate reliably with minimal volitional input demands, while maintaining continuous freedom to alter the dynamics of the device. HVC could allow users to participate in a wider range of activities, helping to further erase the distinction between ability and disability from amputation.

Unmanned Aircraft System Traffic Management (UTM) becomes a highly relevant complex challenge, as the UAV activity is rapidly growing bringing more amateur and professional drones to the urban skies. The main concern of managing such a system is safely navigating and controlling hundreds or thousands of drones simultaneously, flying in a crowded dense environments. This paper introduces an innovative approach of hybrid path planning, which tries to make the best out of the commonly used centralized and decentralized planning approaches. The Hybrid Path Planner (HPP) defines two configuration spaces: the Local Zone, which represents the crowded city zone with many obstacles and constrains, and the Global Zone, which represents the outer suburban zone, mostly open space with predefined flight corridors. The HPP server communicates with each UAV, assigning it a close-to-optimal path in the global zone, while leaving the relatively heavy-duty local zone path planning task to be performed by the UAV, mostly using stochastic methods like RRT*. This approach reduces the complex path panning task of the centralized server to a simpler task of calculating only the entry and exit points to and from the global zone. This robust approach supports handling a high number of UAVs, while keeping close to optimal performance.

Mechanical rubbing of blood clot using miniature magnetic helical robots is a potential way for thrombolysis. In this paper, we report a new strategy for this issue based on mobile coils. Previously, we proposed the concept of magnetic actuation with parallel mobile coils, in which multiple coils can move in 3D space. Enabled by mobility of the coils, additional degree-of-freedom (DOF) could be utilized for actuation performance optimization. Besides the primary helical propulsion by rotating magnetic fields, our strategy aims to optimize the coil motion to make the magnetic force contributes the most to the helical robot forward motion. For this goal, modeling of the magnetic field and force of multiple mobile coils are presented, based on which an optimization algorithm is formulated to output the best coil motion. For validation, an enhanced mobile coil system having a workspace of Φ500 mm ×150 mm is constructed based on the parallel mobile coil concept. Simulations show the effectiveness of the proposed strategy, whose effective workspace for a specific task can also be obtained. After implementing the proposed strategy, preliminary experiments using clot analog demonstrate that the removal speed is accelerated over 50% compared to that without coil motion optimization.

ICP algorithms typically involve a fixed choice of data association method and a fixed choice of error metric. In this paper, we propose Hybrid ICP, a novel and flexible ICP variant which dynamically optimises both the data association method and error metric based on the live image of an object and the current ICP estimate. We show that when used for object pose estimation, Hybrid ICP is more accurate and more robust to noise than other commonly used ICP variants. We also consider the setting where ICP is applied sequentially with a moving camera, and we study the trade-off between the accuracy of each ICP estimate and the number of ICP estimates available within a fixed amount of time.

Kinematic information obtained directly from the skeletal model has been useful for jumping action recognition. Current research focuses on dynamic analysis based on the video stream. Although skeletal data can accurately capture the high-level information of human action, it ignores the brain’s pre-execution command information, which plays a crucial role in identifying jumping action. Therefore, we proposed a hybrid model based on brain network and dynamic skeleton. Specifically, we used a brain network graph convolutional network (BNGCN) to encode brain command information. Also, a dynamic skeleton convolutional network (DSGCN) using the angular velocity of skeleton nodes instead of video is proposed, which can break the fixed experimental area’s limitation. BNGCN and DSGCN are fused through three network nodes to construct an end-to-end Brain Network and Dynamic Skeleton Hybrid Model. Our contribution consists of three parts. First, we have created a data set that can be used for jumping action and its sub-phase recognition. Second, BNGCN is used to extract brain command information for jumping action recognition. Third, a hybrid model is proposed to incorporate brain command and skeleton kinematic information. The results show that our hybrid model can effectively capture the high-level features for jumping action recognition. The method outperforms compared methods for jumping action recognition.

We present a comprehensive comparison of hybrid Data-Driven Control (DDC) applied on a hydraulic excavator. DDC offers a state-of-the-art, high performance control based on data and expert knowledge. On the one hand, expert knowledge is complex to adapt to each unique excavator requiring substantial engineering efforts. On the other hand, purely data based control overcomes this drawback by adapting a Neural Network (NN) inverse control directly on measured input/output data. Yet, coverage of the entire phase space and extrapolation to unknown situations is challenging for solely data-driven approaches. On a real demonstrator, we analyze expert white box methods, solely data-driven black box approaches and a hybrid grey box approach which combines a data-driven and simplified expert model. We examine trajectory tracking performance, engineering effort and safe exploration as goal criteria. Besides various experiments for testing safety, we apply a Support-Vector-Machine (SVM) to analyze the extrapolation fitness of the data-driven components to unknown data.

Mobile social robots should be able to engage in interaction with people effectively. However, greeting someone is a complex task since it implies an exchange of social signals. Adam Kendon modeled human greetings as a set of six phases: initiation of approach, distance salutation, head dip, approach, final approach, and close salutation. Based on Kendon’s model, we propose a system for mobile social robots that manages the greeting process through the exchange of social signals. A Hidden Markov Model keeps track of the greeting stage through the observation of the human gestures, while a behavior tree generates appropriate robot actions. We used publicly available datasets to train the Hidden Markov Model. Evaluation on test sets showed an average greeting phase estimation accuracy of 80.9%. We tested the full system (Hidden Markov Model + Behavior Tree) in simulation and in a real world pilot experiment using the Vizzy robot, and it recognized and replicated the correct phase with an accuracy of 91.8% and 53.8%, respectively.

Human-robot collaboration (HRC) significantly extends robotic systems’ applications when working in spaces like houses, hospitals, or laboratories. However, new challenges appear during a close collaboration between humans and robots and imitating the movement of humans by robots. Learning from demonstration (LfD), or kinesthetic teaching, is a popular approach to help teach a robot human behavior by demonstrations without the need to explicitly reprogram the robot for different procedures. In this paper, we propose a method for object manipulation, including lifting, carrying, and lowering the object through a collaboration of a human with a wheeled mobile manipulator (WMM). The WMM is first trained with the help of a human demonstrator to collaborate with the user to execute the task. Then, the WMM will independently cooperate with the user by reproducing the learned skills to perform the same task. The redundancy of the WMM will also be employed to enhance its force exertion capability in the vertical direction to offset the object’s weight. The advantages and effectiveness of the proposed method are investigated through experiments.