ICRA 2021 Papers

The problem of dynamic locomotion over rough terrain requires both accurate foot placement together with an emphasis on dynamic stability. Existing approaches to this problem prioritize immediate safe foot placement over longer term dynamic stability considerations, or relegate the coordination of foot placement and dynamic stability to heuristic methods. We propose a multi-layered locomotion framework that unifies Control Barrier Functions (CBFs) with Model Predictive Control (MPC) to simultaneously achieve safe foot placement and dynamic stability. Our approach incorporates CBF based safety constraints both in a low frequency kinodynamic MPC formulation and a high frequency inverse dynamics tracking controller. This ensures that safety-critical execution is considered when optimizing locomotion over a longer horizon. We validate the proposed method in a 3D stepping-stone scenario in simulation and experimentally on the ANYmal quadruped platform.

We present a novel method for handling uncertainty about the intentions of non-ego players in trajectory games, with application to motion planning for autonomous vehicles. Our method models the uncertainty about the intention of other agents by constructing multiple hypotheses about the objectives and constraints of other agents in the scene. For each candidate hypothesis, we associate a Bernoulli random variable representing the probability of that hypothesis, which may or may not be independent of the probability of other hypotheses. We leverage constraint asymmetries and feedback information patterns to incorporate the probabilities of hypotheses in a natural way. Specifically, increasing the probability associated with a given hypothesis from 0 to 1 shifts the responsibility of collision avoidance from the hypothesized agent to the ego agent. This method allows the generation of interactive trajectories for the ego agent, where the level of assertiveness or caution that the ego exhibits is directly related to the easy-to-model uncertainty it maintains about the scene.

While there exists many methods for manipulating rigid objects with parallel-jaw grippers, grasping with multi-finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we present Multi-FinGAN, a fast generative multi-finger grasp sampling method that synthesizes high quality grasps directly from RGB-D images in about a second. We achieve this by training in an end-to-end fashion a coarse-to-fine model composed of a classification network that distinguishes grasp types according to a specific taxonomy and a refinement network that produces refined grasp poses and joint angles. We experimentally validate and benchmark our method against a standard grasp-sampling method on 790 grasps in simulation and 20 grasps on a real Franka Emika Panda. All experimental results using our method show consistent improvements both in terms of grasp quality metrics and grasp success rate. Remarkably, our approach is up to 20-30 times faster than the baseline, a significant improvement that opens the door to feedback-based grasp re-planning and task informative grasping. Code is available at https://irobotics.aalto.fi/multi-fingan/.

An important aspect of dynamic urban coverage is how building collision avoidance is incorporated into the overall coverage mission. We consider a multi-agent urban dynamic coverage problem in which a team of flying agents uses downward facing cameras to observe the street-level environment outside of buildings. Cameras are assumed to be ineffective above a maximum altitude (lower than building height), such that agents must move around or over buildings to complete their mission. The main objective of this paper is to compare three different building avoidance strategies that are compatible with dynamic ergodic methods. To provide context for these results, we also compare our results to three other common coverage methods including: boustrophedon coverage (lawn-mower sweep), Voronoi region based coverage, and a naive grid method. All algorithms are evaluated in simulation with respect to four performance metrics (percent coverage, revisit count, revisit time, and the integral of area viewed over time), across team sizes ranging from 1 to 25 agents, and in five types of urban environments of varying density and height. We find that the relative performance of algorithms changes based on the ratio of team size to search area, as well the height and density characteristics of the urban environment.

The active search for objects of interest in an unknown environment has many robotics applications including search and rescue, detecting gas leaks or locating animal poachers. Existing algorithms often prioritize the location accuracy of objects of interest while other practical issues such as the reliability of object detection as a function of distance and lines of sight remain largely ignored. Additionally, in many active search scenarios, communication infrastructure may be unreliable or unestablished, making centralized control of multiple agents impractical. We present an algorithm called Noise-Aware Thompson Sampling (NATS) that addresses these issues for multiple ground-based robots performing active search considering two sources of sensory information from monocular optical imagery and depth maps. By utilizing Thompson Sampling, NATS allows for decentralized coordination among multiple agents. NATS also considers object detection uncertainty from depth as well as environmental occlusions and operates while remaining agnostic of the number of objects of interest. Using simulation results, we show that NATS significantly outperforms existing methods such as information-greedy policies or exhaustive search. We demonstrate the real-world viability of NATS using a pseudo-realistic environment created in the Unreal Engine 4 game development platform with the AirSim plugin.

This paper proposes an approach for (a) recognizing an observed trajectory from a library of pre-learned motions; and (b) predicting the target position of such trajectory. In our approach, motions are represented as Dynamic Movement Primitives (DMPs). We use critical points from the observed trajectory to time-align it with those in the library. To match the observed trajectory with those in the library, we compare the changes in velocity orientation between consecutive critical points. The proposed approach is computationally light and, as such, can be performed at execution time. As for the prediction, we adopt a similar approach: after matching the observed trajectory to one in the library, we use the latter to predict the target point, modulating it to match the observed trajectory. Both recognition and prediction approaches are probabilistic and, as such, provide a measure of certainty/uncertainty in the recognition/prediction process. Such a measure of uncertainty is important in tasks involving human-robot collaboration, as it allows the robot to decide when it is sufficiently certain to act conditioned on the estimated trajectory. We illustrate our approach both in simulation and in a human-robot interaction scenario involving the Baxter robot.

Robotic three-dimensional (3D) ultrasound (US) imaging has been employed to overcome the drawbacks of traditional US examinations, such as high inter-operator variability and lack of repeatability. However, object movement remains a challenge as unexpected motion decreases the quality of the 3D compounding. Furthermore, attempted adjustment of objects, e.g., adjusting limbs to display the entire limb artery tree, is not allowed for conventional robotic US systems. To address this challenge, we propose a vision-based robotic US system that can monitor the object’s motion and automatically update the sweep trajectory to provide 3D compounded images of the target anatomy seamlessly. To achieve these functions, a depth camera is employed to extract the manually planned sweep trajectory after which the normal direction of the object is estimated using the extracted 3D trajectory. Subsequently, to monitor the movement and further compensate for this motion to accurately follow the trajectory, the position of firmly attached passive markers is tracked in real-time. Finally, a stepwise compounding was performed. The experiments on a gel phantom demonstrate that the system can resume a sweep when the object is not stationary during scanning.

This paper formulates a methodology to plan and control flat-terrain motions of an underactuated bipedal robot riding a snakeboard, which is a steerable variant of the skateboard. We use tools from non-holonomic motion planning to study snakeboard gaits and develop feedback control strategies that enable bipedal robots to produce the desired gaits while maintaining balance, regulating the magnitude and direction of the velocity of the snakeboard, achieving sharp turns, and avoiding obstacles.

Multi-segment continuum robots, that offer inherent compliance and distal dexterity, are suitable for deployment in minimally invasive surgical procedures. Cable-driven mechanism is commonly used in continuum surgical robots but could lead to inter-segment motion coupling in a multi-segment robot. In this paper, we present a coupled mechanics model for a two-segment notched continuum robot to analyze the coupled deflection in the proximal segment due to the distal cable force. The model has been developed for two different conditions in which the proximal segment is initially bent (general condition) and initially straight (special condition). It allows us to introduce a decoupled design methodology that systematically determines a stiffness parameter in each of the segments, based on the desired coupled bending angle and other design requirements. Using the method, we fabricated a decoupled notched continuum robot and evaluated the model accuracy compared with experimental data with mean errors of 0.57° and 0.61°, respectively for general and special conditions throughout the 90° distal segment bending angle. It was also shown in a demonstration in a maxillary sinus phantom that the distal segment was capable of independently perform omnidirectional steering without the proximal segment getting in contact with its surrounding nasal wall.

This paper presents our comparative study on how the flight performances of an in-flight morphing quad-rotor are affected by the morph induced inertia variation. A custom-built in-flight morphing quad-rotor was employed in numerical and experimental tests for the study and analysis. In these tests, the quad-rotor is controlled to follow a predefined path and/or to hover in an environment with the constant wind disturbance. Our numerical results indicate that the morphing-size-down quad-rotor exhibits more agile in flight attributed to the compact volume/size, while the big-size one shows more flight stability in a disturbed and windy environment. Compared to regular scaled aerial vehicles whose volume/size changes follow a weight change proportionally, the numerical results reveal that our morphing quad-rotor that changes its volume with identical mass can acquire more merits towards high flight performances. Experimental validations further prove that through adaptatively transform its size in a complex and constrained environment, the in-flight morphing quad-rotor is not only capable of well performing path following tasks when encountering obstacles on the path or nearby the path, but also enhances the flight performance of withstanding external torques by extending its size so as to increase its moment-of-inertia. In summary, our in-flight morphing quad-rotor can acquire higher flight performances by adaptatively morphing when flying in complex environments.

Morphable design and depth-based visual control are two upcoming trends leading to advancements in the field of quadrotor autonomy. Stereo-cameras have struck the perfect balance of weight and accuracy of depth estimation but suffer from the problem of depth range being limited and dictated by the baseline chosen at design time. In this paper, we present a framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on-the-fly. We present a method to calibrate the system at a small number of discrete baselines and interpolate the parameters for the entire baseline range. We present an extensive theoretical analysis of calibration and synchronization errors. We showcase three different applications of such a system for quadrotor navigation: (a) flying through a forest, (b) flying through an unknown shaped/location static/dynamic gap, and (c) accurate 3D pose detection of an independently moving object. We show that our variable baseline system is more accurate and robust in all three scenarios. To our knowledge, this is the first work that applies the concept of morphable design to achieve a variable baseline stereo vision system on a quadrotor.

Salient object detection (SOD) can directly improve the performance of tasks like obstacle detection, semantic segmentation and object recognition. Such tasks are important for robotic and other autonomous navigation systems. State-of-the-art SOD methodologies, provide improved performance by incorporating depth information, usually acquired using additional specialized sensors, e.g., RGB-D cameras. This introduces an overhead to the overall cost and flexibility of such systems. Nevertheless, the recent advances of machine learning, have provided models, capable of generating depth map approximations, given a single RGB image. In this work, we propose a novel monocular SOD (MonoSOD) methodology, based on a two-branch CNN autoencoder architecture capable of predicting depth maps and estimating saliency through a trainable refinement scheme. Its application on benchmark datasets, indicates that its performance is comparable to that of state-of-the-art SOD methods relying on RGB-D data. Therefore, it could be considered as a lower-cost alternative of such methods for future applications.

Robotic exoskeletons are promising devices capable of both administering therapeutic exercises and assessing human movement quality. Although assessing fatigue is crucial to informing effective strategies for rehabilitation, existing metrics for evaluating fatigue during robot-mediated exercise remain underdeveloped. Current techniques focus on monitoring localized muscle fatigue, but do not consider the complex relationship between changes in muscle activity and associated alterations in joint motion during dynamic movement. In this work, we propose a system-based monitoring paradigm for tracking fatigue-induced changes in performance. The method uses a time-series model to approximate the dynamics of a human-exoskeleton system by mapping muscle activity to movement variables. An index of performance is calculated from modeling errors to continuously track changes in this dynamic relationship over time. Results showed that the index effectively captured fatigue-induced degradation in performance over time during an exoskeleton-administered resistive exercise. The index outperformed a traditional indicator of fatigue that is typically used during robotic intervention, suggesting the proposed approach has the potential to improve fatigue monitoring efforts during robot-aided movement training.

Monocular and stereo visions are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots. However, they are usually developed independently and have their respective strengths and limitations. We propose a novel unified learning framework that leverages the strengths of both monocular and stereo cues for 3D human localization. Our method jointly (i) associates humans in left- right images, (ii) deals with occluded and distant cases in stereo settings by relying on the robustness of monocular cues, and (iii) tackles the intrinsic ambiguity of monocular perspective projection by exploiting prior knowledge of the human height distribution. We specifically evaluate outliers as well as challenging instances, such as occluded and far-away pedestrians, by analyzing the entire error distribution and by estimating calibrated confidence intervals. Finally, we critically review the official KITTI 3D metrics and propose a practical 3D localization metric tailored for humans.

As robots begin to collaborate with people in real life, safety needs to be rigorously ensured to reliably employ robots nearby. In addition to collision prevention algorithms, studies are being actively conducted on collision handling methods. Momentum Observer (MOB) was developed to estimate disturbance torque without using joint acceleration. However, the estimated disturbance from MOB contains not only the applied external torque but also model uncertainty such as friction and modeling error due to imprecise system identification. Our proposed method handles this problem by learning the model uncertainty with Long Short-Term Memory (LSTM) and thereby estimates the purely applied external torque with only proprioceptive sensors. The proposed method can be applied even when the information on the robot model is not available. The experiments using a real robot show that the external torque can be estimated and collisions can be detected accordingly even in a limited situation where a precise dynamics model and friction model are not available.

In this paper we present a methodology for optimising the design of a metamaterial structure with one degree of freedom that is able to simultaneously bend and stretch. The structure is intended for assisting flexion-extension of the wrist joint. The metamaterial is comprised of serially connected, individually designed cells. The design parameters can be chosen to optimally fit a desired planar curve such as the curvature of the skin on a plane normal to the flexion/extension axis of the wrist joint. A tool for the optimised design is described and the experimental validation of the output is conducted, to show the ability of the mechanism to conform to a 2D curve, also exhibiting a change in length, which is desirable for reducing sliding and shear on the skin. The design tool allows the generation of metamaterials optimised for a multitude of other applications where actuated mechanisms must be worn by a user for rehabilitation or assistance purposes.

Human running motion is a complex motion that not only based on the movement of legs but also needs the entire body to participate in. Therefore, we focused on the upper body, designed a trunk and pelvic rotation assist suit to apply an external force on the chest and pelvis for assisting the running motion, changing the energy flow between the upper and lower body to improve the running efficiency. However, human motion is very complicated and difficult to analyze, it is hard to explore the specific effect of the external force which is provided by the rotation assist suit on human motion. In this paper, we proposed an advanced spring-loaded inverted pendulum model for simulating the human running motion in natural running status and expanded running status in which the motion is expanded by wearing the trunk and pelvic rotation assist suit, parametrically represent the changes of human due to the intervention of the external force, and discuss the effect of the external assistant force on the human running motion.

Envisioning the employment of rope-assisted humanoid robots to reduce human intervention for operations in the heights, this preliminary work addresses the modeling and motion planning problems for a rope-assisted bipedal robot. The mathematical features of this system outnumber the ones of typical humanoid robots, including: under-actuation of the floating-base joints, the rope pulling effect and the passive connection between the robot body and the rope master-point. These characteristics render the study of a rope-assisted bipedal robot both fascinating and unexplored, raising motion planning challenges when attempting to plan dynamic suspended maneuvers, as rappelling. To this end, we first introduce a template three-mass model of a bipedal robot connected through passive joints to an extensible rope, which is in turn modeled as a two-mass body. Based on this, a family of optimal control problems is presented to plan rappelling maneuvers.

Small-scale robots have great potential in minimally invasive surgery (MIS). In this paper, we propose an untethered magnetic gripper with small scale and build a double-magnet model for it. The gripper is 4.3mm long and its maximum width is 4mm. It contains a spindle and two magnets, which can achieve precise control of orientation, position and open angle with external magnetic driven field. As a result, it can perform operations such as transporting medicines in confined and constrained environments. Modeling and analysis of the magnetic gripper have been carried out. Relationship between the open angle and external magnetic field has been established. Kinematics model of the gripper has been built. A 3-axis Helmholtz-Maxwell coil system has been established to generate the magnetic field, in which orientation and open angle can be controlled with uniform magnetic field while position can be controlled with gradient field. The proposed gripper have been validated with phantom experiments. An opened angle control error of 0.63° and direction control error of 1.1° have been obtained.