ICRA 2021 Papers

This paper considers progressively more demanding off-line shortest path sensory coverage problems in an optimization framework. In the first problem, a robot finds the shortest path to cover a set of target nodes with its sensors. Because this mixed integer nonlinear optimization problem (MINLP) is NP-hard, we develop a polynomial-time approximation algorithm with a bounded approximation ratio. The next problem shortens the coverage path when possible by viewing multiple targets from a single pose. Its polynomial-time approximation simplifies the coverage path geometry. Finally, we show how the complete sensory coverage problem can be formulated as a MINLP over a decomposition of a given region into arbitrary convex polygons. Extensions of the previously introduced algorithms provides a polynomial time solution with bounded approximation. Examples illustrate the methods.

Snails’ unique locomotion abilities help them realise stable movement by muscular exploiting travelling waves and friction modulation. Inspired by these characteristics, snaillike robots have recently become the focus of growing research. In this paper, we present a novel friction-driven three-foot snaillike robot which employs a simple mechanism to partially mimic and replicate snail-like motion, but in a novel form. This robot is driven by two servo motors, which makes it easy and low-cost to fabricate. The robot operates by breaking frictional symmetry in the cyclic motion of the three feet, in much the same way as the three-sphere Golestanian swimmer. The symmetry of its structure and properties of friction give the robot distinctive movements. We present a mathematical model of the robot’s locomotion, focusing on its kinetic harmonic-peristaltic movement. We designed and fabricated the robot, then undertook simulations and experiments, which closely match the analytic solutions. This robot provides a new approach to realising simpler and lower cost biomimetic mobile robots.

In tendon-driven robotic hands, tendons are usually routed along several pulleys. The resulting friction is often substantial, and must therefore be modelled and estimated, for instance for accurate control and contact detection. Common approaches for friction estimation consider special dedicated setups, where the parameters of a static or dynamic friction model at a single contact point are determined. In this paper, we rather combine such individual friction models into an overall friction model for the entire finger. Furthermore, we propose a method for estimating the parameters of this overall model in situ, i.e. from trajectories executed on the assembled hand, avoiding the need for dedicated setups. An important component of the proposed model is a varying bias for treating friction at low velocities, allowing a simpler static friction model to be used. We demonstrate that our approach enables contacts to be detected more accurately on the DLR David hand, without additional sensors.

Several independent approaches exist for state estimation and control of multirotor unmanned aerial systems (UASs) that address specific and constrained operational conditions. This work presents a complete end-to-end pipeline that enables precise, aggressive and agile maneuvers for multirotor UASs under real and challenging outdoor environments. We leverage state-of-the-art optimal methods from the literature for trajectory planning and control, such that designing and executing dynamic paths is fast, robust and easy to customize for a particular application. The complete pipeline, built entirely using commercially available components, is made open-source and fully documented to facilitate adoption. We demonstrate its performance in a variety of operational settings, such as hovering at a spot under dynamic wind speeds of up to 5– 6m/s (12–15mi/h) while staying within 12cm of 3D error. We also characterize its capabilities in flying high-speed trajectories outdoors, and enabling fast aerial docking with a moving target with planning and interception occurring in under 8s.

In this paper, we propose a novel representation and verification technique for software components in a robotic system using a linear logic theorem prover. Linear logic includes consumable resources together with persistent resources, enabling representing and reasoning of robotic domains. We demonstrate model representation and verification of formal specifications through Robot Operating System (ROS) components. The system model can be either statically extracted by HAROS (a ROS based static analysis framework) or dynamically extracted once all system components are running. After ten years of its first release, ROS has become one of the most popular middlewares among robotic programming frameworks. Even though ROS is very popular among robotic developers, we believe that a framework for easily representing and verifying robotic systems is missing. This paper introduces a new technique for formally representing and verifying robotic systems using a linear logic theorem prover and finally presents a number of illustrations of model representation and safety property checking both statically and dynamically for the robot Kobuki.

Advancements in the domain of physical human-robot interaction (pHRI) have tremendously improved the ability of humans and robots to communicate, collaborate, and coexist. In particular, compliant robotic systems offer many characteristics that can be leveraged towards enabling physical interactions that more efficiently and intuitively communicate intent, making compliant systems potentially useful in more physically demanding subsets of human-robot collaborative scenarios. Tensegrity robots are an example of compliant systems that are well-suited to physical interactions while still retaining useful rigid properties that make them practical for a variety of applications. In this paper, we present the design and preliminary testing of a 6-bar spherical tensegrity with force-sensing capabilities. Using this prototype, we demonstrate the ability of its force-sensor array to detect a variety of physical interaction types that might arise in a human context. We then train and test a series of classifiers using data from unique and representative interactions in order to demonstrate the feasibility of using this physical modality of sensing to reliably communicate goals and intents from a human operator in a human-robot collaborative setting.

Foot devices have been ubiquitously used in surgery to control surgical equipment. Most common applications are foot switches for electro-surgery, endoscope positioning and tele-robotic consoles. Switches fall short of providing continuous control as required for precise use of instruments. We developed a haptic foot interface to provide continuous assistance in surgical procedures. This paper concerns the foot control of simultaneous five degrees of freedom (DoF) of a surgical laparoscopic gripper. We assess systematically precision at controlling position and orientation at the target and closing of the forceps. Our controller provides position:position mapping between the foot and the robotic tool, as well as haptic feedback, compensating for gravity of the lower limb of the operator so as to alleviate fatigue. A dynamic model compensation and closed loop force feedback is used to achieve high transparency and backdrivability. The assistance is based on a novel type of haptic fixtures combining spring-damper with selective dynamic compensation in the direction aligned with the task of grasping, so as to simplify control of certain poses, made difficult due to the coupling between human lower limbs’ DoF’s. We experimentally evaluated the control strategy with six users on a position control surgical task in simulation. Results show the proposed assistance greatly eases the foot grasping task leading to higher completeness, efficiency, and lower mental and physical load.

In this work, a neural network is trained to simultaneously perform segmentation and pixel-wise Out-of-Distribution (OoD) detection, such that the segmentation of unknown regions of scenes can be rejected. This is made possible by leveraging an OoD dataset with a novel contrastive objective and data augmentation scheme. By including unknown classes in the training data, a more robust feature representation is learned with known classes represented distinctly from those unknown. In comparison, when presented with unknown classes or conditions, many current approaches for segmentation frequently exhibit high confidence in their inaccurate segmentations and cannot be trusted in many operational environments. We validate our system on a real-world dataset of unusual driving scenes, and show that by selectively segmenting scenes based on what is predicted as OoD, we can increase the segmentation accuracy by an IoU of 0.2 with respect to alternative techniques.

Learning driving policies using an end-to-end network has been proved a promising solution for autonomous driving. Due to the lack of a benchmark driver behavior dataset that contains both the visual and the LiDAR data, existing works solely focus on learning driving from visual sensors. Besides, most works are limited to predict steering angle yet neglect the more challenging vehicle speed control problem. In this paper, we propose a novel end-to-end network, FlowDriveNet, which takes advantages of sequential visual data and LiDAR data jointly to predict steering angle and vehicle speed. The main challenges of this problem are how to efficiently extract driving-related information from images and point clouds, and how to fuse them effectively. To tackle these challenges, we propose a concept of point flow and declare that image optical flow and LiDAR point flow are significant motion cues for driving policy learning. Specifically, we first create an enhanced dataset that consists of images, point clouds and corresponding human driver behaviors. Then, in FlowDriveNet, a deep but efficient visual feature extraction module and a point feature extraction module are utilized to extract spatial features from optical flow and point flow, respectively. Additionally, a novel temporal fusion and prediction module is designed to fuse temporal information from the extracted spatial feature sequences and predict vehicle driving commands. Finally, a series of ablation experiments verify the importance of optical flow and point flow and comparison experiments show that our flow-based method outperforms the existing image-based approaches on the task of driving policy learning.

In this paper, we show how the Federated Learning (FL) framework enables learning collectively from distributed data in connected robot teams. This framework typically works with clients collecting data locally, updating neural network weights of their model, and sending updates to a server for aggregation into a global model. We explore the design space of FL by comparing two variants of this concept. The first variant follows the traditional FL approach in which a server aggregates the local models. In the second variant, that we call Flow-FL, the aggregation process is serverless thanks to the use of a gossip-based shared data structure. In both variants, we use a data-driven mechanism to synchronize the learning process in which robots contribute model updates when they collect sufficient data. We validate our approach with an agent trajectory forecasting problem in a multi-agent setting. Using a centralized implementation as a baseline, we study the effects of staggered online data collection, and variations in data flow, number of participating robots, and time delays introduced by the decentralization of the framework in a multi-robot setting.

This paper presents a novel approach that allows a swarm of heterogeneous robots to produce simultaneously segregative and flocking behaviors using only local sensing. These behaviors have been widely studied in swarm robotics and their combination allows the execution of several complex tasks. Our approach consists of modeling the swarm as a Gibbs Random Field (GRF) and using appropriate potential functions to reach segregation, cohesion and consensus on the velocity of the swarm. Simulations and proof-of-concept experiments using real robots are presented to evaluate the performance of our methodology in comparison to some of the state-of-the-art works that tackle segregative behaviors.

Dynamic Movement Primitives (DMPs) are a well-known tool for encoding robotic motions. Their popularity stems from invariance properties in time and space, the ability to describe complex coordinated motions in multiple degrees of freedom with a relatively small number of parameters, and the linearity in the parameters that describe the motion. The latter allows easily fitting a DMP to motions e.g. demonstrated by a human. DMPs are at their core second order autonomous differential equations. However, feedforward controls of robots with flexible joints are known to require reference trajectories up to the fourth derivative of position. Consequently, classical DMPs are mechanically not compatible with flexible joint robots. In this paper, we propose an extension of DMPs by introducing FlexDMPs. This concept retains the structural properties and benefits of classical DMPs but generates trajectories up to the fourth derivative that can theoretically be tracked ideally (i.e. with zero tracking error) by flexible joint robots. The concept is demonstrated on a high fidelity simulation model of an industrial robot and in experimental results on a collaborative manipulator.

Precise aerial manipulation is important for multirotor robots. For multirotors equipped with arms, the root pose error due to the floating body affects the precision at the end effector. Fixed-root approaches, such as perching on surfaces using the rotor suction force, are useful to address this problem. Furthermore, it is difficult for arm-equipped multirotors to generate large wrenches at the end effector owing to joint torque limitations. For multilink aerial robots with rotors distributed to each link, the thrust of rotors can produce large torques. Therefore, such multirotor robots can generate comparatively large wrenches at the end effector. In this paper, we introduce a rotor-distributed multilink robot that can perch on surfaces. First, we designed a root footplate and arm module for a multilink aerial robot. During perching, the joint between these two links can be passive to prevent peeling. Second, we propose a quadratic programming (QP) based controller to calculate the desired thrust for perching motion, considering the static friction and zero moment point (ZMP) conditions on the footplate. Finally, we conducted root-body perching motion tests. The manipulations of the multilink aerial robot during perching become more accurate than those during flight because the root position adheres to the environment.

Robots have begun operating and collaborating with humans in industrial and social settings. This collaboration introduces challenges: the robot must plan while taking the human’s actions into account. In prior work, the problem was posed as a 2-player deterministic game, with a limited number of human moves. The limit on human moves is unintuitive, and in many settings determinism is undesirable. In this paper, we present a novel planning method for collaborative human-robot manipulation tasks via probabilistic synthesis. We introduce a probabilistic manipulation domain that captures the interaction by allowing for both robot and human actions with states that represent the configurations of the objects in the workspace. The task is specified using Linear Temporal Logic over finite traces (LTLf). We then transform our manipulation domain into a Markov Decision Process (MDP) and synthesize an optimal policy to satisfy the specification on this MDP. We present two novel contributions: a formalization of probabilistic manipulation domains allowing us to apply existing techniques and a comparison of different encodings of these domains. Our framework is validated on a physical UR5 robot.

To improve robotic grasping, we are interested in developing a new non-contact fingertip-mounted sensor for near-distance ranging and material sensing. Here we report new progress in combining direct pulse-echo ultrasound and optoacoustic effects in sensor design to deal with optically and/or acoustically challenging targets (OACTs). Our dual-modal and dual sensing mechanisms (DMDSM) sensor design is enabled by a novel wideband ultrasound transmitter embedded inside a piezoelectric (lead zirconate titanate - PZT) ring transducer. The new DMDSM sensor is capable of differentiating a variety of OACTs. To verify our design, both distance ranging tests and material sensing tests have been conducted. The ranging tests show the sensor can perform both optoacoustic ranging (for light-absorbing materials) and pulse-echo ultrasound ranging (for reflective or transparent materials). For material sensing, the dual-modal spectra from OACTs are collected to compare the new sensor with previous designs. The overall 100% accuracy from the confusion matrices indicates the initial success of our sensor design in differentiating conventional targets as well as the OACTs with the new DMDSM sensor.

This paper enhances the feedback linearization controller for multirotors with a learned acceleration error model and a thrust input delay mitigation model. Feedback linearization controllers are theoretically appealing but their performance suffers on real systems, where the true system does not match the known system model. We take a step in reducing these robustness issues by learning an acceleration error model, applying this model in the position controller, and further propagating it forward to the attitude controller. We show how this approach improves performance over the standard feedback linearization controller in the presence of unmodeled dynamics and repeatable external disturbances in both simulation and hardware experiments. We also show that our thrust control input delay model improves the step response on hardware systems.

In environmental perception of autonomous driving, zero-shot semantic segmentation that can make prediction of new categories without using any labeled training samples is considered as a challenging task. One key step in this task is to transfer knowledge across categories via auxiliary semantic word embeddings. In this paper, we propose a feature enhanced projection network (FEPNet) that takes full advantage of transferred knowledge to enrich semantic representations. In FEPNet, two projection layers are added to a segmentation network so as to map features into seen (S) and unseen (U) category spaces, respectively. During training, U-space features are transferred to S-space using similarity relations to enhance the representation of seen categories. In the inference stage, the representation of unseen categories is also strengthened by incorporating features transferred from S-space. Moreover, a novel strategy is proposed to effectively alleviate prediction bias by performing segmentation independently in separate areas that contain seen and unseen categories. We conduct extensive experiments on three benchmark datasets. The experimental results show that our FEPNet achieves new state-of-the-art results compared to existing approaches.

Predicting trajectories of participating vehicles is a crucial task towards full and safe autonomous driving. General unconstrained machine learning methods often report unrealistic predictions, and need to be combined with different motion constraints. Existing work either defines some shallow maneuvers and modes to regulate the output, or uses vehicle dynamics as the main source of constraints, for instance via kinematic models. In contrast, we present a new approach that guides the learning models by complex semantic maneuvers, constructing from both vehicle states and the surrounding objects. We propose a novel Maneuver Fusion layer to incorporate the logic-based semantic maneuvers into deep neural networks. We also incorporate and refine the different loss functions to account for the feasibility of the trajectories, adapting to different maneuver types. Finally, we design a hierarchical multi-task learning framework with adaptive loss to provide a multimodal trajectory prediction. Our method was evaluated on a large-scale real world data set for urban driving and was shown to give promising improvement over the states of the art.

This paper proposes a novel adaptive anchoring module that can be integrated into robots to enhance their mobility and manipulation abilities. The module can deploy a suitable mode of attachment, via spines or vacuum suction, to different contact surfaces in response to the textural properties of the surfaces. In order to make a decision on the suitable mode of attachment, an original dataset of 100 images of outdoor and indoor surfaces was enhanced using a WGAN-GP to generate an additional 200 synthetic images. The enhanced dataset was then used to train a visual surface examination model using Faster RCNN. The addition of synthetic images increased the mean average precision of the Faster R-CNN model from 81.6% to 93.9%. We have also conducted a series of load tests to characterize the module’s strength of attachments. The results of the experiments indicate that the anchoring module can withstand an applied detachment force of around 22N and 20N when attached using spines and vacuum suction on the ideal surfaces, respectively.