ICRA 2019 Papers

This paper aims to develop a controller for a robot manipulator equipped with series elastic actuators (SEAs) to precisely track the desired link position coping with deflections from the intrinsic compliance. The well-known time-delay estimation (TDE) technique is modified for devising the new controller. In this paper, we first report that the conventional use of a constant gain matrix for the TDE framework is insufficient for high accuracy tracking because the tracking accuracy is significantly deteriorated and the closed-loop stability may be threatened. Accordingly, the new controller employs link inertia information with dynamic coupling terms to define the gain for TDE and then employs terminal sliding mode (TSM) control to enhance robustness and convergence speed. Particularly, the modified TDE is applied in the two-staged manner which enables to compensate the complicated nonlinear dynamics terms in SEA dynamics. The TSM synergistically amalgamates the accuracy, robustness, and convergence in tracking. The proposed controller is numerically validated by comparative experiments with a SEA-driven manipulator.

Current methods for estimating force from tactile sensor signals are either inaccurate analytic models or task-specific learned models. In this paper, we explore learning a robust model that maps tactile sensor signals to force. We specifically explore learning a mapping for the SynTouch BioTac sensor via neural networks. We propose a voxelized input feature layer for spatial signals and leverage information about the sensor surface to regularize the loss function. To learn a robust tactile force model that transfers across tasks, we generate ground truth data from three different sources: (1) the BioTac rigidly mounted to a force torque (FT) sensor, (2) a robot interacting with a ball rigidly attached to the same FT sensor, and (3) through force inference on a planar pushing task by formalizing the mechanics as a system of particles and optimizing over the object motion. A total of 140k samples were collected from the three sources. We achieve a median angular accuracy of 3.5 degrees in predicting force direction (66% improvement over the current state of the art) and a median magnitude accuracy of 0.06 N (93% improvement) on a test dataset. Additionally, we evaluate the learned force model in a force feedback grasp controller performing object lifting and gentle placement. Our results can be found on https: //sites.google.com/view/tactile-force.

Point set registration (PSR) is an important problem in computer vision, robotics and biomedical engineering communities. Usually, only positional information at each point is adopted in a registration. In this paper, the orientational vector (or normal vector) associated with each point is also utilized. Generalized point set registration is formulated and solved under the Expectation-Maximization (EM) framework. In the E-step, the posterior probabilities representing the correspondence probabilities are computed. In the Mstep, rigid transformation parameters including the rotation matrix, the translation vector are updated. The proposed algorithm stops when it converges to the optimal solution or a maximum number of iterations is achieved. The observed position set and normal vector set are assumed to follow Gaussian Mixture Models (GMMs) and Fisher distribution Mixture Models (FMMs), respectively. To further improve our algorithm's robustness, the hybrid mixture models (HMMs) are assumed to be inhomogeneous. Experimental results on the surface points extracted from a human femur' CT model show that our algorithm can achieve lower registration error, is more robust to noise and outliers than the state-of-the-art registration methods.

In hybrid force-velocity control, the robot can use velocity control in some directions to follow a trajectory, while performing force control in other directions to maintain contacts with the environment regardless of positional errors. We call this way of executing a trajectory hybrid servoing. We propose an algorithm to compute hybrid force-velocity control actions for hybrid servoing. We quantity the robustness of a control action and make trade-offs between different requirements by formulating the control synthesis as optimization problems. Our method can efficiently compute the dimensions, directions and magnitudes of force and velocity controls. We demonstrated by experiments the effectiveness of our method in several contact-rich manipulation tasks. Link to the video: https://youtu.be/KtSNmvwOenM.

Robot swarms herald the ability to solve complex tasks using a large collection of simple devices. However, engineering a robotic swarm is far from trivial, with a major hurdle being the definition of the control laws leading to the desired globally coordinated behavior. Communication is a key element for coordination and it is considered one of the current most important challenges for swarm robotics. In this paper, we study the problem of maintaining robust swarm connectivity while performing a coverage task based on the Voronoi tessellation of an area of interest. We implement our methodology in a team of eight Khepera IV robots. With the assumptions that robots have a limited sensing and communication range-and cannot rely on centralized processing-we propose a tri-objective control law that outperforms other simpler strategies (e.g. a potential-based coverage) in terms of network connectivity, robustness to failure, and area coverage.

Object classification is an important capability for robots as it provides vital semantic information that underpin most practical high-level tasks. Classic handcrafted features, such as point pair features, have demonstrated their robustness for this task. Combining these features with modern deep learning methods provide discriminative features that are rotation invariant and robust to various sources of noise. In this work, we aim to improve the descriptiveness of point pair features while retaining their robustness. We propose a method to achieve more structured sampling of pairs and combine this information through the use of graph convolutional networks. We introduce a novel attention model based on a repeatable local reference frame. Experiments show that our approach significantly improves the state of the art for object classification on large scale reconstruction such as the Stanford 3D indoor dataset and ScanNet and obtains competitive accuracy on the artificial dataset ModelNet.

We present a distributed control strategy for a team of agents to autonomously achieve a desired 3D formation. Our approach is based on local relative position measurements and can be applied to multirotor aerial vehicles. We assume that agents have a common sense of direction, which is used to align the z-axes of their local coordinate frames. However, this assumption is not crucial, and our approach is provably robust to misalignments in the local coordinate frames or measurement inaccuracies. In particular, agents can move along any direction that projects positively onto the desired direction of motion. This property is exploited to design a fully-distributed collision avoidance strategy. We validate the proposed approach experimentally and show that a team of quadrotors can achieve a desired 3D formation without collisions.

Co-speech gestures enhance interaction experiences between humans as well as between humans and robots. Most existing robots use rule-based speech-gesture association, but this requires human labor and prior knowledge of experts to be implemented. We present a learning-based co-speech gesture generation that is learned from 52 h of TED talks. The proposed end-to-end neural network model consists of an encoder for speech text understanding and a decoder to generate a sequence of gestures. The model successfully produces various gestures including iconic, metaphoric, deictic, and beat gestures. In a subjective evaluation, participants reported that the gestures were human-like and matched the speech content. We also demonstrate a co-speech gesture with a NAO robot working in real time.

This paper introduces the KUKA Robot Learning Lab at KIT - a remotely accessible robotics testbed. The motivation behind the laboratory is to make state-of-the-art industrial lightweight robots more accessible for education and research. Such expensive hardware is usually not available to students or less privileged researchers to conduct experiments. This paper describes the design and operation of the Robot Learning Lab and discusses the challenges that one faces when making experimental robot cells remotely accessible. Especially safety and security must be ensured, while giving users as much freedom as possible when developing programs to control the robots. A fully automated and efficient processing pipeline for experiments makes the lab suitable for a large amount of users and allows a high usage rate of the robots.

The use of flexible endoscope has been rising inconveniences. Steering of the distal section is not intuitive and the weight of the endoscope burdens a physical pressure on physicians who use it continuously for a long time. Also, the limited dexterity of an instrument makes therapeutic procedures more difficult, and further the unintended communications often occur during cooperation with assistants. These degrade the efficiency and thus increase the procedure time. In this paper, we propose a robotic endoscopy system (easyEndo) that can be mounted on a conventional endoscope and facilitate solo-endoscopy with two intuitive hand-held controllers. Furthermore, a robotic arm is presented that can be attached to the endoscope to assist with tissue traction. To validate the robotic endoscopy system, experiments to simulate biopsy and lesion marking were conducted with novices. The results showed that the robotic manipulations improved efficiency and reduced workload than manual manipulation. Subsequently, a prototype of the robotic arm was attached at the distal end of the endoscope, and the feasibility of tissue traction was confirmed by a simulation of pulling a rubber band.

Robotic bronchoscopy has the potential to improve the early detection of lung cancer. For the technology to be broadly adopted, the physician needs to be able to control the robotic bronchoscope in an instinctive and effective manner. In this paper, we describe the algorithms used to manipulate/drive the Auris Monarch Platform, a 10 degree-of-freedom bronchoscope and sheath, using a 3 degree-of-freedom user input. We introduce the concept of paired driving where the devices co-insert and co-articulate depending on their relative insertion. The paper presents safety algorithms such as auto-relax on retract and tension monitoring. The drive modes were developed, optimized, and clinically tested in lung models, human cadavers and live porcine models prior to their commercial release. Clinical studies show that the physician is able to reach significantly deeper in the lung than with classic bronchoscopes.

Robotic manipulation of deformable objects (vs. rigid objects) has been a classic topic in robotics. Compared to deformable synthetic objects such as rubber balls and clothes, biological cells are highly deformable and more prone to damage. This paper presents robotic manipulation of deformable cells for orientation control (both out-of-plane and in-plane), which is required in both clinical (e.g., in vitro fertilization) and biomedical (e.g., clone) applications. Compared to manual cell rotation control based on empirical experience, the robotic approach, based on mathematical modeling and path planning, effectively rotates a cell while consistently maintaining minimal cell deformation to avoid cell damage. A force model is established to determine the minimal force applied by the micropipette to rotate a spherical or more generally, an ellipsoidal mouse oocyte. The force information is translated into indentation through a contact mechanics model, and the manipulation path of the micropipette is formed by connecting the indentation positions on the oocyte. A compensation controller is designed to compensate for the variations of mechanical properties across cells. The polar body of an oocyte is detected by deep neural networks with robustness to shape and size differences. Experimental results demonstrate that the system achieved an accuracy of 97.6% in polar body detection and an accuracy of 0.7° in oocyte orientation control with maximum oocyte deformation of 2.69 μm.

The control of human hands and fingers represents one of the most complex functions of the motor cortex. In order to implement anthropomorphic hands that mimic accurately the motion ability of the human hands, the basic biological mechanisms of the natural muscle control should be modelled. This paper presents the design of a significantly improved control system based on analogue neural networks that can be used to replicate the biological control mechanisms of the natural muscles. In order to demonstrate the proposed concept, experiments were performed using a single-joint robotic arm that can be flexed as the human elbow by an artificial muscle connected as the biceps. In order to bring more biological plausibility to the robotic arm, the artificial muscle is implemented using a shape memory alloy wire which actuates by contraction as the natural muscles. Moreover, the contraction force of the actuator wire is directly determined by the spiking frequency of the electronic neurons as the motor neurons determines the contraction strength of the natural muscles. The experimental results show that using excitatory neurons and several inhibitory neurons unevenly distributed on the inputs of the artificial motor neurons that drive the shape memory alloy actuator, the spiking neural network is able to control with high precision the rotation of the arm mobile lever to random target positions even if the arm is slightly loaded. These results validate the control method of the robotic arm junction showing that the analogue spiking neural networks represents a very good alternative to control the contraction of SMA actuators in a biological plausible manner.

In this study, a robotic forceps with a wrist joint using visual SLAM for joint angle sensing was developed. The forceps has a flexible joint connected to the wrist joint at its rear end and the motion of the rear joint is driven by a parallel linkage. A monocular camera attached on the rear of the parallel linkage is in charge of position sensing, and the joint angles are estimated from the pose of the camera. The pose of the camera is obtained by a visual SLAM. The visual servo system realizes a simple attaching mechanism. The static and dynamic positioning experiments are conducted. We confirmed that the visual servoing system controls the forceps tip within the error of 3 deg in the motion range of 50 deg.

Trash deposits in aquatic environments have a destructive effect on marine ecosystems and pose a long-term economic and environmental threat. Autonomous underwater vehicles (AUVs) could very well contribute to the solution of this problem by finding and eventually removing trash. This paper evaluates a number of deep-learning algorithms performing the task of visually detecting trash in realistic underwater environments, with the eventual goal of exploration, mapping, and extraction of such debris by using AUVs. A large and publicly-available dataset of actual debris in open-water locations is annotated for training a number of convolutional neural network architectures for object detection. The trained networks are then evaluated on a set of images from other portions of that dataset, providing insight into approaches for developing the detection capabilities of an AUV for underwater trash removal. In addition, the evaluation is performed on three different platforms of varying processing power, which serves to assess these algorithms' fitness for real-time applications.

Effectiveness of cutting is measured by the ability to achieve material fracture with smooth knife movements. The work performed by a knife overcomes the material toughness, acts against the blade-material friction, and generates shape deformation. This paper studies how to control a 2-DOF robotic arm equipped with a force/torque sensor to cut through an object in a sequence of three moves: press, push, and slice. For each move, a separate control strategy in the Cartesian space is designed to incorporate contact and/or force constraints while following some prescribed trajectory. Experiments conducted over several types of natural foods have demonstrated smooth motions like would be commanded by a human hand.

Fetoscopy is a technically challenging surgery, due to the dynamic environment and low diameter endoscopes often resulting in a limited field of view. In this paper, we report on the design and operation of a robotic multimodal endoscope with optical ultrasound and white light stereo camera. The manufacture and control of the endoscope is presented, along with large area (80 mm ×80 mm) surface visualisations of a placenta phantom using the optical ultrasound sensor. The repeatability of the surface visualisations was found to be 0. 446 ± 0.139 mm and 0. 267 ± 0.017 mm for a raster and spiral scan, respectively.

We present a robot eye-hand coordination learning method that can directly learn visual task specification by watching human demonstrations. Task specification is represented as a task function, which is learned using inverse reinforcement learning(IRL [1]) by inferring a reward model from state transitions. The learned reward model is then used as continuous feedbacks in an uncalibrated visual servoing(UVS [2]) controller designed for the execution phase. Our proposed method can directly learn from raw videos, which removes the need for hand-engineered task specification. Benefiting from the use of a traditional UVS controller, the training on real robot only happens at initial Jacobian estimation which takes an average of 4-7 seconds for a new task. Besides, the learned policy is independent from a particular robot, thus has the potential of fast adapting to other robot platforms. Various experiments were designed to show that, for a task with certain DOFs, our method can adapt to task/environment changes in target positions, backgrounds, illuminations, and occlusions.

The ability to reference objects in the environment is a key communication skill that robots need for complex, task-oriented human-robot collaborations. In this paper we explore the use of projections, which are a powerful communication channel for robot-to-human information transfer as they allow for situated, instantaneous, and parallelized visual referencing. We focus on the question of what makes a good projection for referencing a target object. To that end, we mathematically formulatelegibility of projections intended to reference an object, and propose alternative arrow-object match functions for optimally computing the placement of an arrow to indicate a target object in a cluttered scene. We implement our approach on a PR2 robot with a head-mounted projector. Through an online (48 participants) and an in-person (12 participants) user study we validate the effectiveness of our approach, identify the types of scenes where projections may fail, and characterize the differences between alternative match functions.