ICRA 2019 Papers

Automatic repair of mechanical structures would enable a robot to recover or improve functions after physical damage. Little work exists on real-world execution of automatic repair in robotic systems. State-of-the-art takes a modular approach where the robotic system is modular and a replacement module is available. However, the modular approach suffers from low granularity in repair even with tens of motors. In addition, there is a lack of quantitative evaluation of the effect of automatic repair on robot functionality. Here we propose a cooperative method for automatic repair in a robotic system. Our method is regeneration-based rather than module-based and does not assume availability of a replacement part. It integrates a fabrication process on the fly for robot structure regeneration. With a system that consists of a regenerating robot, a legged robot and a pre-engineered ribbon, we demonstrate end-to-end execution of automated repair of the legged robot's leg by the regenerating robot in 335 seconds. Experiments on repeatability show a 100% success rate for sub-processes such as positioning, leg fabrication, and legged robot disengagement and a 90% success rate for leg detachment. We quantify the effect of leg regeneration on mobility recovery and found a 90% recovery of forward speed, a 19.7% increase of peak power and a 9.3% reduction of cost of transport with a regenerated leg.

Following improvements in deep neural networks, state-of-the-art networks have been proposed for human recognition using point clouds captured by LiDAR. However, the performance of these networks strongly depends on the training data. An issue with collecting training data is labeling. Labeling by humans is necessary to obtain the ground truth label; however, labeling requires huge costs. Therefore, we propose an automatic labeled data generation pipeline, for which we can change any parameters or data generation environments. Our approach uses a human model named Dhaiba and a background of Miraikan and consequently generated realistic artificial data. We present 500k + data generated by the proposed pipeline. This paper also describes the specification of the pipeline and data details with evaluations of various approaches.

This paper presents a prototype of an automated seedling height assessment system for tree nurseries. The proposed system can acquire and store real-time 3D point-cloud data of seedlings; and perform offline identification, measurement, and report generation of seedling heights with an overall system accuracy that meets a 5mm accuracy specification. Periodic growth information of seedlings allows quantifying effects of different factors on the overall seedling development process for research and production optimization purposes. However, current manual sampling approaches used at these facilities produce quite limited data samples, and the process is rather time-consuming and labor intensive for industrial scale operations. In contrast, the proposed system is capable of significantly increasing the measurement sample size, measurement resolution, and frequency of measurement by automating the seedling measurement process using a scanning laser profilometer and an application specific point-cloud processing algorithm. The performance of the proposed profilometry solution for point-cloud generation is compared with several other point-cloud generation methods such as a 3D structured light sensing, light intensity detection and ranging (LiDAR), stereovision, and photogrammetry. This comparison results demonstrate a superior performance of the laser-profilometer over other sensing solutions available for seedling height measurement. The proposed system is experimentally validated for its measurement accuracy and repeatability. The field-test of the measurement system was conducted at Centre for Agriculture and Forestry Development, Wooddale, Newfoundland and Labrador (NL), Canada, and the results demonstrate the practical applicability and technological readiness of the proposed system for field deployment.

In this paper, we will describe AMEvA (Automated Models of Everyday Activities), a special-purpose knowledge acquisition, interpretation, and processing system for human everyday manipulation activity that can automatically: (1) create and simulate virtual human living and working environments (such as kitchens and apartments) with a scope, extent, level of detail, physics, and close to photorealism that facilitates and promotes the natural and realistic execution of human everyday manipulation activities; (2) record human manipulation activities performed in the respective virtual reality environment as well as their effects on the environment and detect force-dynamic states and events; (3) decompose and segment the recorded activity data into meaningful motions and categorize the motions according to action models used in cognitive science; and (4) represent the interpreted activities symbolically in KNOWROB [1] using a first-order time interval logic representation.

The ability to patterning cells is an important technique to facilitate cell-based assay and characterization. In this paper, an automated cell patterning system was developed for the fabrication of large-scale cell patterns. To resolve the challenge of the limited printable area, the cell-printing microchip and the substrate were mounted on the movable stages of the system, and large-scale cell patterns were realized through coordination between the stages. An autofocusing technique was integrated in the system to evaluate the gap between the microchip and the substrate. In order to enhance the performance of the patterning system, different experimental parameters, including the velocity of the moving stage, were examined. Yeast cells suspending in 6-aminohexanoic acid (AHA) solution were considered in this study, and a sequence of characters was successfully printed using the proposed system. The results confirm that this system offers an automatic method with high flexibility to construct large-scale cell patterns for various applications.

This paper presents the first system for automated ex vivo perfusion of an isolated heart and regulating the heart's aortic pressure (AoP). An adaptive controller was developed for AoP regulation and maintained the heart's physiological aerobic metabolism. A mathematical model of the perfusion system was established based on a nonlinear equivalent circuit fluid flow model. The model combined with a virtual controller forms a reference model to generate the ideal trajectory of AoP. An adaptation algorithm tunes the control parameters based on the reference model and the isolated heart. Experiments were conducted using large animal hearts (55±5 kg porcine, n=6) to validate the adaptive controller's performance for stepwise and fast switching AoP references. The results confirmed that the the proposed controller is able to regulate the AoP of an isolated porcine heart in an accurate (mean error less than 2 mmHg) and fast (4~8 s of settling time) manner.

Learning from demonstration is a powerful tool for teaching manipulation actions to a robot. It is, however, an unsolved problem how to consider knowledge about the world and action-induced reactions such as forces imposed onto the gripper or measured liquid levels during pouring without explicit and case dependent programming. In this paper, we present a novel approach to include such knowledge directly in form of measured features. To this end, we use action demonstrations together with external features to learn a motion encoded by a dynamic system in a Gaussian Mixture Model (GMM) representation. Accordingly, during action imitation, the system is able to couple the geometric trajectory of the motion to measured features in the scene. We demonstrate the feasibility of our approach with a broad range of external features in real-world robot experiments including a drinking, a handover and a pouring task.

Surgical robots offer the exciting potential for remote telesurgery, but advances are needed to make this technology efficient and accurate to ensure patient safety. Achieving these goals is hindered by the deleterious effects of latency between the remote operator and the bedside robot. Predictive displays have found success in overcoming these effects by giving the operator immediate visual feedback. However, previously developed predictive displays can not be directly applied to telesurgery due to the unique challenges in tracking the 3D geometry of the surgical environment. In this paper, we present the first predictive display for teleoperated surgical robots. The predicted display is stereoscopic, utilizes Augmented Reality (AR) to show the predicted motions alongside the complex tissue found in-situ within surgical environments, and overcomes the challenges in accurately tracking slave-tools in real-time. We call this a Stereoscopic AR Predictive Display (SARPD). To test the SARPD's performance, we conducted a user study with ten participants on the da Vinci® Surgical System. The results showed with statistical significance that using SARPD decreased time to complete task while having no effect on error rates when operating under delay.

In robotic-assisted laparoscopic surgery, the first assistant (FA) stands at the bedside assisting the intervention, while the surgeon sits at the console teleoperating the robot. Tasks for the FA include navigating new instruments into the surgeon's field-of-view and passing in or retracting materials from the body using hand-held tools. We previously developed ARssist, an augmented reality application based on an optical see-through head-mounted display, to aid the FA. In this paper, we refine the system and first perform a pilot study with three experienced surgeons for two specific tasks: instrument insertion and tool manipulation. The results suggest that ARssist would be especially useful for less experienced assistants and for difficult hand-eye configurations. We then perform a multi-user study with inexperienced subjects. The results show that ARssist can reduce navigation time by 34.57%, enhance insertion path consistency by 41.74%, reduce root-mean-square path deviation by 40.04%, and reduce tool manipulation time by 72.25%. Thus, ARssist has the potential to improve efficiency, safety and hand-eye coordination, especially for novice assistants.

Lane change prediction of surrounding vehicles is a key building block of path planning. The focus has been on increasing the accuracy of prediction by posing it purely as a function estimation problem at the cost of model understandability. However, the efficacy of any lane change prediction model can be improved when both corner and failure cases are humanly understandable. We propose an attention-based recurrent model to tackle both understandability and prediction quality. We also propose metrics which reflect the discomfort felt by the driver. We show encouraging results on a publicly available dataset and proprietary fleet data.

In this paper, we study the Online Network Formation Problem (ONFP) for a mobile multi-robot system. Consider a group of robots with a bounded communication range operating in a large open area. One of the robots has a piece of information which has to be propagated to all other robots. What strategy should the robots pursue to disseminate the information to the rest of the robots as quickly as possible? The initial locations of the robots are unknown to each other, therefore the problem must be solved in an online fashion. For this problem, we present an algorithm whose competitive ratio is O(H \cdot\max\{M,\ \sqrt{MH}\})O(H \cdot\max\{M,\ \sqrt{MH}\}) for arbitrary robot deployments, where M is the largest edge length in the Euclidean minimum spanning tree on the initial robot configuration and H is the height of the tree. We also study the case when the robot initial positions are chosen uniformly at random and improve the ratio to O(M)O(M). Finally, we present simulation results to validate the performance in larger scales and demonstrate our algorithm using three robots in a field experiment.

Person re-identification is one of the key issues in both machine learning and video monitor application. In particular, defining an appropriate distance metric between the person images is very important. Existing metric learning approaches used in person re-identification either learn a single measure, or ignore the positive semi-definite (PSD) of measurement matrix, at the same time, since the number of negative sample pairs largely exceeds the number of positive sample pairs, some metric learning methods are largely influenced by the sample imbalance. Considering the above issues, we propose a new adaptive local metric learning method with positive semi-definite (PSD) constraint. Unlike existing metric learning methods which learn a single distance metric, we use an approximation error bound of a smooth metric matrix function over the data manifold to learn local metrics as linear combinations of basis metrics defined on anchor points over different regions of the instance space. Besides, we develop an efficient two stage algorithm that first learns the anchor points and the linear combinations of each instance, then learns the metric matrices of the anchor points. We employ the fast iterative shrinkage-thresholding algorithm which is a fast first-order optimization algorithm in the learning process of the linear combinations as well as the basis metrics of the anchor points. Our metric learning method has excellent performance. We firstly apply the proposed method on 5 UCI databases, which are widely used in machine learning, to test and evaluate the effectiveness of the proposed method. Then the proposed approach is applied for person re-identification, achieving better performance on three challenging databases (GRID, VIPeR, CUHK01) than the existing methods. The experimental results show that the proposed method can prvide the theoretical and practical support for the person re-identification problem.

In this paper, we have fabricated a multilayered hepatic lobule-like vascular network in a 3D tissue using a heptapole magnetic tweezer. The tissue consists of cell-laden hydrogels with 3D channel networks. To fabricate multilayered channel system, magnetic hydrogel fibers were manipulated by a magnetic tweezer. The hepatic lobule tissue shows a hexagonal structure with different sizes of veins. Six portal veins transfer the blood including nutrients and oxygen to a central vein by sinusoids. The portal and central veins are made by steel rods, whereas the magnetic hydrogel fibers has a role of sinusoids. An important point of this research is to connect two veins - portal and central vein - by magnetic fibers. For this, we used magnetic tweezer with seven poles to magnetize the steel rods. In order to generate high magnetic fields, we design magnetic tweezer with a flat tip and additional lower tweezer based on simulation data. The manipulation was performed in fibrin gel inside rat liver cells. By applying high magnetic fields, we attracted magnetic fibers to the steel rods and constructed 3D channel network in cellular structure. To verify the efficiency of the channel, we supply culture medium to the channel and then analyze the cell viability according to the distance from the channel. As a result, the cells located at close to the channel show higher cell viability than others.

Applications of mobile ground robots demand high speed and agility while navigating in complex indoor environments. These present an ongoing challenge in mobile robotics. A system with these specifications would be of great use for a wide range of indoor inspection tasks. This paper introduces Ascento, a compact wheeled bipedal robot that is able to move quickly on flat terrain, and to overcome obstacles by jumping. The mechanical design and overall architecture of the system is presented, as well as the development of various controllers for different scenarios. A series of experiments1 with the final prototype system validate these behaviors in realistic scenarios.

Despite impressive results in visual-inertial state estimation in recent years, high speed trajectories with six degree of freedom motion remain challenging for existing estimation algorithms. Aggressive trajectories feature large accelerations and rapid rotational motions, and when they pass close to objects in the environment, this induces large apparent motions in the vision sensors, all of which increase the difficulty in estimation. Existing benchmark datasets do not address these types of trajectories, instead focusing on slow speed or constrained trajectories, targeting other tasks such as inspection or driving. We introduce the UZH-FPV Drone Racing dataset, consisting of over 27 sequences, with more than 10 km of flight distance, captured on a first-person-view (FPV) racing quadrotor flown by an expert pilot. The dataset features camera images, inertial measurements, event-camera data, and precise ground truth poses. These sequences are faster and more challenging, in terms of apparent scene motion, than any existing dataset. Our goal is to enable advancement of the state of the art in aggressive motion estimation by providing a dataset that is beyond the capabilities of existing state estimation algorithms.

We introduce a fast approximate stability analysis into an automated dry stacking procedure. Evaluating structural stability is essential for any type of construction, but especially challenging in techniques where building elements remain distinct and do not use fasteners or adhesives. Due to the irregular shape of construction materials, autonomous agents have restricted knowledge of contact geometry, which makes existing analysis tools difficult to deploy. In this paper, a geometric safety factor called kern is used to estimate how much the contact interface can shrink and the structure still be feasible, where feasibility can be checked efficiently using linear programming. We validate the stability measure by comparing the proposed methods with a fully simulated shaking test in 2D. We also improve existing heuristics-based planning by adding the proposed measure into the assembly process.

In this paper, we formulate a combinatorial optimization problem that aims to maximize the accuracy of a lower bound estimate of the probability of security of a multi-robot system (MRS), while minimizing the computational complexity involved in its calculation. Security of an MRS is defined using the well-known control theoretic notion of left invertiblility, and the probability of security of an MRS can be calculated using binary decision diagrams (BDDs). The complexity of a BDD depends on the number of disjoint path sets considered during its construction. Taking into account all possible disjoint paths results in an exact probability of security, however, selecting an optimal subset of disjoint paths leads to a good estimate of the probability while significantly reducing computation. To deal with the dynamic nature of MRSs, we introduce two methods: (1) multi-point optimization, a technique that requires some a priori knowledge of the topology of the MRS over time, and (2) online optimization, a technique that does not require a priori knowledge, but must construct BDDs while the MRS is operating. Finally, our approach is validated on an MRS performing a rendezvous objective while exchanging information according to a noisy state agreement process.

Many applications of stereo depth estimation in robotics require the generation of accurate disparity maps in real time under significant computational constraints. Current state-of-the-art algorithms force a choice between either generating accurate mappings at a slow pace, or quickly generating inaccurate ones, and additionally these methods typically require far too many parameters to be usable on power- or memory-constrained devices. Motivated by these shortcomings, we propose a novel approach for disparity prediction in the anytime setting. In contrast to prior work, our end-to-end learned approach can trade off computation and accuracy at inference time. Depth estimation is performed in stages, during which the model can be queried at any time to output its current best estimate. Our final model can process 1242×375 resolution images within a range of 10-35 FPS on an NVIDIA Jetson TX2 module with only marginal increases in error - using two orders of magnitude fewer parameters than the most competitive baseline. The source code is available at https://github.com/mileyan/AnyNet.

By modeling the system with the mechanical, electrical, and magnetic field, we can derive the system equation for the actuator modeling. As it is not easy to conclude the output signal from the equations, we used Simulink to simulate and check the performance aspect of the system. After that, we did several experiments to verify whether experimental force meets with the needed condition for impact hot embossing and matches with simulation force. We tried to adjust the parameters of the system to match the force of experiment result and that of the simulation result. From the comparison, we can consider the analysis of the actuator as precise. A successful study can contribute to the better application of new type hot embossing techniques and better understanding and usage of the electromagnetic actuator when it is applied to another technology and research.