ICRA 2018 Papers

Measuring the motility and morphology parameters of motile cells is important for revealing their functional characteristics. This paper presents automation techniques that, for the first time, enable automated, non-invasive measurement of motility and morphology parameters of individual sperms. Compared to the status quo of qualitative estimation of single sperm's motility and morphology based on embryologists' empirical experience, the automation techniques provide quantitative data in nearly real time. An adapted joint probabilistic data association filter (JPDAF) was used for multi-sperm tracking and tackled challenges of identifying sperms that intersect or have small spatial distances. Since the standard differential interference contrast (DIC) imaging method has side illumination effect which causes inherent inhomogeneous image intensity and poses difficulties for accurate sperm morphology measurement, we integrated total variation norm into the quadratic cost function method, which together effectively removed inhomogeneous image intensity and retained sperm's subcellular structures after DIC image reconstruction. In order to relocate the same sperm of interest identified under low magnification after switching to high magnification, coordinate transformation was conducted to handle the changes in the field of view caused by magnification switch. Experimental results demonstrated an accuracy of 95.6% in sperm motility measurement and errors <;10% in morphology measurement.

Using industrial robots to spray structures has been investigated extensively, however interesting challenges emerge when using handheld spraying robots. In previous work we have demonstrated the use of shared control of a handheld spraying robot to assist a user in a 3D spraying task. In this paper we demonstrate the use of Augmented Reality Interfaces to increase the user's progress and task awareness. We describe our solutions to challenging calibration issues between the Microsoft Hololens system and a motion capture system without the need for well defined markers or careful alignment on the part of the user. Error relative to the motion capture system was shown to be 10mm after only a 4 second calibration routine. Secondly we outline a logical approach for visualising liquid density for an augmented reality spraying task, this system allows the user to see target regions to complete, areas that are complete and areas that have been overdosed clearly. Finally we produced a user study to investigate the level of assistance that a handheld robot utilising shared control methods should provide during a spraying task. Using a handheld spraying robot with a moving spray head did not aid the user much over simply actuating spray nozzle for them. Compared to manual control the automatic modes significantly reduced the task load experienced by the user and significantly increased the quality of the result of the spraying task, reducing the error by 33-45%.

Soft robotic actuators are well suited for use in exoskeleton applications due to their innate compliance and low weight. We have developed a wearable soft robotic sleeve that uses fiber reinforced elastomeric enclosures (FREEs) to provide actuation and stiffness at the elbow for augmented lifting and carrying ability. The sleeve includes novel linear and helical actuator architectures to induce and resist joint movement respectively, and is intended to be comfortable, lightweight, and low profile. We developed test protocols to measure actuation and stiffness performance of different helical and linear architectures, and to compare helical and linear actuator groups when used individually and together. Our findings indicate that nested linear actuators have superior contraction ratios compared to parallel linear actuators, resulting in greater angular displacement. Stiffness from helical actuators increased with pressure and number of parallel actuators. A combined linear-helical actuator configuration considerably outperformed helical and linear actuator groups when used on their own.

Coordinating a team of robots to fulfill a common task is still a demanding problem. This is even more the case when considering uncertainty in the environment, as well as temporal dependencies within the task specification. A multi-robot cooperation from a single goal specification requires mechanisms for decomposing the goal as well as an efficient planning for the team. However, planning action sequences offline is insufficient in real world applications. Rather, due to uncertainties, the robots also need to closely coordinate during execution and adjust their policies when additional observations are made. The framework presented in this paper enables the robot team to cooperatively fulfill tasks given as temporal logic specifications while explicitly considering uncertainty and incorporating observations during execution. We present the effectiveness of our ROS implementation of this approach in a case study scenario.

This paper revisits the problem of estimating the attitude, linear velocity and depth of an IMU-Camera with respect to a planar target. The considered solution relies on the measurement of the optical flow (extracted from the continuous homography) complemented with gyrometer and accelerometer measurements. The proposed deterministic observer is accompanied with an observability analysis that points out camera's motion excitation conditions whose satisfaction grants stability of the observer and convergence of the estimation errors to zero. The performance of the observer is illustrated by performing experiments on a testbed IMU-Camera system.

In this paper, we address the problem of optimally fusing multiple heterogeneous and asynchronous sensors for use in 3D mapping and localization of autonomous vehicles. To this end, based on the factor graph-based optimization framework, we design a modular sensor-fusion system that allows for efficient and accurate incorporation of multiple navigation sensors operating at different sampling rates. In particular, we develop a general method of out-of-sequence (asynchronous) measurement alignment to incorporate heterogeneous sensors into a factor graph for mapping and localization in 3D, without requiring the addition of new graph nodes, thus allowing the graph to have an overall reduced complexity. The proposed sensor-fusion system is validated on a real-world experimental dataset, in which the asynchronous-measurement alignment is shown to have an improved performance when compared to a naive approach without alignment.

We consider a system composed of a bar tethered to two unmanned aerial vehicles (UAVs), where the cables behave as rigid links under tensile forces, and with the control objective of stabilizing the bar's pose around a desired pose. Each UAV is equipped with a PID control law, and we verify that the bar's motion is decomposable into three decoupled motions, namely a longitudinal, a lateral and a vertical. We then provide relations between the UAV s' gains, which, if satisfied, allows us to decompose each of those motions into two cascaded motions; the latter relations between the UAV s' gains are found so as to counteract the system asymmetries, such as the different cable lengths and the different UAV s' weights. Finally, we provide conditions, based on the system's physical parameters, that describe good and bad types of asymmetries. We present experiments that demonstrate the stabilization of the bar's pose.

Place recognition of pre-visited areas, widely known as Loop Closure Detection (LCD), constitutes one of the most important components in robotic applications, where the robot needs to estimate its pose while navigating through the field (e.g., simultaneous localization and mapping). In this paper, we present a novel approach for LCD based on the assignment of Visual Words (VWs) to particular places of the traversed path. The system operates in real time and does not require any pre-training procedure, such as visual vocabulary construction or descriptor-space dimensionality reduction. A place is defined through a dynamic segmentation of the incoming image stream and is assigned with VWs through the usage of an on-line clustering algorithm. At query time, image descriptors are converted into VWs on the map accumulating votes to the corresponding places. By means of a probability function, the mechanism is capable of identifying a loop closing candidate place. A nearest neighbor voting scheme on the descriptors' space allows the system to select the most appropriate image match at the chosen place. Geometrical and temporal consistency checks are applied on the proposed loop closing pair increasing the system's performance. Evaluation took place on several publicly available and challenging datasets offering high precision and recall scores as compared to other state-of-the-art approaches.

We present a semi-autonomous robotic pen-drawing system that is capable of creating pen art on an arbitrary surface with varying thickness of pen strokes but without reconstructing the surface explicitly. Our robotic system relies on an industrial, seven-degree-of-freedom (7DoF) manipulator that can be both position- and impedance-controlled. We use a vector-graphics engine to take an artist's pen drawing as input and generate Bézier spline curves with varying offsets. In order to estimate geometric details of the target, unknown surface, during drawing, we rely on incremental and adaptive sampling on the surface using a combination of position and impedance control. Then, our control algorithm physically replicates this drawing on any arbitrary, continuous surface by impedance-controlling the manipulator. We demonstrate that our system can create visually-pleasing and complicated artistic pen drawings on general surfaces without explicit surface-reconstruction nor visual feedback.

For online robots, incremental SLAM algorithms offer huge potential computational savings over batch algorithms. The dominant incremental algorithms are iSAM and iSAM2 which offer radically different approaches to computing incremental updates, balancing issues like 1) the need to re-linearize, 2) changes in the desirable variable marginalization order, and 3) the underlying conceptual approach (i.e. the “matrix” story versus the “factor graph” story). In this paper, we propose a new incremental algorithm that computes solutions with lower absolute error and generally provides lower error solutions for a fixed computational budget than either iSAM or iSAM2. Key to AprilSAM's performance are a new dynamic variable reordering algorithm for fast incremental Cholesky factorizations, a method for reducing the work involved in backsubstitutions, and a new algorithm for deciding between incremental and batch updates.

This paper considers the problem of finding the shortest tour to cover a given set of inverted cone views with apex angle α and height H when their apex points lie on a planar surface. This is a novel variant of the 3D Traveling Salesman Problem with intersecting Neighborhoods (TSPN) called Cone-TSPN. When the cones are allowed to tilt by an angle c we have the tilted Cone-TSPN problem, to which we present an algorithm that returns a solution with an approximation ratio of O (1+tan α/1-tan ϵ tan α (1 + log max(H)/min(H)). We demonstrate through simulations that our algorithm can be implemented in a practical way and by exploiting the structure of the cones we can achieve shorter tours. Finally, we present results from covering a reflective surface (lake area) that shows the importance of selecting different view angles under strong sunlight specularities.

Path planning under uncertainty is a difficult and often intractable problem. Autonomous agents must model and reason about complex stochastic processes to quickly derive high quality plans. Most approaches separate the model of uncertainty from the planning; a model is selected and then a controller derived. This work proposes an approach for fast path planning under uncertainty that scales the model of uncertainty such that good policies receive the most effort. To do this, we use an innovative form of the problem's chance constraint to formulate a convex, stochastic path planning problem from the non-convex problem. Next, a bound on the path's expected cost is developed that allows a trade-off between speed of computation and accuracy. The bound is trivially parallelized on a GPU. Finally, a modified branch and bound algorithm is introduced that scales computational effort for more promising solutions. The method is benchmarked against existing approaches including those using Boole's inequality, a MILP approach, and a parallelized sampling-based approach. It outperforms other approaches based on speed and the ability to meet the chance constraint while not being overly conservative.

In this paper, we propose a new methodology to embed deep learning-based algorithms in both visual recognition and motion planning for general mobile robotic platforms. A framework for an asynchronous deep classification network is introduced to integrate heavy deep classification networks into a mobile robot with no loss of system bandwidth. Moreover, a gaming reinforcement learning-based motion planner, a novel and convenient embodiment of reinforcement learning, is introduced for simple implementation and high applicability. The proposed approaches are implemented and evaluated on a developed robot, TT2-bot. The evaluation was based on a mission devised for a qualitative evaluation of the general purposes and performances of a mobile robotic platform. The robot was required to recognize targets with a deep classifier and plan the path effectively using a deep motion planner. As a result, the robot verified that the proposed approaches successfully integrate deep learning technologies on the stand-alone mobile robot. The embedded neural networks for recognition and path planning were critical components for the robot.

Close human-robot cooperation is a key enabler for new developments in advanced manufacturing and assistive applications. Close cooperation require robots that can predict human actions and intent, understanding human non-verbal cues. Recent approaches based on neural networks have led to encouraging results in the human action prediction problem both in continuous and discrete spaces. Our approach extends the research in this direction. Our contributions are three-fold. First, we validate the use of gaze and body pose cues as a means of predicting human action through a feature selection method. Next, we address two shortcomings of existing literature: predicting multiple and variable-length action sequences. This is achieved by applying an encoder-decoder recurrent neural network topology in the discrete action prediction problem. In addition, we theoretically demonstrate the importance of predicting multiple action sequences as a means of estimating the stochastic reward in a human robot cooperation scenario. Finally, we show the ability to effectively train the prediction model on an action prediction dataset, involving human motion data, and explore the influence of the model's parameters on its performance.

Fluent and safe interactions of humans and robots require both partners to anticipate the others' actions. The bottleneck of most methods is the lack of an accurate model of natural human motion. In this work, we present a conditional variational autoencoder that is trained to predict a window of future human motion given a window of past frames. Using skeletal data obtained from RGB depth images, we show how this unsupervised approach can be used for online motion prediction for up to 1660 ms. Additionally, we demonstrate online target prediction within the first 300-500 ms after motion onset without the use of target specific training data. The advantage of our probabilistic approach is the possibility to draw samples of possible future motion patterns. Finally, we investigate how movements and kinematic cues are represented on the learned low dimensional manifold.

Autonomous agents typically need a navigation mesh of a 3D virtual environment to allow efficient path planning. This mesh needs as input a continuous representation of the walkable areas. However, the walkable environment (WE), i.e. the parts of the 3D environment that an agent can walk on, may contain gaps. These may be due to the filtering steps performed to compute the WE, because of modelling errors in the 3D model, or simply be part of the geometry of the environment. We provide an algorithm that identifies and fills these gaps. We detect gaps, up to a given distance, between pairs of boundary edges of the walkable environment, and fill them with polygons. We employ a heuristic for choosing which pairs of edges should be connected. We compare our algorithm to Recast [10], a voxel-based method for navigation mesh generation. We find that our method gives more accurate results in many environments: it retains the exact representation of the walkable environment, semantically separates the gaps from the walkable areas, and requires no tweaking of parameters to obtain good results. However, our method is currently slower than Recast, and requires more memory.

An algorithm that uses low-cost inertial measurement units (IMUs) for estimating link angles for floating base robotic platforms is proposed. Each link has four IMUs attached on its surfaces, and an Extended Kalman Filter (EKF) and a Complementary Filter (CF) are used for fusing the sensors' data. The algorithm is validated with a commercial mobile working machine, which consist of six degrees-of-freedom (DOF) wheeled base platform, and a 3-DOF hydraulic anthropomorphic arm. Although there are vibrational disturbances from the machine's diesel engine and deformation of the links themselves, the measured results from the planar motion of a floating base hydraulic arm show that the accuracy of the angle estimation is impressively less than 1 degree in the root mean square (RMS) error.

Robust and dexterous manipulation is identified as one of the critical challenges in the field of robotic hand design and control. A key requirement of dexterous manipulation is the ability to modulate fingertip force directions and magnitudes. Cartesian stiffness control is a strategy to generate position dependent fingertip forces. However the stability conditions for the Cartesian stiffness controllers vary nonlinearly because of dependency on the manipulator's configuration and loading forces. The challenge is enhanced in case of tendon-driven robotic hands due to passive joint coupling. In this work, we derive a generalized passivity based stability boundary for Cartesian stiffness. We then present a methodology to analyze the stability boundaries of Cartesian stiffness controlled series elastic tendon-driven robotic fingers. We also present a solution to improve stability by optimizing the arrangement of optimized passive compliance in parallel to the actuators based on the stability criteria. Our analysis not only allows for informed design of new robotic hands but also applies to improving performance of existing robotic hands.

This paper presents a minimally invasive ultrasonic bone cutting tool designed for the da Vinci® research kit (dVRK). An ultrasonic transducer is modelled using finite element software, and correlated with testing results using an impedance analyzer. A multi-objective genetic algorithm is then used to design and analyze the remaining components of the ultrasonic system, in order to maximize system performance. The system is fabricated and mounted to the da Vinci® research kit system and tested on a cutting phantom.