IROS 2019 Papers

The ability of deep learning models to generalize well across different scenarios depends primarily on the quality and quantity of annotated data. Labeling large amounts of data for all possible scenarios that a model may encounter would not be feasible; if even possible. We propose a framework to deal with limited labeled training data and demonstrate it on the application of vision-based vehicle control. We show how limited steering angle data available for only one condition can be transferred to multiple different weather scenarios. This is done by leveraging unlabeled images in a teacher-student learning paradigm complemented with an image-to-image translation network. The translation network transfers the images to a new domain, whereas the teacher provides soft supervised targets to train the student on this domain. Furthermore, we demonstrate how utilization of auxiliary networks can reduce the size of a model at inference time, without affecting the accuracy. The experiments show that our approach generalizes well across multiple different weather conditions using only ground truth labels from one domain.

Shared control plays a pivotal role in establishing effective human-robot interactions. Traditional control-sharing methods strive to complement a human's capabilities at safely completing a task, and thereby rely on users forming a mental model of the expected robot behaviour. However, these methods can often bewilder or frustrate users whenever their actions do not elicit the intended system response, forming a misalignment between the respective internal models of the robot and human. To resolve this model misalignment, we introduce Explainable Shared Control as a paradigm in which assistance and information feedback are jointly considered. Augmented reality is presented as an integral component of this paradigm, by visually unveiling the robot's inner workings to human operators. Explainable Shared Control is instantiated and tested for assistive navigation in a setup involving a robotic wheelchair and a Microsoft HoloLens with add-on eye tracking. Experimental results indicate that the introduced paradigm facilitates transparent assistance by improving recovery times from adverse events associated with model misalignment.

In this paper, we propose a control framework for a multi-human and mobile-robot collaborative team, that takes into account the co-workers' ergonomic requirements as well as the demand for high flexibility in the manufacturing industries. The new MObile Collaborative robotic Assistant (MOCA), which is composed of a lightweight manipulator arm, an underactuated hand, and a mobile platform driven by four omni-directional wheels enabling mobility in the workspace, is able to accomplish multiple tasks in a wide area with a high level of adaptability. In addition, an ergonomics module to anticipate and mitigate the human risk factors by means of a multi-object optimisation is integrated into the framework to ensure human safety and improvement of working conditions. The main advantage of this approach is that MOCA can assist multiple human operators, reducing their physical risks, with fast-adaptive capacities due to agile mobility and advanced interaction and manipulation. We validated the proposed method with an experiment simulating a simple manufacturing line which involves two subjects and the MOCA. The results demonstrate that the proposed framework is able to address multi-workers' ergonomics with a high level of flexibility in the workplace.

Both higher efficiency and cost reduction can be gained from automating bathymetric surveying for offshore applications such as pipeline, telecommunication or power cables installation and inspection on the seabed. We present a SLAM system that optimizes the geo-referencing of bathymetry surveys by fusing the dead-reckoning sensor data from the surveying vehicle with constraints from the maximization of the geometric consistency of overlapping regions of the survey. The framework has been extensively tested on bathymetric maps from both simulation and several actual industrial surveys and has proved robustness over different types of terrain. We demonstrate that our system is able to maximize the consistency of the final map even when there are large sections of the survey with reduced topographic variation. The framework has been made publicly available together with the simulation environment used to test it and some of the datasets.

This paper proposes a generic cleaning module to address lens fogging and soiling problems for insertable robotic cameras in laparoscopic surgery. The proposed lens cleaning module features minimal intraoperative interruption for surgeons to maintain clear visual field. The technical challenges for developing such a compact modular design involve confining the wiping mechanism within small space, yet delivering sufficient energy for the lens cleaning task. Inspired by thermo-activated phase transformation of shape memory alloy, we develop an effective mesoscale actuation mechanism to overcome the design challenges. A prototype is designed and manufactured for performance evaluation. The design effectiveness was verified by experiments.

The results of scientific experiments performed by different groups are rarely directly comparable. Efforts such as the European Robotics League offer to the community, in the form of competitions, well documented and stable benchmarks to assess the performance of existing systems. However, benchmarks can be equally useful at design time: the Plug&Bench Benchmark Meta-model provides robot designers with a valuable addition to their toolkit. Additionally, it enables -with benchmark composition-to predict system performance given the benchmark results of individual components.

Human-computer interaction (HCI) plays an important role in the near-field mixed reality, in which the hand-based interaction is one of the most widely-used interaction modes, especially in the applications based on optical see-through head-mounted displays (OST-HMDs). In this paper, such interaction modes as gesture-based interaction (GBI) and physics-based interaction (PBI) are developed to construct a mixed reality system to evaluate the advantages and disadvantages of different interaction modes. The ultimate goal is to find an efficient hybrid paradigm for mixed reality applications based on OST-HMDs to deal with the situations that a single interaction mode cannot handle. The results of the experiment, which compares GBI and PBI, show that PBI leads to a better performance of users regarding their work efficiency in the proposed two tasks. Some statistical tests, including T-test and one-way ANOVA, have also been adopted to prove that the difference regarding the efficiency between different interaction modes is significant. Experiments for combining both interaction modes are put forward in order to seek a good experience for manipulation, which proves that the partially-overlapping style would help to improve work efficiency for manipulation tasks. The experimental results of the proposed two hand-based interaction modes and their hybrid forms can provide some practical suggestions for the development of mixed reality systems based on OST-HMDs.

Cardiovascular diseases remain as the most common cause of death worldwide. Remotely manipulated robotic systems are utilized to perform minimally invasive endovascular interventions. The main benefits of this methodology include reduced recovery time, improvement of clinical skills and procedural facilitation. Currently, robotic assistance, precision, and stability of instrument manipulation are compensated by the lack of haptic feedback and an excessive amount of radiation to the patient. This paper proposes a novel master-slave robotic platform that aims to bring the haptic feedback benefit on the master side, providing an intuitive user interface, and clinical familiar workflow. The slave robot is capable of manipulating conventional catheters and guidewires in multi-modal imaging environments. The system has been initially tested in a phantom cannulation study under fluoroscopic guidance, evaluating its reliability and procedural protocol. As the slave robot has been entirely produced by additive manufacturing and using pneumatic actuation, MR compatibility is enabled and was evaluated in a preliminary study. Results of both studies strongly support the applicability of the robot in different imaging environments and prospective clinical translation.

In this study, we proposed a novel human-machine interface (HMI) for robotic manipulator control. The specific target was to adjust the impedance coefficients of the robot controller in real time by measuring the human forearm muscle contractions. We firstly designed a HMI system. Different from the frequently used sEMG technologies, the interface in our study could detect muscle morphological changes within the skin by the electrical impedance tomography (EIT). The sensing front-end was a soft elastic fabric band which was compatible to different arm shapes. With the specific designed sensing hardware and the re-construction algorithms, EIT images indicating forearm muscle shapes were obtained. We then designed a hybrid positon/impedance controller on a UR5 with the impedance coefficients being tuned in real time by the grasp force estimation. A sigmoid regression algorithm was used to map the EIT images to the grasp forces. After implementation of the whole system, two experiments were carried out. The first experiment was the off-line grasp force estimation. With the 1:1 cross validation, an average R2 of 0.83±0.04 and an average of the relative root mean square error (RRMSE) of 0.31±0.10 across 5 subjects were yielded. The second experiment was the real-time robot control. Trajectory tracking task with dynamic uncertainties were investigated and grasp forces were estimated in real-time. With higher muscle contraction levels, smaller position errors were observed and shorter time was needed to return to the expected trajectory when there were external disturbances. The results proved the feasibility of the new approach on human-robot interaction tasks. Future endeavours will be made to get more promising results.

Variable walking styles in bipedal robots may be used to communicate information about purpose and/or personality to human viewers- and may also help accommodate features of the environment. This paper presents variable gait in a simulation and results from a hardware prototype for gravity-driven rolling ball mechanism in a previously proposed bipedal design. First, optimal control inputs that produce a range of variable feasible gaits is generated on a simplified under-actuated planar model. Then, a tray-like mechanism that provides a curved path for a ball to roll on is presented. This mechanism is designed to replicate a notion of pelvic shift, described in Bartenieff Fundamentals, with movement of the ball (which is shaped by the tray) creating a shift of weight. Analysis of two gait styles and two tray designs shows comparable ranges of forces in the direction of travel between the simulated planar model and the hardware mechanism. This work is an important first step in generating feasible, stable- and variable- bipedal gaits using this hardware design.

When robotic assistance is present into vitreoretinal surgery, the surgeon will experience reduced sensory input that is otherwise derived from the tool's interaction with the eye wall (sclera). We speculate that disconnecting the surgeon from this sensory input may increase the risk of injury to the eye and affect the surgeon's usual technique. On the other hand, robot autonomous motion to enhance patient safety might inhibit the surgeons tool manipulation and diminish surgeon comfort with the procedure. In this study, to investigate the parameters of patient safety and surgeon comfort in a robot-assisted eye surgery, we implemented three different approaches designed to keep the scleral force in a safe range during a synergic eye manipulation task. To assess the surgeon comfort during these procedures, the amount of interference with the surgeons usual maneuvers has been analyzed by defining quantitative comfort metrics. The first two utilized scleral force control approaches are based on an adaptive force control method in which the robot actively counteracts any excessive force on the sclera. The third control method is based on a virtual fixture approach in which a virtual wall is created for the surgeon in the unsafe directions of manipulation. The performance of the utilized approaches was evaluated in user studies with two experienced retinal surgeons and the outcomes of the procedure were assessed using the defined safety and comfort metrics. Results of these analyses indicate the significance of the opted control paradigm on the outcome of a safe and comfortable robot-assisted eye surgery.

With the primary objective of human-robot interaction being to support humans' goals, there exists a need to formally synthesize robot controllers that can provide the desired service. Synthesis techniques have the benefit of providing formal guarantees for specification satisfaction. There is potential to apply these techniques for devising robot controllers whose specifications are coupled with human needs. This paper explores the use of formal methods to construct human-aware robot controllers to support the productivity requirements of humans. We tackle these types of scenarios via human workload-informed models and reactive synthesis. This strategy allows us to synthesize controllers that fulfill formal specifications that are expressed as linear temporal logic formulas. We present a case study in which we reason about a work delivery and pickup task such that the robot increases worker productivity, but not stress induced by high work backlog. We demonstrate our controller using the Toyota HSR, a mobile manipulator robot. The results demonstrate the realization of a robust robot controller that is guaranteed to properly reason and react in collaborative tasks with human partners.

This work presents a new human-centered method for indoor service robots to provide people with physical assistance and active guidance while traveling through congested and narrow spaces. As most previous work is robotcentered, this paper develops an end-to-end framework which includes a feedback path of the measured human positions. The framework combines a planning algorithm and a humanrobot interaction module to guide the led person to a specified planned position. The approach is deployed on a person-size dynamically stable mobile robot, the CMU ballbot. Trials were conducted where the ballbot physically led a blindfolded person to safely navigate in a cluttered environment.

Recent developments in virtual reality and robotic technologies have allowed investigating the behavioural and brain mechanisms that grounds self-consciousness in the multi-sensory (e.g. vision and touch) and sensorimotor processing of bodily signals. Yet, previous technological solutions to apply tactile stimuli for body illusion induction limit participants movements, do not allow for stimulations in dynamic environments (e.g., the subject walking), and can hardly be integrated into real-life settings and complex, interactive, virtual reality environments. Here, we present the development and first validation of a new semi-wearable haptic system, based on vibration technology, to induce a range of bodily illusions that are of relevance for research in psychiatry. This is a first step towards the development of wearable haptic systems able to administer touch and induce specific bodily illusions under dynamic conditions and in real-life settings.

A system for measuring radiation intensity and for radiation mapping by a micro unmanned robot using the Timepix detector is presented in this paper. Timepix detectors are extremely small, but powerful 14 × 14 mm, 256 × 256 px CMOS hybrid pixel detectors, capable of measuring ionizing alpha, beta, gamma radiation, and heaving ions. The detectors, developed at CERN, produce an image free of any digital noise thanks to per-pixel calibration and signal digitization. Traces of individual ionizing particles passing through the sensors can be resolved in the detector images. Particle type and energy estimates can be extracted automatically using machine learning algorithms. This opens unique possibilities in the task of flexible radiation detection by very small unmanned robotic platforms. The detectors are well suited for the use of mobile robots thanks to their small size, lightweight, and minimal power consumption. This sensor is especially appealing for micro aerial vehicles due to their high maneuverability, which can increase the range and resolution of such novel sensory system. We present a ROS-based readout software and real-time image processing pipeline and review options for 3-D localization of radiation sources using pixel detectors. The provided software supports off-the-shelf FITPix, USB Lite readout electronics with Timepix detectors.

This paper proposes the use of piecewise Cn smooth curve for mobile-base motion planning and control, coined Timed-Elastic Smooth Curve (TESC) planner. Based on a Timed-Elastic Band, the problem is defined so that the trajectory lies on a spline in SE(2) with non-vanishing n-th derivatives at every point. Formulated as a multi-objective nonlinear optimization problem, it allows imposing soft constraints such as collision-avoidance, velocity, acceleration and jerk limits, and more. The planning process is realtime-capable allowing the robot to navigate in dynamic complex scenarios. The proposed method is compared against the state-of-the-art in various scenarios. Results show that trajectories generated by the TESC planner have smaller average acceleration and are more efficient in terms of total curvature and pseudo-kinetic energy while being produced with more consistency than state-of-the-art planners do.

In this work we present a novel approach which compensates the destabilising effects of the time delay intrinsic in the control loop of an admittance-controlled robot employed for satellite dynamics simulation. The method is based on an energy storing element, the tank, which is exploited by the controller to preserve the passivity of the system and to avoid instability. Furthermore, we compare the performance of the proposed method with existing energy-based approaches, namely time-domain-passivity and wave variable transformation. The performance comparison and robustness of the methods are analysed in a Montecarlo simulation and validated experimentally.

The use of autonomous robots for surveillance is one of the most interesting applications of graph-patrolling algorithms. In recent years, considerable effort has been devoted to tackling the problem of efficiently computing effective patrolling strategies. One of the mainstream approaches is adversarial patrolling, where a model of a strategic attacker is explicitly taken into account. A common assumption made by these techniques is to consider a worst-case attacker, characterized by ubiquitous and perfect observation capabilities. Motivated by the domain of robotic applications, we instead consider a more realistic and limited attacker model capable of gathering noisy observations in a locally limited range of the environment. We assume that the modeled attacker follows a behavior induced by its observations. Thus, we devise a randomized patrolling strategy based on Markov chains that makes observations reveal very little information, while still maintaining a reasonable level of protection in the environment. Our experimental results obtained in simulation confirm time-variance as a practical approach for our objective.

This paper presents a general framework to find time-optimal trajectories for dynamic vehicles like drones and autonomous cars. Hindered by its nonlinear objective and complex constraints, this problem is hard even for state-of the-art nonlinear programming (NLP) solvers. The proposed framework addresses the problem by bilevel optimization. Specifically, the original problem is divided into an inner layer, which computes a time-optimal velocity profile along a fixed geometric path, and an outer layer, which refines the geometric path by a Quasi-Newton method. The inner optimization is convex and efficiently solved by interior-point methods. A novel variable reordering method is introduced to accelerate the optimization of the velocity profile. The gradients of the outer layer can be derived from the Lagrange multipliers using sensitivity analysis of parametric optimization problems. The method is guaranteed to return a feasible solution at any time, and numerical experiments on a ground vehicle with friction circle dynamics model show that the proposed method performs more robustly than general NLP solvers.