IROS 2016 Papers

Contact force play is a significant role in success of the cardiac ablation. However, it is still challenging to estimate contact force when a catheter is under large bending and multiple contacts. This paper develops a new multi-section static model of the tendon-driven catheters for both real-time intrinsic force sensing and interaction control. The model allows estimating the catheter shape by the external force at arbitrary location. Also, an algorithm is developed for the contact force estimation using the shape estimation with the catheter end-position tracking and tension feedback. In this study, we validated the contact force and shape estimation using a robotic platform, which steers a catheter consisting of 4 tendons with tension feedback. The shape estimation results show that the model can accurately predict the catheter shape; the position difference between measured and estimated was 2.5mm. The results of the contact force estimation show that 3-dimensional contact forces can be estimated accurately using the proposed method. The magnitude of contact force error was 0.0117N with 350Hz update rate.

Robotically Assisted Minimally Invasive Surgery (RAMIS) offers many advantages over manual surgical techniques. Most of the limitations of RAMIS stem from its nonintuitive user interface and costs. One way to mitigate some of the limitations is to automate surgical subtasks (e.g. suturing) such that they are performed faster while allowing the surgeon to plan the next step of the procedure. One component of successful suture automation is minimizing the internal tissue deformation forces generated by driving a needle through tissue. Minimizing the internal tissue forces requires segmenting the tissue deformation forces from other components of the needle tissue interaction (e.g. friction force). This paper proposes an Unscented Kalman Filter which can successfully model the force components, in particular the internal deformation force, generated by a needle as it is driven through a sample of tissue.

We consider multiple robots operating in a shared workspace, each given a high-level task specification in the form of Linear Temporal Logic (LTL) formulas. The robots have no a priori knowledge about the tasks of the other robots and might run into conflicts during task execution. In this work, we develop algorithms that allow the robots to autonomously resolve conflicts and complete their tasks with correctness guarantees, when possible. In our approach, the robots autonomously detect conflicts and trigger coordination within the subgroup of robots in conflict. The need-based coordination initiates the execution of a global robot controller on each robot in the subgroup until the robots all reach their current goals. The subgroup of robots then return to their local controllers and continue their execution. Our approach captures both the advantage of decentralized robot control and that of centralized robot control. The transition between centralized and decentralized robot control is seamless with guarantees provided.

This paper presents a heuristic framework that generates trajectories for quadrotor UAVs in complex environments that smoothly follow a series of waypoints with desired boundary states on the derivatives. The waypoints can be chosen manually or found using a sampling-based path planner such as RRT for complex environments with obstacles. A trajectory in the virtual domain is found using polynomials parametrised by an abstract argument that are numerically optimised to give the shortest geometrical path length. A mapping function is used to ensure the kinodynamic limits on the velocity and acceleration throughout the trajectory are satisfied. The mapping function is found heuristically with an algorithm that minimises the trajectory time.

This paper presents a framework for planning safe and efficient paths for a legged robot in rough and unstructured terrain. The proposed approach allows to exploit the distinctive obstacle negotiation capabilities of legged robots, while keeping the complexity low enough to enable planning over considerable distances in short time. We compute typical terrain characteristics such as slope, roughness, and steps to build a traversability map. This map is used to assess the costs of individual robot footprints as a function of the robot-specific obstacle negotiating capabilities for steps, gaps and stairs. Our sampling-based planner employs the RRT* algorithm to optimize path length and safety. The planning framework has a hierarchical architecture to frequently replan the path during execution as new terrain is perceived with onboard sensors. Furthermore, a cascaded planning structure makes use of different levels of simplification to allow for fast search in simple environments, while retaining the ability to find complex solutions, such as paths through narrow passages. The proposed navigation planning framework is integrated on the quadrupedal robot StarlETH and extensively tested in simulation as well as on the real platform.

In this paper we present the first planner for the problem of Navigation Among Movable Obstacles (NAMO) on a real robot that can handle environments with under-specified object dynamics. This result makes use of recent progress from two threads of the Reinforcement Learning literature. The first is a hierarchical Markov-Decision Process formulation of the NAMO problem designed to handle dynamics uncertainty. The second is a physics-based Reinforcement Learning framework which offers a way to ground this uncertainty in a compact model space that can be efficiently updated from data received by the robot online. Our results demonstrate the ability of a robot to adapt to unexpected object behavior in a real office scenario.

Nanorobot has the potential to automate the manipulation and observation processes at molecular scale. Current drug discovery process is labor and cost intensive. Thus, a strong demand exists to automate this process. Here, we developed an Atomic Force Microscopy (AFM) based nanorobot for in situ sensing molecular interactions for drug discovery. The AFM tip and sample substrate are functionalized by the molecules of interest. Via measuring the interactive binding forces between these two molecules using the AFM based nanorobot, we are able to test the effectiveness of drug candidates on attenuating or enhancing the molecular interactions involved in cell signaling pathways. To measure the single molecular interactions precisely, we developed a new substrate coating method. The new method functionalizes the substrate much more evenly compared with the previous method. We further optimized settings during measuring the interactive binding force, such as coating with bovine serum albumin (BSA) and the level of indentation of AFM tip onto the substrate. With these progresses, this nanorobot measures single molecular interactions automated. We further used this nanorobot to test a small peptide, which was designed to attenuate the interactive binding force between focal adhesion kinase (FAK) and protein kinase B (AKT), two molecules involved in cell adhesion.

This paper examines the problem of distributed path planning for a mobile sensor network comprised of communication-aware robots performing general information gathering missions. Mutual information is derived for distributed sensing over packet erasure channels that model multi-hop communication. We model distributed path planning as a non-cooperative game and derive utility functions that are optimized locally by each robot. Each robot computes the control input in a distributed manner that results in a combined action that can be bounded by the optimal centralized result by utilizing sub-modularity in certain cases. It is shown that when the communication model includes multi-hop communication to expand the coverage of the sensor network, the property of sub-modularity is lost. We further show that the additional global knowledge required for the local computation of utility functions can be learned by simple consensus approaches. Finally, we discuss a sampling approach to approximate the proposed utility functions in order to reduce the associated computational requirements.

Conventional path planning algorithms compute a single path through the configuration space. There is no guarantee that a physical robot will be able to track the trajectory while avoiding collisions, particularly in the presence of environmental perturbations and errors in the process model. Sequential composition combines planning and control by computing a sequence of controllers to execute rather than a single trajectory, offering greater safety guarantees. In this paper, we apply sequential composition to multirobot systems in a scalable fashion using M*, an advanced multirobot path planning algorithm. Controllers will vary in size and geometry, and thus take different amounts of time to execute. To handle these differences, we introduce the time augmented joint prepares graph and the approximate time augmented joint prepares graph which simplifies implementation by discretizing time. We validate our approach in a mixed reality test framework.

Although many tasks intrinsically involve multiple modalities, often only data from a single modality are used to improve complex robots acquisition of new skills. We present a method to equip robots with multimodal learning skills to achieve multimodal imitation on-the-fly on multiple concurrent task spaces, including vision, touch and proprioception, only using self-learned multimodal sensorimotor relations, without the need of solving inverse kinematic problems or explicit analytical models formulation. We evaluate the proposed method on a humanoid iCub robot learning to interact with a piano keyboard and imitating a human demonstration. Since no assumptions are made on the kinematic structure of the robot, the method can be also applied to different robotic platforms.

In this paper, we examine multi-target search, where one or more targets must be found by a moving robot. Given the target's initial probability distribution or the expected search region, we present an analysis of three search strategies - Global maxima search, Local maxima search, and Spiral search. We aim at minimizing the mean-time-to-find and maximizing the total probability of finding the target. This leads to two types of illustrative performance metrics: minimum time capture and guaranteed capture. We validate the search strategies with respect to these two performance metrics. In addition, we study the effect of different target distributions on the performance of the search strategies. We also consider the practical realization of the proposed algorithms for multi-target search. The search strategies are analytically evaluated, through simulations and illustrative deployments, in open-water with an Autonomous Surface Vehicle (ASV) and drifting sensor targets.

Despite the recent flight control regulations, Unmanned Aerial Vehicles (UAVs) are still gaining popularity in civilian and military applications, as much as for personal use. Such emerging interest is pushing the development of effective collision avoidance systems. Such systems play a critical role UAVs operations especially in a crowded airspace setting. Because of cost and weight limitations associated with UAVs payload, camera based technologies are the de-facto choice for collision avoidance navigation systems. This requires multi-target detection and tracking algorithms from a video, which can be run on board efficiently. While there has been a great deal of research on object detection and tracking from a stationary camera, few have attempted to detect and track small UAVs from a moving camera. In this paper, we present a new approach to detect and track UAVs from a single camera mounted on a different UAV. Initially, we estimate background motions via a perspective transformation model and then identify distinctive points in the background subtracted image. We find spatio-temporal traits of each moving object through optical flow matching and then classify those candidate targets based on their motion patterns compared with the background. The performance is boosted through Kalman filter tracking. This results in temporal consistency among the candidate detections. The algorithm was validated on video datasets taken from a UAV. Results show that our algorithm can effectively detect and track small UAVs with limited computing resources.

In this paper, we address coordination within a team of cooperative autonomous robots that need to accomplish a common goal. Our survey of the vast literature on the subject highlights two directions to further improve the performance of a multi-robot team. In particular, in a dynamic environment, coordination needs to be adapted to the different situations at hand (for example, when there is a dramatic loss of performance due to unreliable communication network). To this end, we contribute a novel approach for coordinating robots. Such an approach allows a robotic team to exploit environmental knowledge to adapt to various circumstances encountered, enhancing its overall performance. This result is achieved by dynamically adapting the underlying task assignment and distributed world representation, based on the current state of the environment. We demonstrate the effectiveness of our coordination system by applying it to the problem of locating a moving, non-adversarial target. In particular, we report on experiments carried out with a team of humanoid robots in a soccer scenario and a team of mobile bases in an office environment.

We propose a self-organising map (SOM) algorithm as a solution to a new multi-goal path planning problem for active perception and data collection tasks. We optimise paths for a multi-robot team that aims to maximally observe a set of nodes in the environment. The selected nodes are observed by visiting associated viewpoint regions defined by a sensor model. The key problem characteristics are that the viewpoint regions are overlapping polygonal continuous regions, each node has an observation reward, and the robots are constrained by travel budgets. The SOM algorithm jointly selects and allocates nodes to the robots and finds favourable sequences of sensing locations. The algorithm has polynomial-bounded runtime independent of the number of robots. We demonstrate feasibility for the active perception task of observing a set of 3D objects. The viewpoint regions consider sensing ranges and self-occlusions, and the rewards are measured as discriminability in the ensemble of shape functions feature space. Simulations were performed using a 3D point cloud dataset from a real robot in a large outdoor environment. Our results show the proposed methods enable multi-robot planning for budgeted active perception tasks with continuous sets of candidate viewpoints and long planning horizons.

In this paper we develop a new approach for decentralized multi-robot belief space planning in high-dimensional state spaces while operating in unknown environments. State of the art approaches often address related problems within a sampling based motion planning paradigm, where robots generate candidate paths and are to choose the best paths according to a given objective function. As exhaustive evaluation of all candidate path combinations from different robots is computationally intractable, a commonly used (sub-optimal) framework is for each robot, at each time epoch, to evaluate its own candidate paths while only considering the best paths announced by other robots. Yet, even this approach can become computationally expensive, especially for high-dimensional state spaces and for numerous candidate paths that need to be evaluated. In particular, upon an update in the announced path from one of the robots, state of the art approaches re-evaluate belief evolution for all candidate paths and do so from scratch. In this work we develop a framework to identify and efficiently update only those paths that are actually impacted as a result of an update in the announced path. Our approach is based on appropriately propagating belief evolution along impacted paths while employing insights from factor graph and incremental smoothing for efficient inference that is required for evaluating the utility of each impacted path. We demonstrate our approach in a synthetic simulation.

We present two multi-modal panoramic 3D outdoor (MPO) datasets for semantic place categorization with six categories: forest, coast, residential area, urban area and indoor/outdoor parking lot. The first dataset consists of 650 static panoramic scans of dense (9,000,000 points) 3D color and reflectance point clouds obtained using a FARO laser scanner with synchronized color images. The second dataset consists of 34,200 real-time panoramic scans of sparse (70,000 points) 3D reflectance point clouds obtained using a Velodyne laser scanner while driving a car. The datasets were obtained in the city of Fukuoka, Japan and are publicly available in [1], [2]. In addition, we compare several approaches for semantic place categorization with best results of 96.42% (dense) and 89.67% (sparse).

Robots in domestic environments are receiving more attention, especially in scenarios where they should interact with parent-like trainers for dynamically acquiring and refining knowledge. A prominent paradigm for dynamically learning new tasks has been reinforcement learning. However, due to excessive time needed for the learning process, a promising extension has been made by incorporating an external parent-like trainer into the learning cycle in order to scaffold and speed up the apprenticeship using advice about what actions should be performed for achieving a goal. In interactive reinforcement learning, different uni-modal control interfaces have been proposed that are often quite limited and do not take into account multiple sensor modalities. In this paper, we propose the integration of audiovisual patterns to provide advice to the agent using multi-modal information. In our approach, advice can be given using either speech, gestures, or a combination of both. We introduce a neural network-based approach to integrate multi-modal information from uni-modal modules based on their confidence. Results show that multi-modal integration leads to a better performance of interactive reinforcement learning with the robot being able to learn faster with greater rewards compared to uni-modal scenarios.

An extensive, precise and robust recognition and modeling of the environment is a key factor for next generations of Advanced Driver Assistance Systems and development of autonomous vehicles. In this paper, a real-time approach for the perception of multiple lanes on highways is proposed. Lane markings detected by camera systems and observations of other traffic participants provide the input data for the algorithm. The information is accumulated and fused using GraphSLAM and the result constitutes the basis for a multi-lane clothoid model. To allow incorporation of additional information sources, input data is processed in a generic format. Evaluation of the method is performed by comparing real data, collected with an experimental vehicle on highways, to a ground truth map. The results show that ego and adjacent lanes are robustly detected with high quality up to a distance of 120 m. In comparison to serial lane detection, an increase in the detection range of the ego lane and a continuous perception of neighboring lanes is achieved. The method can potentially be utilized for the longitudinal and lateral control of self-driving vehicles.

Humanoid robots that need to traverse constrained and uncertain environments require a suitable combination of perception, planning and control. This paper presents an integrated pipeline that allows the robot to autonomously acquire visual information, define step locations, compute feasible multi-contact stances using hands and feet, and generate a motion plan to reach the desired goal even going through different contact states. The execution of the desired path is guaranteed through a passivity-based multi-contact controller. The approach is evaluated in simulations and experiments in different scenarios using the humanoid robot TORO.