IROS 2019 Papers

Auto-exploration is a task for self-driving robots to explore unknown environments, which becomes much complicated when they move on irregular outdoor terrains. To improve the situation, a new frontier-based exploration algorithm is presented in this paper. It starts from original 3D cloud points of the environment to analyze the traversability of the scanned area, and further provides a reachability map to mark all map grid cells as reachable, dangerous or unknown. Frontier candidates are obtained from the reachable map, then clustered and reduced using an improved K-means. Finally, the target of next exploration step is selected from the frontiers left by evaluating their travel cost. The algorithm is validated on an irregular outdoor terrain and shows the capability for a field robot to explore on an irregular terrain.

Spatial cognition, as gained through the sense of vision, is one of the most important capabilities of human beings. However, for the visually impaired (VI), lack of this perceptual capability poses great challenges in their life. Therefore, we have designed Point-to-Tell-and-Touch, a wearable system with an ergonomic human-machine interface, for assisting the VI with active environmental exploration, with a particular focus on spatial intelligence and navigation to objects of interest in an alien environment. Our key idea is to link visual signals, as decoded synthetically, to the VI’s proprioception for more intelligible guidance, in addition to vision-to-audio assistance, i.e., finger pose, as indicated by pointing, is used as “proprioceptive laser pointer” to target an object in that line of sight. The whole system consists of two features, Point-to-Tell and Point-to-Touch, both of which can work independently or cooperatively. The Point-to-Tell feature contains a camera with a novel one-stage neural network tailored for blind-centered object detection and recognition, and a headphone telling the VI the semantic label and distance from the pointed object. the Point-to-Touch, the second feature, leverages a vibrating wrist band to create a haptic feedback tool that supplements the initial vectorial guidance provided by the first stage (hand pose being direction and the distance being the extent, offered through audio cues). Both platform features utilize proprioception or joint position sense. Through hand pose, the VI end user knows where he or she is pointing relative to their egocentric coordinate system and we are able to use this foundation to build spatial intelligence. Our successful indoor experiments demonstrate the proposed system to be effective and reliable in helping the VI gain spatial cognition and explore the world in a more intuitive way.

This paper presents an assisted telemanipulation approach with integrated grasp planning. It also studies how the human teleoperation performance benefits from the incorporated visual and haptic cues while manipulating objects in cluttered environments. The developed system combines the widely used master-slave teleoperation with our previous model-free and learning-free grasping algorithm by means of a dynamic grasp re-ranking strategy and a semi-autonomous reach-to-grasptrajectory guidance. The proposed re-ranking metric helps in dynamically updating the stable grasps based on the current state of the slave device. The trajectory guidance system assists in maintaining smooth trajectory by controlling the haptic forces. A virtual pose controller has been integrated with the guidance scheme to automatically correct the end-effector orientation while reaching towards the grasp. Various experiments are conducted evaluating the proposed method using a six degrees of freedom (dof) haptic master and a seven dof slave robot. Results obtained with these tests along with the results gathered from the performed human-factor trials demonstrate the efficiency of our method in terms of objective metrics of task completion, and also subjective metrics of user experience.

We consider multi-agent transport task problems where, e.g. in a factory setting, items have to be delivered from a given start to a goal pose while the delivering robots need to avoid collisions with each other on the floor.We introduce a Task Conflict-Based Search (TCBS) Algorithm to solve the combined delivery task allocation and multiagent path planning problem optimally. The problem is known to be NP-hard and the optimal solver cannot scale. However, we introduce it as a baseline to evaluate the sub-optimality of other approaches. We show experimental results that compare our solver with different sub-optimal ones in terms of regret.

This paper describes the design, manufacture, and performance of a highly dexterous, low-profile, 7 Degree-of-Freedom (DoF) robotic arm for CT-guided percutaneous needle biopsy. Direct CT guidance allows physicians to localize tumours quickly; however, needle insertion is still performed by hand. This system is mounted to a fully active motion stage superior to the patient's head and teleoperated by a radiologist. Unlike other similar robots, this robot's fully serial-link approach uses a unique combination of belt and cable drives for high-transparency and minimal-backlash, allowing for an expansive working area and numerous approach angles to targets, all while maintaining a small in-bore cross-section of less than 16cm2. Simulations verified the system's expansive collision free work-space and ability to hit targets across the entire chest, as required for lung cancer biopsy. Targeting error was on average <; 1mm on a teleoperated accuracy task, illustrating the system's sufficient accuracy to perform biopsy procedures. While this system was developed for lung biopsies, it can be easily modified for other CT-guided needle based procedures. Additionally, this system is open-hardware.

Learning manipulations from videos is an inspiriting way for robots to acquire new skills. In this paper, we propose a framework that can generate robotic manipulation plans by observing human demonstration videos without special marks or unnatural demonstrated behaviors. More specifically, the framework contains a video parsing module and a robot execution module. The first module recognizes the demonstrator's actions using two-stream convolution neural networks, and classifies the operated objects by adopting a Mask R-CNN. After that, two XGBoost classifiers are applied to further classify the objects into subject object and patient object respectively, according to the demonstrator's actions. In the second module, a grammar-based parser is used to summarize the videos and generate the common instructions for robot execution. Extensive experiments are conducted on a publicly available video datasets consisting of 273 videos and manifest that our approach is able to learn manipulation plans from demonstration videos with high accuracy (73.36%). Furthermore, we integrate our framework with a humanoid robot Baxter to perform the manipulation learning from demonstration videos, which effectively verifies the performance of our framework.

The concept of teleoperation has been studied since the advent of robotics and has found use in a wide range of applications, including exploration of remote or dangerous environments (e.g., space missions, disaster management), telepresence based time optimisation (e.g., remote surgery) and robot learning. While a significant amount of research has been invested into the field, intricate manipulation tasks still remain challenging from the user perspective due to control complexity. In this paper, we propose an intuitive, affordances oriented telemanipulation framework for a dual robot arm hand system. An object recognition module is utilised to extract scene information and provide grasping and manipulation assistance to the user, simplifying the control of adaptive, multi-fingered hands through a commercial Virtual Reality (VR) interface. The system's performance was experimentally validated in a remote operation setting, where the user successfully performed a set of bimanual manipulation tasks.

While suction cups are widely used in Robotics, the literature is underdeveloped when it comes to the modelling and simulation of the suction phenomenon. In this paper, we present a novel physically-based approach to simulate the behavior of active suction cups. Our model relies on a novel formulation which assumes the pressure exerted on a suction cup during active control is based on constraint resolution. Our algorithmic implementation uses a classification process to handle the contacts during the suction phenomenon of the suction cup on a surface. Then, we formulate a convenient way for coupling the pressure constraint with the multiple contact constraints. We propose an evaluation of our approach through a comparison with real data, showing the ability of our model to reproduce the behavior of suction cups. Our approach paves the way for improving the design as well as the control of robotic actuators based on suction cups such as vaccum grippers.

With the rapid advance of sophisticated control algorithms, the capabilities of drones to stabilise, fly and manoeuvre autonomously have dramatically improved, enabling us to pay greater attention to entire missions and the interaction of a drone with humans and with its environment during the course of such a mission. In this paper, we present an indoor office drone assistant that is tasked to run errands and carry out simple tasks at our laboratory, while given instructions from and interacting with humans in the space. To accomplish its mission, the system has to be able to understand verbal instructions from humans, and perform subject to constraints from control and hardware limitations, uncertain localisation information, unpredictable and uncertain obstacles and environmental factors. We combine and evaluate the dialogue, navigation, flight control, depth perception and collision avoidance components. We discuss performance and limitations of our assistant at the component as well as the mission level. A 78% mission success rate was obtained over the course of 27 missions.

This study presents an in-pipe inspection robot module, called AIRo-2.3s. This robot module can control its joint angle and torque and adapt to any directed bent pipe regardless of its orientation. Stretching the drive wheels against the inner wall of pipes is essential for adapting robots to be used in vertical pipes and slippery inner surfaces. To achieve this, a series elastic actuator (SEA) with a high reduction system and a polyurethane rubber is installed to sense the joint torque. More than 100 N constant traction force and a wide range of adaptive inner diameters (4 to 6 in.) are achieved despite the short body length, a minimum number of the drive wheels, and a simple joint of 1 degree of freedom. Experiments to verify the performance in bent pipes are conducted after the robot module configuration is described.

The handover of objects is a fundamental task in robot-human interaction. The nature of the handover can be varied by adjusting various parameters in order to achieve a human-like interaction. In this paper the final handover configuration is examined and analyzed. For this purpose, we conducted an experiment in which subjects taught a robot proper and improper end poses for different objects in different situations. We analyze the learned poses to determine what factors proper poses depend on. In addition to the poses, the configurations of the individual joints of the commercial robot are evaluated. Our results demonstrate that proper poses can be grouped into three clusters. These differ mainly in the rotation of the forearm. The height of a proper handover is also determined based on external factors.

In this paper, we address the problem of pose estimation from N 2D/3D point correspondences, known as the Perspective-n-Point (PnP) problem. Although many solutions have been proposed, it is hard to optimize both computational complexity and accuracy at the same time. In this paper, we propose an accurate and simultaneously efficient solution to the PnP problem. Previous PnP algorithms generally involve two sets of unknowns including the depth of each pixel and the pose of the camera. Our formulation does not involve the depth of each pixel. By introducing some intermediate variables, this formulation leads to a fourth degree polynomial cost function with 3 unknowns that only involves the rotation. In contrast to previous works, we do not address this minimization problem by solving the first-order optimality conditions using the off-the-shelf Gröbner basis method, as the Gröbner basis method may encounter numeric problems. Instead, we present a method based on linear system null space analysis to provide a robust initial estimation for a Newton iteration. Experimental results demonstrate that our algorithm is comparable to the start-of-the-art algorithms in terms of accuracy, and the speed of our algorithm is among the fastest algorithms.

Robust autonomous flight without GPS is key to many emerging drone applications, such as delivery, search and rescue, and warehouse inspection. These and other applications require accurate trajectory tracking through cluttered static environments, where GPS can be unreliable, while high-speed, agile, flight can increase efficiency. We describe the hardware and software of a quadrotor system that meets these requirements with onboard processing: a custom 300 mm wide quadrotor that uses two wide-field-of-view cameras for visual-inertial motion tracking and relocalization to a prior map. Collision-free trajectories are planned offline and tracked online with a custom tracking controller. This controller includes compensation for drag and variability in propeller performance, enabling accurate trajectory tracking, even at high speeds where aerodynamic effects are significant. We describe a system identification approach that identifies quadrotor-specific parameters via maximum likelihood estimation from flight data. Results from flight experiments are presented, which 1) validate the system identification method, 2) show that our controller with aerodynamic compensation reduces tracking error by more than 50% in both horizontal flights at up to 8.5 m/s and vertical flights at up to 3.1 m/s compared to the state-of-the-art, and 3) demonstrate our system tracking complex, aggressive, trajectories.

In the autonomous driving industry, vehicle dynamic models are important to control-in-the-loop simulations. For current commercial self-driving simulators, vehicle dynamic models are expressed explicitly by sophisticated analytical equations, which are accurate but difficult to build and expensive to scale to fleets of vehicles of different brands. In this paper, we introduce a highly automated learning-based vehicle dynamic modeling procedure, which has been deployed on Baidu Apollo self-driving platform, to support cross-vehicle data-driven applications on a large scale. Compared with our previous analytical models, the end-to-end learning-based dynamic models can achieve high accuracy with significantly reduced re-development effort.

As robots start to become ubiquitous in the personal workspace, it is necessary to have simple and intuitive interfaces to interact with them. In this paper, we propose an augmented reality (AR) interface for human-robot interaction (HRI) in a shared working environment. By fusing marker-based and markerless AR technologies, a mobile AR interface is created that enables a smartphone to detect planar surfaces and localize a manipulator robot in its working environment while obviating the need for a controlled or constrained environment. The AR interface and robot manipulator are integrated to render a system that enables users to perform pick-and-place task effortlessly. Specifically, a smartphone-based AR application is developed that allows a user to select any location within the robot's workspace by merely touching on the smartphone screen. Virtual objects, rendered at user-selected locations, are used to determine the pick and place locations of objects in the real world. The virtual object's start and end points, originally specified in the smartphone camera coordinate frame, are transformed into the robot coordinate frame for the robot manipulator to autonomously perform the assigned task. A user study is conducted with participants to evaluate the system performance and user experience. The results show that the proposed AR interface is user-friendly and intuitive to operate the robot, and it allows users to communicate their intentions through the virtual object easily.

Surgical robots for laparoscopy consist of several patient side slave manipulators that are controlled via surgeon operated master telemanipulators. Commercial surgical robots do not perform any sub-tasks - even of repetitive or noninvasive nature - autonomously or provide intelligent assistance. While this is primarily due to safety and regulatory reasons, the state of such automation intelligence also lacks the reliability and robustness for use in high-risk applications. Recent developments in continuous control using Artificial Intelligence and Reinforcement Learning have prompted growing research interest in automating mundane sub-tasks. To build on this, we present an inspired Asynchronous Framework which incorporates realtime dynamic simulation - manipulable with the masters of a surgical robot and various other input devices - and interfaces with learning agents to train and potentially allow for the execution of shared sub-tasks. The scope of this framework is generic to cater to various surgical (as well as non-surgical) training and control applications. This scope is demonstrated by examples of multi-user and multi-manual applications which allow for realistic interactions by incorporating distributed control, shared task allocation and a well-defined communication pipe-line for learning agents. These examples are discussed in conjunction with the design philosophy, specifications, system-architecture and metrics of the Asynchronous Framework and the accompanying Simulator. We show the stability of Simulator while achieving real-time dynamic simulation and interfacing with several haptic input devices and a training agent at the same time.

In this paper, a simple tracking control scheme is proposed for quadrotor systems with uncertain dynamics. It precludes the necessity for prohibitive analytic computation of the derivatives of the desired (virtual) attitude that is typically employed in controlling quadrotor systems. Moreover, this control scheme is approximation-free in the sense that it does not incorporate any adaptive laws, observers, or command filters to compensate for unknown parameters in the dynamics and the absence of the analytic differentiation, thus exhibiting remarkably low complexity levels and making its implementation straightforward. The thrust saturation is approached in the position control design which also enables the singularity in desired attitude extraction to be avoided entirely. It is demonstrated that based on the proposed scheme, the tracking errors can be made arbitrarily small by appropriately selecting design parameters. Extensive simulations and experiments are performed to verify the effectiveness of our scheme.

Self-healing robotics has been of considerable interest. We believe the function will have a major role in legged robots as a typical high-load application of robotics. Some pioneering works have been ongoing on self-healing soft robots. However, the development of large load self-healing component and its system integration with a life-sized legged robot is a challenging task. This study is to try the problem by a constructing self-healing component oriented for facilitated investigation. Proposed part enhances visibility and manufacturing by specializing tension transmission system. The developed module was evaluated by several experiments. First, healing visualization experiment was conducted to evaluate healing progress. In addition, the module's strength was tested using a motor-driven tendon module previously developed in our laboratory. Results of these experiments suggested that the stirring process have a major role in performing self-healing behaviour. Finally, we conducted a preliminary experiment on a tendon-driven legged robot. The experiment demonstrated that the module functioned in a real robot once.

This paper presents a new algorithm for a real-time obstacle avoidance for autonomous surface vehicles (ASV) that is capable of undertaking preemptive actions in complex and challenging scenarios. The algorithm is called adaptive velocity obstacle avoidance (AVOA) and takes into consideration the kinematic and dynamic constraints of autonomous vessels along with a protective zone concept to determine the safe crossing distance to obstacles. A configuration space that includes both the position and velocity of static or dynamic elements within the field-of-view of the ASV is supporting a particle swarm optimization procedure that minimizes the risk of harm and the deviation towards a predefined course while generating a navigation path with capabilities to prevent potential collisions. Extensive experiments demonstrate the ability of AVOA to select a velocity estimative for ASVs that originates a smoother, safer and, at least, two times more effective collision-free path when compared to existing techniques.