IROS 2018 Papers

Given that the success of an interaction task depends on the capability of the robot system to handle physical contact with its environment, pure motion control is often insufficient. This is especially true in the context of medical freehand ultrasound where the human body is a deformable surface and an unstructured environment, representing both a safety concern and a challenge for trajectory planning and control. The systematic tuning of practical high degree-of-freedom physical human-robot interaction (pHRI) tasks is not trivial and there are many parameters to be tuned. While traditional tuning is generally performed ad hoc and requires knowledge of the robot and environment dynamics, we propose a simple and effective online tuning framework using differential evolution (DE) to optimize the motion parameters for parallel force/impedance control in a pHRI and medical ultrasound motion application. Through real-world experiments with a KUKA LBR iiwa 7 R800 collaborative robot, the DE framework tuned motion control for optimal and safe trajectories along a human leg phantom. The optimization process was able to successfully reduce the mean absolute error of the motion contact force to 0.537 N through the evolution of eight motion control parameters.

Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis. The unique properties encompassed in the readings of such sensors provide high temporal resolution, superior sensitivity to light and low latency. These properties provide the grounds to estimate motion efficiently and reliably in the most sophisticated scenarios, but these advantages come at a price - modern event-based vision sensors have extremely low resolution, produce a lot of noise and require the development of novel algorithms to handle the asynchronous event stream. This paper presents a new, efficient approach to object tracking with asynchronous cameras. We present a novel event stream representation which enables us to utilize information about the dynamic (temporal)component of the event stream. The 3D geometry of the event stream is approximated with a parametric model to motion-compensate for the camera (without feature tracking or explicit optical flow computation), and then moving objects that don't conform to the model are detected in an iterative process. We demonstrate our framework on the task of independent motion detection and tracking, where we use the temporal model inconsistencies to locate differently moving objects in challenging situations of very fast motion.

This paper describes the development of a whole-body robotic simulator of an elderly person for evaluating robotics devices for nursing care. To improve the quality of life of the elderly persons, physical assistance such as transfer, movement, and bathroom assistance is important. It is also important to reduce the workload of caregivers in an aging society. In recent years, assistive robotic devices for nursing care have been developed and commercialized for such purposes. However, such devices have not become popular in the care facilities yet. One of the reasons is that it is still difficult to evaluate the effects of the devices on the care receivers and caregivers. In particular, it is necessary to quantitatively evaluate the effect of the devices on the human body from the viewpoint of safety and comfort. We have developed a whole-body robotic system to simulate the pose and motion of the elderly persons. The purpose of this system is to realize quantitative physical evaluation of robotics devices for nursing care of the human body. The experimental results of the preliminary evaluation of assistive robotic devices are also presented.

How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effctive. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as “pick up component A from the right side of the table.” Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.

Providing force feedback as a feature in current Robot-Assisted Minimally Invasive Surgery systems still remains a challenge. In recent years, Vision-Based Force Sensing (VBFS) has emerged as a promising approach to address this problem. Existing methods have been developed in a Supervised Learning (SL) setting. Nonetheless, most of the video sequences related to robotic surgery are not provided with ground-truth force data, which can be easily acquired in a controlled environment. A powerful approach to process unlabeled video sequences and find a compact representation for each video frame relies on using an Unsupervised Learning (UL) method. Afterward, a model trained in an SL setting can take advantage of the available ground-truth force data. In the present work, UL and SL techniques are used to investigate a model in a Semi-Supervised Learning (SSL) framework, consisting of an encoder network and a Long-Short Term Memory (LSTM) network. First, a Convolutional Auto-Encoder (CAE) is trained to learn a compact representation for each RGB frame in a video sequence. To facilitate the reconstruction of high and low frequencies found in images, this CAE is optimized using an adversarial framework and a L1-loss, respectively. Thereafter, the encoder network of the CAE is serially connected with an LSTM network and trained jointly to minimize the difference between ground-truth and estimated force data. Datasets addressing the force estimation task are scarce. Therefore, the experiments have been validated in a custom dataset. The results suggest that the proposed approach is promising.

The usage of articulated tools for autonomous robots is still a challenging task. One of the difficulties is to automatically estimate the tool's kinematics model. This model cannot be obtained from a single passive observation, because some information, such as a rotation axis (hinge), can only be detected when the tool is being used. Inspired by a baby using its hands while playing with an articulated toy, we employ a dual arm robotic setup and propose an interactive manipulation strategy based on visual-tactile servoing to estimate the tool's kinematics model. In our proposed method, one hand is holding the tool's handle stably, and the other arm equipped with tactile finger flips the movable part of the articulated tool. An innovative visuo-tactile servoing controller is introduced to implement the flipping task by integrating the vision and tactile feedback in a compact control loop. In order to deal with the temporary invisibility of the movable part in camera, a data fusion method which integrates the visual measurement of the movable part and the fingertip's motion trajectory is used to optimally estimate the orientation of the tool's movable part. The important tool's kinematic parameters are estimated by geometric calculations while the movable part is flipped by the finger. We evaluate our method by flipping a pivoting cleaning head (flap) of a wiper and estimating the wiper's kinematic parameters. We demonstrate that the flap of the wiper is flipped robustly, even the flap is shortly invisible. The orientation of the flap is tracked well compared to the ground truth data. The kinematic parameters of the wiper are estimated correctly.

A monocular 3D object tracking system generally has only up-to-scale pose estimation results without any prior knowledge of the tracked object. In this paper, we propose a novel idea to recover the metric scale of an arbitrary dynamic object by optimizing the trajectory of the objects in the world frame, without motion assumptions. By introducing an additional constraint in the time domain, our monocular visual-inertial tracking system can obtain continuous six degree of freedom (6-DoF) pose estimation without scale ambiguity. Our method requires neither fixed multi-camera nor depth sensor settings for scale observability, instead, the IMU inside the monocular sensing suite provides scale information for both camera itself and the tracked object. We build the proposed system on top of our monocular visual-inertial system (VINS) to obtain accurate state estimation of the monocular camera in the world frame. The whole system consists of a 2D object tracker, an object region-based visual bundle adjustment (BA), VINS and a correlation analysis-based metric scale estimator. Experimental comparisons with ground truth demonstrate the tracking accuracy of our 3D tracking performance while a mobile augmented reality (AR) demo shows the feasibility of potential applications.

We propose a method for a robot to manipulate an unknown door based on a single user instruction. The primary contributions of this paper are (i) to reduce the user instruction to a single click and (ii) to develop an efficient method to estimate an appropriate shape and manipulation model for a target door by integrating visual and touch information obtained by a robot. The proposed method first detects door candidates using a 3-D camera and then estimates the manipulation model of each candidate based on prior learning results. During door manipulation, the system integrates visual and touch information to estimate the shape and manipulation model to generate an appropriate motion. We evaluated the proposed method experimentally, and the results prove that the proposed method is effective.

Mobile robots are increasingly being used to assist with active pursuit and law enforcement. One major limitation for such missions is the resource (battery) allocated to the robot. Factors like nature and agility of evader, terrain over which pursuit is being carried out, plausible traversal velocity and the amount of necessary data to be collected all influence how long the robot can last in the field and how far it can travel. In this paper, we develop an analytical model that analyzes the energy utilization for a variety of components mounted on a robot to estimate the maximum operational range achievable by the robot operating on a single battery discharge. We categorize the major consumers of energy as: 1.) ancillary robotic functions such as computation, communication, sensing etc., and 2.) maneuvering which involves propulsion, steering etc. Both these consumers draw power from the common power source but the achievable range is largely affected by the proportion of power available for maneuvering. For this case study, we performed experiments with real robots on planar and graded surfaces and evaluated the estimation error for each case.

In order to effectively interact with or supervise a robot, humans need to have an accurate mental model of its capabilities and how it acts. Learned neural network policies make that particularly challenging. We propose an approach for helping end-users build a mental model of such policies. Our key observation is that for most tasks, the essence of the policy is captured in a few critical states: states in which it is very important to take a certain action. Our user studies show that if the robot shows a human what its understanding of the task's critical states is, then the human can make a more informed decision about whether to deploy the policy, and if she does deploy it, when she needs to take control from it at execution time.

Enhancing transparency of a teleoperation system by increasing the command-following bandwidth has not received lots of attention so far. This is considered a challenging task since in a teleoperation system the command-following bandwidth of the slave robot motion controller cannot be increased with a conventional motion controller as the desired trajectory is instantaneously commanded by the human user and thus, cannot be considered to be given in a pre-computed, smooth second order derivative form. We propose a method to increase the command-following bandwidth by extending the previously introduced Successive Stiffness Increment (SSI) approach to bilateral teleoperation. The approach allows realizing a very high motion controller gain, which cannot be realized with a conventional bilateral teleoperation controller as confirmed by experimental results.

Many robots make use of gravitational potential energy, generated by another mode, to enhance mobility through gliding locomotion. However, unstructured environments can create situations in which the initial conditions for steady-state gliding cannot be achieved; for example, jumping out of a hole, where the obstacle is very close to the robot. This paper suggests an optimization methodology for finding airfoil configurations and control strategies to maximize the effective non-steady-state gliding ratio for the most challenging initial condition, that of zero velocity. Parameters for the optimization are a location of a robot's center-of-mass in relation to its center-of-pressure and, through the addition of a tail, an active pitch control strategy. The optimal center-of-mass location produces the best passive gliding performance (morphological intelligence), and the optimal control strategy improves the gliding distance. Due to the aerodynamic complexities of modeling the collapsible airfoils, we find the optimal location of the center-of-mass from gliding experiments performed on the robot at different center-of-mass locations and initial pitch angles. An optimal location of the center-of-mass was found to be 40% of the wing chord for our robotic platform; measured from the wing's leading edge. The optimal location has a wide range of initial pitch angles which result in stable, yet non-steady-state, gliding behaviors. The morphological intelligence built into our robotic platform creates two observable dynamic behaviors, that of horizontal velocity gain and sink rate minimization. We then estimate the drag coefficients from the experiments, and conduct dynamic simulations to optimize the pitch control strategy. The design methodology presented here can enhance the non-steady-state gliding performance of a broad range of gliding robots, and the control strategy can further enhance performance on those which utilize an active tail. Show More

This work presents a method to increase the peak output speed of surface permanent magnet synchronous machine (SPMSM) motor drives with application in robotics using field weakening control. Contrary to most existing works, the strategy is stateless and operates using only a motor torque reference as input, making it suitable for robotics applications in which reference torque and speed are continuously and rapidly changing. Based on the system dynamics and constraints, we obtain four different operating modes. The strategy is extensively validated using three different experiments, which show an increase in peak velocity of up to 33%. The results demonstrate that the proposed strategy is effective in extending the dynamic performance and explosive motion capabilities of robots.

In this paper, we consider a non-conventional hexarotor whose propellers can be simultaneously tilted about two orthogonal axes: in this way, its underactuation degree can be easily adapted to the task at hand. For a given tilt profile, the minimum-energy trajectory between two prescribed boundary states is explicitly determined by solving an optimal control problem with respect to the angular accelerations of the six brushless motors. We also perform, for the first time, a systematic study of the singularities of the control allocation matrix of the hexarotor, showing the presence of subtle singular configurations that should be carefully avoided in the design phase. Numerical experiments conducted with the FAST-Hex platform illustrate the theory and delineate the pros and cons of dual-tilting paradigm in terms of maneuverability and energy efficiency.

Vibro-driven robotic (VDR) systems use stick-slip motions for locomotion. Due to the underactuated nature of the system, efficient design and control are still open problems. We present a new energy preserving design based on a spring-augmented pendulum. We indirectly control the friction-induced stick-slip motions by exploiting the passive dynamics in order to achieve an improvement in overall travelling distance and energy efficiency. Both collocated and non-collocated constraint conditions are elaborately analysed and considered to obtain a desired trajectory generation profile. For tracking control, we develop a partial feedback controller for the driving pendulum which counteracts the dynamic contributions from the platform. Comparative simulation studies show the effectiveness and intriguing performance of the proposed approach, while its feasibility is experimentally verified through a physical robot. Our robot is to the best of our knowledge the first nonlinear-motion prototype in literature towards the VDR systems.

Embedding the dynamics of the Spring Loaded Inverted Pendulum (SLIP) and applying a compositional controller around it can simplify dynamic legged robot locomotion control, but what is the energetic cost of this convenience? This paper measures the magnitude of this effect in such a way that the results are applicable to a wide class of jumping robots. A three-link monoped model with revolute joints is used to compare the energetic costs of locomotion using two different control approaches: 1) SLIP-embedding with a Raibert-style controller optimized for energetic efficiency, and 2) a trajectory optimized only for energetic efficiency. By performing this comparison in simulation for a large number of different monopeds randomly sampled from a space of realistic robot designs, it is found that the SLIP-Raibert approach requires, on average, almost twice the energy of the trajectory-optimized controller to traverse a given distance. Furthermore, the increase in energetic cost does not depend much on the particulars of the robot design, as the SLIP-Raibert approach requires at least 50% more energy for approximately 88% of realistic robot designs.

In this paper, we propose an end-effector for realizing various locomotion modes and simple work of a legged robot. The locomotion modes include climbing a vertical ladder, crawling, and walking. The simple work includes grasping and switching motions required at a disaster site. The developed end-effector has a two-pronged hook shape and two fingers for grasping and working and can be used to perform the locomotion and manipulation tasks described above. The experimental results confirmed that the four-limb robot WAREC-1 (WAseda REsCuer-No. 1) equipped with our proposed end-effector was able to climb a vertical ladder and perform the crawling motion. We also confirmed that the end-effector could grasp and switch five types of objects: a cylinder, cylinder with trigger, T-shaped, disk, and thin plate.

Convolutional neural networks are commonly used to control the steering angle for autonomous cars. Most of the time, multiple long range cameras are used to generate lateral failure cases. In this paper we present a novel model to generate this data and label augmentation using only one short range fisheye camera. We present our simulator and how it can be used as a consistent metric for lateral end-to-end control evaluation. Experiments are conducted on a custom dataset corresponding to more than 10000 km and 200 hours of open road driving. Finally we evaluate this model on real world driving scenarios, open road and a custom test track with challenging obstacle avoidance and sharp turns. In our simulator based on real-world videos, the final model was capable of more than 99% autonomy on urban road.

The functionalities and behaviours of socially assistive robots for the care of older people are usually defined by the robot's designers with limited room for runtime adaptation to meet the preferences, expectations and needs of the assisted person. However, adaptation plays a crucial role for the robot's acceptability and ultimately for its effectiveness. Culture, which deeply influences a person's preferences and habits, can be viewed as an invaluable “enabling technology” to achieve such level of adaptation. This paper discusses how guidelines describing culturally competent assistive behaviours can be encoded in a robot to effectively tune its actions, gestures and words. The proposed system is implemented on a Pepper robot and tested with an Indian persona, whose habits and preferences the robot discovers and adapts to at runtime.