IROS 2018 Papers

Pose estimation and mapping are key capabilities of most autonomous vehicles and thus a number of localization and SLAM algorithms have been developed in the past. Autonomous robots and cars are typically equipped with multiple sensors. Often, the sensor suite includes a camera and a laser range finder. In this paper, we consider the problem of incremental ego-motion estimation, using both, a monocular camera and a laser range finder jointly. We propose a new algorithm, that exploits the advantages of both sensors-the ability of cameras to determine orientations well and the ability of laser range finders to estimate the scale and to directly obtain 3D point clouds. Our approach estimates the 5 degrees of freedom relative orientation from image pairs through feature point correspondences and formulates the remaining scale estimation as a new variant of the iterative closest point problem with only one degree of freedom. We furthermore exploit the camera information in a new way to constrain the data association between laser point clouds. The experiments presented in this paper suggest that our approach is able to accurately estimate the ego-motion of a vehicle and that we obtain more accurate frame-to-frame alignments than with one sensor modality alone.

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network. The proposed RPN uses a novel architecture capable of performing multimodal feature fusion on high resolution feature maps to generate reliable 3D object proposals for multiple object classes in road scenes. Using these proposals, the second stage detection network performs accurate oriented 3D bounding box regression and category classification to predict the extents, orientation, and classification of objects in 3D space. Our proposed architecture is shown to produce state of the art results on the KITTI 3D object detection benchmark [1] while running in real time with a low memory footprint, making it a suitable candidate for deployment on autonomous vehicles. Code is available at: https://github.com/kujason/avod.

This paper reports the latest progress of an ongoing project utilizing a ducted-fan propulsion system to improve a humanoid robot's ability to step over a broad ditch with a height difference between the two sides. This work focuses on the methods of calculating the boundary and optimizing stepping posture to use less thrust and keep the robot balanced while stepping over the ditch. With the proposed methods and new two-dimensional gaits, the prototype robot, named Jet-HRl (Jet Humanoid Robot ver.l) was able to completely step over a broad ditch with 450mm in width (up to 97% of the robot's leg's length), and a height difference of 100mm between two sides.

Grasping skill is a major ability that a wide number of real-life applications require for robotisation. State-of-the-art robotic grasping methods perform prediction of object grasp locations based on deep neural networks. However, such networks require huge amount of labeled data for training making this approach often impracticable in robotics. In this paper, we propose a method to generate a large scale synthetic dataset with ground truth, which we refer to as the Jacquard grasping dataset. Jacquard is built on a subset of ShapeNet, a large CAD models dataset, and contains both RGB-D images and annotations of successful grasping positions based on grasp attempts performed in a simulated environment. We carried out experiments using an off-the-shelf CNN, with three different evaluation metrics, including real grasping robot trials. The results show that Jacquard enables much better generalization skills than a human labeled dataset thanks to its diversity of objects and grasping positions. For the purpose of reproducible research in robotics, we are releasing along with the Jacquard dataset a web interface for researchers to evaluate the successfulness of their grasping position detections using our dataset.

Dynamic walking robots have the potential for efficient and lifelike locomotion, but computing efficient gaits and tracking them is difficult in the presence of under-modeling. Iterative Learning Control (ILC) is a method to learn the control signal to track a periodic reference over several attempts, augmenting a model with online data. Terminal ILC (TILC), a variant of ILC, allows other performance objectives to be addressed at the cost of ignoring parts of the reference. However, dynamic walking robot gaits are not necessarily periodic in time. In this paper, we adapt TILC to jointly optimize final foot placement and energy efficiency on dynamic walking robots by indexing by a phase variable instead of time, yielding “phase-indexed TILC” (θ - TILC). When implemented on a five-link walker in simulation, θ- TILC learns a more energy-efficient walking motion compared to traditional time-indexed TILC.

Robotics and Functional Electrical Stimulation (FES) are well-established technologies for the rehabilitation of stroke and spinal cord injured (SCI) patients. We propose a hybrid solution that combines feedback-controlled FES of biceps and triceps as well as posterior and anterior deltoid with a cable-driven robotic system to support repetitive arm movements, like “breaststroke swimming” exercises. The robotic system partially compensates the arm weight by controlling the cable tension forces, and the FES promotes motion in the transversal plane. To adjust the FES support to the needs of the individual patients we use an iterative learning vector field (ILVF) which encodes the stimulation intensities that are applied to guide the patient along a pre-specified reference trajectory in the joint angle space. In contrast to previous iterative learning control approaches, the ILVF allows the patient to perform the motion at self-selected cadence. The proposed learning algorithm explicitly takes the dynamics of the artificially activated muscles into account and assures smooth stimulation intensity profiles. The control algorithm is tested in simulations using a complex neuro-musculoskeletal model. For “breaststroke” motions, the initial RMS error of purely volitional movements is reduced from 38° to 10° within 21 cycles by the adaptive FES support. After 50 iterations of the ILVF, the algorithm converges to a steady state RMS error of 4°. Changes in the patient's muscle activity and cadence were well tolerated by the control system and did not cause a noticable increase in the steady state RMS error.

Inverse reinforcement learning (IRL) is an important basis for learning from demonstrations. Observing an agent, human or robotic, perform a task provides information and facilitates learning the task. We show how the agent's preferences learned using IRL can be incorporated in a subject robot's decision making and planning, to enable the robot to spontaneously collaborate with the previously observed agent on the task. We prioritize a real-world application, where a line robot will autonomously collaborate with another robot in sorting ripe and unripe fruit such as oranges. Toward this, our evaluations utilize a colored-ball sorting task as an analog using simulated TurtleBots equipped with Phantom X arms. Our method is comprehensive providing first answers to questions such as how should the robot acquire the complete model for the collaborative planning problem and how should it solve the problem to obtain a plan that permits collaboration without disrupting the line robot's behavior.

Capturing clear images while a camera is moving fast, is integral to the development of mobile robots that can respond quickly and effectively to visual stimuli. This paper proposes to generate camera trajectories, with position and time constraints, that result in higher reconstructed image quality. The degradation in of an image captured during motion is known as motion blur. Three main methods exist for mitigating the effects of motion blur: (i) controlling optical parameters, (ii) controlling camera motion, and (iii) image reconstruction. Given control of a camera's motion, trajectories can be generated that result in an expected blur kernel or point-spread function. This work compares the motion blur effects and reconstructed image quality of three trajectories: (i) linear, (ii) polynomial, and (iii) inverse error. Where inverse error trajectories result in Gaussian blur kernels. Residence time analysis provides a basis for characterizing the motion blur effects of the trajectories.

In this paper we propose a (non-linear) smoothing algorithm for group-affine observation systems, a recently introduced class of estimation problems on Lie groups that bear a particular structure. As most non-linear smoothing methods, the proposed algorithm is based on a maximum a posteriori estimator, determined by optimization. But owing to the specific properties of the considered class of problems, the involved linearizations are proved to have a form of independence with respect to the current estimates, leveraged to avoid (partially or sometimes totally) the need to relinearize. The method is validated on a robot localization example, both in simulations and on real experimental data.

Flexible endoscopy is a routinely performed procedure that has predominantly remained unchanged for decades despite its many challenges. This paper introduces a novel, more intuitive and ergonomic platform that can be used with any flexible endoscope, allowing easier navigation and manipulation. A standard endoscope is robotized and a gaze control system based on eye-tracking is developed and implemented, allowing hands-free manipulation. The system characteristics and step response has been evaluated using visual servoing. Further, the robotized system has been compared with a manually controlled endoscope during a user study. The users (n=11) showed a preference for the gaze controlled endoscope and a lower task load when the task was performed with the gaze control. In addition, gaze control was related to a higher success rate and a lower time to perform the task. The results presented validate the system's technical performance and demonstrate the intuitiveness of hands-free gaze control in flexible endoscopy.

Deep learning has been successful in a variety of applications that have high-dimensional state spaces such as object recognition, video games, and machine translation. Deep neural networks can automatically learn important features from high-dimensional state given large training datasets. However, the success of deep learning in robot systems in the realworld is limited due to the cost of obtaining these large datasets. To overcome this problem, we propose an information-theoretic, intrinsically motivated, self-labeling mechanism using closed-loop control states. Taking this approach biases exploration to informative interactions-as such, a robot requires much less training to achieve reliable performance. In this paper, we explore the impact such an approach has on learning how to grasp objects. We evaluate different intrinsic motivators present in the literature applied appropriately in our framework and discuss the benefits and drawbacks of each.

An interval-based approach to cooperative localization for a group of unmanned aerial vehicles (UAVs) is proposed. It computes a pose uncertainty domain for each robot, i.e., a set that contains the true robot pose, assuming bounded error measurements. The algorithm combines distances measurements to the ground station and between UAVs, with the tracking of known landmarks in camera images, and provides a guaranteed enclosure of the robots pose domains. Pose uncertainty domains are computed using interval constraint propagation techniques, thanks to a branch and bound algorithm. We show that the proposed method also provides a good point estimate, that can be further refined using nonlinear iterative weighted least squares. Results are presented for simulated two-robots configurations, for experimental data, and compared with a classical Extended Kalman Filter.

The paper aims to develop a methodology of transferring the knowledge obtained from the experiments of laboratory animals to human musculoskeletal system. To achieve the goal, we propose a method for estimating the homologous posture of the mammalian skeletal system corresponding to the human body posture. We hypothesize the homology of bone geometry between mammalian species implies that of biomechanical functions. The method relies on this homology and determines the homologous postures according to the anatomical landmarks of bone geometry. This paper shows the results of the analysis on homologous postures between the human and mouse skeletal models to validate our hypothesis. A pilot study also introduces comparison of mechanical functions between the two models by using the homologous postures.

In recent years the topic of combining motion and symbolic planning to perform complex tasks in the field of robotics has received a lot of attention. The underlying idea is to have access at once to the reasoning capabilities of a task planner and to the ability of the motion planner to verify that the plan is feasible from a physical and geometrical point of view. The present work describes a framework to perform manipulation tasks that require the use of two robotic manipulators. To do so we employ a Hierarchical Task Network (HTN) planner interleaved with geometric constraint verification. In this framework we also consider observation actions and handle noisy perceptions from a probabilistic perspective. These ideas are put into practice by means of an experimental set-up in which two Barrett WAM robots have to cooperatively solve a geometric puzzle. Our findings provide further evidence that considering explicitly physical constraints during task planning, rather than deferring their validation to the moment of execution, is advantageous in terms of execution time and breadth of situations that can be handled.

For many robotic tasks, particularly those of service robots operating in human environments, the scope of object detection needs is greater than the available data. Either public datasets do not contain the entire set of objects needed for the task, and/or it is a commercial application that cannot use public datasets for training. Instead of hiring people to hand-label more data to support the integration of new objects into robot perception, we propose an interactive training process requiring zero hand labeling. With as little as 4 minutes of interaction with the robot per object, we demonstrate 99% precision and 57% recall in stationary object detection tasks.

In this paper, we address the challenge of robotic manipulation of elastically deforming objects. To this end, we model elastic objects using the Finite Element Method. Through a quasi-static assumption, we leverage sensitivity analysis to mathematically model how changes in the robot's configuration affect the deformed shape of the object being manipulated. This enables an interactive, simulation-based control methodology, wherein user-specified deformations for the elastic objects are automatically mapped to joint angle commands. The optimization formulation we introduce is general, operates directly within a robot's workspace and can readily incorporate joint limits as well as collision avoidance between the links. We validate our control methodology on a YuMi® IRB 14000, which we use to manipulate a variety of elastic objects.

Planning for autonomous driving in complex, urban scenarios requires accurate prediction of the trajectories of surrounding traffic participants. Their future behavior depends on their route intentions, the road-geometry, traffic rules and mutual interaction, resulting in interdependencies between their trajectories. We present a probabilistic prediction framework based on a dynamic Bayesian network, which represents the state of the complete scene including all agents and respects the aforementioned dependencies. We propose Markovian, context-dependent motion models to define the interaction-aware behavior of drivers. At first, the state of the dynamic Bayesian network is estimated over time by tracking the single agents via sequential Monte Carlo inference. Secondly, we perform a probabilistic forward simulation of the network's estimated belief state to generate the different combinatorial scene developments. This provides the corresponding trajectories for the set of possible, future scenes. Our framework can handle various road layouts and number of traffic participants. We evaluate the approach in online simulations and real-world scenarios. It is shown that our interaction-aware prediction outperforms interaction-unaware physics- and map-based approaches.

In this paper an interaction system based on a three dimensional virtual head projected onto a pyramidal display is proposed. The proposed system makes use of a social robot behavioral architecture already developed in our lab, which allows us to interchange developments between our robotic realizations and the 3D avatar. The overall system is divided into two parts: back projection subsystem and expression generator subsystem. The back projection subsystem projects a three-dimensional avatar onto a pyramidal structure in order to achieve a sensation of depth and realism. The expression generator subsystem carries out the avatar animations using shape keys and bones, following the Facial Action Coding System (FACS). The system consists in several nodes that are integrated in ROS middleware (Robotic Operating System), and includes a user interface that makes the avatar teleoperation easier (the package is avaible in github public respository). In order to evaluate the expressiveness of the system, two sets of experiments have been performed: one to analyze the avatar's gestural ability, that is, its capability to perform expressions that can be identified by an observer, and a second experiment to measure the emotion displaying ability in terms of valence and arousal.

Establishing a symbiotic relationship between a human and a exoskeleton is the end goal in many applications in order to provide benefits to the user. However, the literature focusing on the human side of human-exoskeleton interaction has remained less exhaustive than the literature focusing on the design (hardware/software) of the exoskeleton device itself. It is, though, essential to understand how a human adapts his motor control when interacting with an exoskeleton. Motor adaptation is an implicit process carried out by the central nervous system when the body encounters a perturbation, a paradigm that has been extensively studied in the field of human motor control research. When wearing an exoskeleton, even “as-transparent-as-possible”, contact/interaction forces may impact well-known motor control laws in a way that may be detrimental to the user, and even compromise usability in real applications. The present paper investigates how interaction with a backdrivable upper-limb exoskeleton (ABLE) set in “transparent” mode of control affects the kinematics/dynamics of human movement in a simple task. We find that important motor control features are preserved when moving with ABLE but an overall movement slowness occurs, likely as a response to increased inertia according to optimal control simulations. Such a human motor control approach illustrates one possible way to assess the degree of symbiosis between human and exoskeleton, i.e. by grounding on well-known findings in motor control research.