IROS 2019 Papers

A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multi-purpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task. Additionally, some mechanism is required to resolve the ambiguity or missing pieces of information in natural language instruction. Thus, we also develop a dialogue strategy that aims to gather additional information with minimal question-answer iterations and only when it is necessary. This work makes a significant stride towards enabling a human-like task understanding capability in a robot.

Nowadays the inspection of products at the end of line represents a critical phase. At this stage, it is necessary to look for defects in order to prevent the quality check to fail, and to provide information for improving the production as well. This task can be performed by using several sensing technologies, and the contactless optical inspection plays a key role. In this regard, the use of advanced robotic manipulators offers the capability to change the viewpoint of a given object and to inspect its multiple faces. We propose an approach that combines the use of photometric stereo to derive a 3D model of objects, empowered by the super-resolution that is applied on the original dataset (upstream) or on the normal images (downstream) in order to increase the quality of the final 3D model. The vision system is mounted on a robotic manipulator, able to grasp and change the viewpoint, thus offering a more complete view of the object to be inspected. The obtained results show that the developed solution increases the quality of the derived 3D models used for inspection tasks on different faces of the objects; this is achieved by using the manipulation ability offered by the adopted robotic platform.

Light-weight robotic manipulators in combination with power wheelchairs can help to restore the mobility of people with disabilities. While such systems are available on the market, they typically are limited to fully manual control modes. In research, shared control methods are employed, to increase the usability of these systems. Here, we present an additional extension, by introducing a whole-body control concept to the assistive robotic system EDAN. Combined with shared control, the whole-body controller allows the realization of complex tasks which necessitate the coordination of arm and platform, while ensuring compliant behavior resulting from the impedance control law. The implemented approach is analyzed and validated in an exemplary task of opening a door, passing through it and closing it afterwards. While this task would exceed the reachability of the arm in a classical approach, the combination of whole-body control with a shared control scheme allows for quick and efficient execution.

Adaptive, underactuated and compliant robot hands have received an increased interest over the last decade. Possible applications of these systems range from the development of simple grippers for industrial automation to the creation of anthropomorphic devices that can be used as prosthetic hands. These hands are particularly capable of extracting stable grasps even under significant object pose or other environmental uncertainties, due to the underactuation and the structural compliance of their designs. Despite the increased interest and the promising performance, adaptive hands suffer from several disadvantages and drawbacks. For example, the use of underactuation can lead to a post-contact reconfiguration of the fingers that compromises the force exertion capabilities of the system during pinch grasping. In this paper, we focus on the design, modelling, development, and evaluation of an adaptive robot gripper that uses magnets to adjust the reconfiguration profile of the fingers. The effect of the magnets increases the gripper's force exertion capabilities in pinch grasps, without compromising the full/caging grasps. The efficiency of the proposed gripper is experimentally validated through two different tests: i) a contact force test that compares the results of a theoretical model with the actual experimental results and ii) a grasping test that assesses the force exertion capabilities and the reconfiguration behaviour of the adaptive fingers for different implementations of the magnetic joints.

Recently, brushless motors with especially high torque densities have been developed for applications in autonomous aerial vehicles (i.e. drones), which usually employ exterior rotortype geometries (ER-BLDC motors). These motors are promising for other applications, such as humanoids and wearable robots; however, the emerging companies that produce motors for drone applications do not typically provide adequate technical specifications that would permit their general use across robotics-for example, the specifications are often tested in unrealistic forced convection environments, or are drone-specific, such as thrust efficiency. Furthermore, the high magnetic pole count in many ER-BLDC motors restricts the brushless drives able to efficiently commutate these motors at speeds needed for lightly-geared operation. This paper provides an empirical characterization of a popular ER-BLDC motor and a new brushless drive, which includes efficiencies of the motor across different power regimes, identification of the motor transfer function coefficients, thermal response properties, and closed loop control performance in the time and frequency domains. The intent of this work is to serve as a benchmark and reference for other researchers seeking to utilize these exciting and emerging motor geometries.

In this work we address the challenge of predicting the missing dimension (elevation angle) from 2D underwater sonar images. The high noise levels in these images, from phenomena such as non-diffuse reflections, frequently limits the usefulness of physical models. We thus propose the utilization of Convolutional Neural Networks (CNNs) as a powerful method to extract meaningful information without being misled by noisy data. We also introduce a self-supervised method that uses the physics of the sonar sensor to train the network on real data without ground-truth elevation maps. Our method can produce accurate elevation angle estimates given only a single image. Finally, we demonstrate that our method produces more accurate 3D reconstructions than competing methods, both in simulation and on real data.

In this paper, we introduce the Eigen-Factor (EF) method, which estimates a planar surface from a set of point clouds (PCs), with the peculiarity that these points have been observed from different poses, i.e. the trajectory described by a sensor. We propose to use multiple Eigen-Factors (EFs) or different planes' estimations, that allow to solve the multi-frame alignment over a sequence of observed PCs. Moreover, the complexity of the EFs optimization is independent of the number of points, but depends on the number of planes and poses. To achieve this, a closed-form of the gradient is derived by differentiating over the minimum eigenvalue with respect to poses, hence the name Eigen-Factor. In addition, a time-continuous trajectory version of EFs is proposed. The EFs approach is evaluated on a simulated environment and compared with two variants of ICP, showing that it is possible to optimize over all point errors, improving both the accuracy and computational time. Code has been made publicly available.

In this paper, we present CPL-Sync, a certifiably correct algorithm to solve planar pose graph optimization (PGO) using the complex number representation. We formulate planar PGO as the maximum likelihood estimation (MLE) on the product of unit complex numbers, and relax this nonconvex quadratic complex optimization problem to complex semidefinite programming (SDP). Furthermore, we simplify the corresponding semidefinite programming to Riemannian staircase optimization (RSO) on complex oblique manifolds that can be solved with the Riemannian trust region (RTR) method. In addition, we prove that the SDP relaxation and RSO simplification are tight as long as the noise magnitude is below a certain threshold. The efficacy of this work is validated through comparisons with existing methods as well as applications on planar PGO in simultaneous localization and mapping (SLAM), which indicates that the proposed algorithm is capable of solving planar PGO certifiably, and is more efficient in numerical computation and more robust to measurement noises than existing state-of-the-art methods. The C++ code for CPL-Sync is available at https://github.com/fantaosha/CPL-Sync.

Remote inspection of a complex environment is a difficult, time consuming task for human operators to perform. The need to manually avoid obstacles whilst considering other performance factors i.e. time taken, joint effort and information gained represents significant challenges to continuous operation. This paper proposes an autonomous robotic solution for exploration of an unknown, complex environment using a high DoF robot arm with an eye in hand depth sensor. The main contribution of this work is a new strategy to find the next best view by evaluating frontier regions of the map to maximise coverage, in contrast to many current approaches which densely sample joint or workspace configurations of the robot. Multiple utility functions were evaluated that showed different behaviours. Our results indicated that the presented algorithm can explore an arbitrary environment efficiently while optimising various performance criteria based on the utility function chosen, application constraints and the desires of the user.

Having an appropriate flapping path to yield efficient propulsion is an interesting issue in fish robotics. In most works, especially two-segmental structures, the flapping motion is limited to sinusoidal functions. In this paper, to cope with the aforementioned limitation, a conceptual non-sinusoidal path is proposed. The proposed flapping path and the conventional one, both are optimized for a sample fish robot. According to some simulation results, it is shown that if a proper actuator is employed to generate both optimized paths, the proposed approach yields more propulsion efficiency. Furthermore, it is discussed that our method can better imitate fish muscle output power. Finally, through experiments, some practical issues are considered.

The natural world hosts a few mammals that display both a high degree of agility and speed. However these animals have complex leg morphologies. Robot designers are thus faced with the dilemma of which morphology to employ when designing the next generation of agile legged robots. Thus this letter presents a novel investigation into the effects of limb morphology of quadrupeds during rapid transient maneuvers such as acceleration and deceleration. Three leg configurations inspired by nature (O-, X-Type and All-Ankle) as well as All-Knee configurations are compared. Extensive large scale Monte Carlo simulations utilizing contact-implicit trajectory optimization methods were employed on 100 randomly generated robots of varying sizes to determine the optimal configuration for the task of rapid transient locomotion. After extensive analysis, the investigation revealed that an X-Type leg configuration outperformed all other configurations. Ultimately, these results will provide insight for the mechanical design of future agile quadruped robots.

It is expected that human locomotion, in which the trunk twists and arms swing, provides stability as well as compensation for the angular momentum because the trunk and arms are placed over legs and then their position and the acceleration influence the bipedal locomotion. This paper presents a bipedal robot equipped with a trunk, embedding a viscoelastic joint and two arms away from the midline of the trunk. Physical experiments show that a passive oscillation of the viscoelastic trunk joint around the yaw axis vertical to the ground and a swing of heavy arms provided an oscillation of zero moment point (ZMP) in the lateral direction and a small oscillation in the anteroposterior direction. This is despite the fact that the robot does not have a roll joint and the arm swings back and forth. Numerical analysis supports the results of the physical experiments through a simple model and by expressing ZMP trajectory, showing that the arm swinging facilitates antiphase locomotion in which right arm and left leg swings ahead and vice versa.

This paper studies an effect of vibration on twisted string actuation through conduit at high bending angles. In our previous work [1], we have demonstrated that twisted string actuators can be used to transmit power even at significant deflection angles of the conduit. However, several undesirable effects, namely pull-back, hysteresis, and chattering, were present during actuation due to friction between strings and the internal sheath of the conduit. This paper reports the results of experimental study on effects of vibration on twisted string actuation inside deflected conduits. We have demonstrated that applying vibration generated near natural frequency of the system during the later stages of twisting and untwisting cycles helped reduce pull-back and hysteresis and increase string contraction. In case when sheath was deflected by 180° under a constant load of 3 kg, we were able to achieve over 70% decrease in pull-back and 30% decrease in hysteresis, compared with no vibration case.

We control a biped robot moving in a crowd with a Model Predictive Control (MPC) scheme that generates stable walking motions, with automatic footstep placement. Most walking strategies propose to re-plan the walking motion to adapt to changing environments only once at every footstep. This is because a footstep is planted on the ground, it usually stays there at a constant position until the next footstep is initiated, what naturally constrains the capacity for the robot to react and adapt its motion in between footsteps. The objective of this paper is to measure if re-planning the walking motion more often than once at every footstep can lead to an improvement in collision avoidance when navigating in a crowd. Our result is that re-planning twice (or more) during each footstep leads to a significant reduction of the number of collisions when walking in a crowd, but depends on the density of the crowd.

We present the first event-based learning approach for motion segmentation in indoor scenes and the first event-based dataset - EV-IMO- which includes accurate pixel-wise motion masks, egomotion and ground truth depth. Our approach is based on an efficient implementation of the SfM learning pipeline using a low parameter neural network architecture on event data. In addition to camera egomotion and a dense depth map, the network estimates independently moving object segmentation at the pixel-level and computes per-object 3D translational velocities of moving objects. We also train a shallow network with just 40k parameters, which is able to compute depth and egomotion. Our EV-IMO dataset features 32 minutes of indoor recording with up to 3 fast moving objects in the camera field of view. The objects and the camera are tracked using a VICON® motion capture system. By 3D scanning the room and the objects, ground truth of the depth map and pixel-wise object masks are obtained. We then train and evaluate our learning pipeline on EV-IMO and demonstrate that it is well suited for scene constrained robotics applications. SUPPLEMENTARY MATERIAL The supplementary video, code, trained models, appendix and a dataset will be made available at http://prg.cs.umd.edu/EV-IMO.html.

We present a new dataset with the goal of advancing the state-of-the-art in object pose estimation especially for stored porcelain and glass crockery in kitchen scenes. Specifically the ESKO 6d (EASE Stored Kitchen Objects with 6d poses) dataset features texture-less, glossy or glassy ordinary used objects which were naturally stored in a cupboard, drawer or dishwasher. There is a large degree of occlusion being the specific challenge in these scenes. Each scene was recorded in video sequences by two cameras (RGB-D (Kinect) and binocular) within multiple setup stages. The dataset contains an RGB-D image or binocular RGB image plus stereo-matched depth image as well as 6d pose ground truth and instance segmentation. Our dataset contains twelve stored object scenes with a combined amount of 47 video sequences captured by each camera, resulting in over 17k annotated Kinect images and more than 42k annotated stereo images showing around 50 different objects. The ground truth annotation is precise to 3. 5mm ADD (details see paper). The dataset can be accessed under http://www.informatik.uni-bremen.de/esko6d-dataset. Besides the concrete dataset we propose a method of ground truth pose measurement based on an external 3d tracking system that allows on the one hand to precisely measure the object’s pose inside a tight packed storage and on the other hand to obtain the object pose in several images with just one manual measurement.

Performance of current 3D point based detectors is limited by the number of points they can process, consequently limiting their accuracy. In this paper we propose a novel architecture coined as Edge-Aware PointNet, that incorporates geometric shape priors as binary maps, integrated in parallel with the PointNet++ framework, through convolutional neural networks (CNNs). The proposed architecture takes individual object instances as input and learns the task of object recognition for 3D shapes. To train the network, we present a dataset of 31k 2.5D synthetic point clouds rendered from ModelNet40. Through 2.5D representation, the network learns object recognition despite occlusion that enables improved performance on objects from real world, while 2D binary maps enable feature learning that is independent of number of points in the point cloud. Comprehensive experimentation shows that the proposed network is able to improve performance by 2.5% on ModelNet40 and 2.6% on ModelNet10 datasets, as compared to the baseline PointNet++. We also show improved performance as compared to state-of-the-art methods, on a real world RGBD dataset where our network improves results by 8%. Our code and dataset is publicly available at github.com/Merium88/Edge-Aware-PointNet.

Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline. Modern vision architectures require the agent to build a summary representation of the entire scene, even if most of the input is irrelevant to the agent's current goal. In this work, we flip this paradigm, by introducing EARLYFUSION vision models that condition on a goal to build custom representations for downstream tasks. We show that these goal specific representations can be learned more quickly, are substantially more parameter efficient, and more robust than existing attention mechanisms in our domain. We demonstrate the effectiveness of these methods on a simulated item retrieval problem that is trained in a fully end-to-end manner via imitation learning.

The flexible body has advantages over the rigid body in terms of environmental contact thanks to its underactuation. On the other hand, when applying conventional control methods to realize dynamic tasks with the flexible body, there are two difficulties: accurate modeling of the flexible body and the derivation of intermediate postures to achieve the tasks. Learning-based methods are considered to be more effective than accurate modeling, but they require explicit intermediate postures. To solve these two difficulties at the same time, we developed a real-time task control method with a deep recurrent neural network named Dynamic Task Execution Network (DTXNET), which acquires the relationship among the control command, robot state including image information, and task state. Once the network is trained, only the target event and its timing are needed to realize a given task. To demonstrate the effectiveness of our method, we applied it to the task of Wadaiko (traditional Japanese drum) drumming as an example, and verified the best configuration of DTXNET.