IROS 2019 Papers

In many real-world decision making problems, reaching an optimal decision requires taking into account a variable number of objects around the agent. Autonomous driving is a domain in which this is especially relevant, since the number of cars surrounding the agent varies considerably over time and affects the optimal action to be taken. Classical methods that process object lists can deal with this requirement. However, to take advantage of recent high-performing methods based on deep reinforcement learning in modular pipelines, special architectures are necessary. For these, a number of options exist, but a thorough comparison of the different possibilities is missing. In this paper, we elaborate limitations of fully-connected neural networks and other established approaches like convolutional and recurrent neural networks in the context of reinforcement learning problems that have to deal with variable sized inputs. We employ the structure of Deep Sets in off-policy reinforcement learning for high-level decision making, highlight their capabilities to alleviate these limitations, and show that Deep Sets not only yield the best overall performance but also offer better generalization to unseen situations than the other approaches.

This paper presents the technique of flex-and-flip manipulation. It is suitable for grasping thin, flexible linear objects lying on a flat surface. During the manipulation process, the object is first flexed by a robotic gripper whose fingers are placed on top of it, and later the increased internal energy of the object helps the gripper obtain a stable pinch grasp while the object flips into the space between the fingers. The dynamic interaction between the flexible object and the gripper is elaborated by analyzing how energy is exchanged. We also discuss the condition on friction to prevent loss of contact. Our flex-and-flip manipulation technique can be implemented with open-loop control and lends itself to underactuated, compliant finger mechanism. A set of experiments in robotic page turning performed with our customized hardware and software system demonstrates the effectiveness and robustness of the manipulation technique.

This paper presents a method with dynamic density topological structure generation for low-cost real-time vertical ladder detection from 3D point cloud data. Dynamic Density Growing Neural Gas (DD-GNG) is proposed to generate a dynamic density of the topological structure. The density of the structure and the number of nodes will be increased in the targeted object area. Feature extraction model is required to classify suspected objects for being processed in the next time process. After that, rungs of the vertical ladder is processed using an inlier-outlier method. Thus, the ladder detection model represents the ladder with a set of nodes and edges. Next, affordance detection is processed for detecting the feasible grasped location. To validate the effectiveness of the proposed method, a series of experiments are conducted on a 4-legged robot with a non-GPU board for real-time vertical ladder detection and climbing to validate the effectiveness of the proposed method. Results show that our proposed method able to detect and track the ladder structure in real-time with a much lower computational cost. The affordance of the ladder provides safety information for robot grasping.

The development of effective reduced order models for soft robots is paving the way toward the development of a new generation of model based techniques, which leverage classic rigid robot control. However, several soft robot features differentiate the soft-bodied case from the rigid-bodied one. First, soft robots are built to work in the environment, so the presence of obstacles in their path should always be explicitly accounted by their control systems. Second, due to the complex kinematics, the actuation of soft robots is mapped to the state space nonlinearly resulting in spaces with different sizes. Moreover, soft robots often include internal constraints and thus actuation is typically limited in the range of action and it is often unidirectional. This paper proposes a control pipeline to tackle the challenge of controlling soft robots with internal constraints in environments with obstacles. We show how the constraints on actuation can be propagated and integrated with geometrical constraints, taking into account physical limits imposed by the presence of obstacles. We present a hierarchical control architecture capable of handling these constraints, with which we are able to regulate the position in space of the tip of a soft robot with the discussed characteristics.

Duckiepond is an education and research development environment that includes software systems, educational materials, and of a fleet of autonomous surface vehicles Duckieboat. Duckieboats are designed to be easily reproducible with parts from a 3D printer and other commercially available parts, with flexible software that leverages several open source packages. The Duckiepond environment is modeled after Duckietown and AI Driving Olympics environments: Duckieboats rely only on one monocular camera, IMU, and GPS, and perform all ML processing using onboard embedded computers. Duckiepond coordinates commonly used middlewares (ROS and MOOS) and containerized software packages in Docker, making it easy to deploy. The combination of learning-based methods together with classic methods enables important maritime missions: track and trail, navigation, and coordinate among Duckieboats to avoid collisions. Duckieboats have been operating in a man-made lake, reservoir and river environments. All software, hardware, and educational materials are openly available (https://robotx-nctu.github.io/duckiepond), with the goal of supporting research and education communities across related domains.

Planning dual-arm assembly of more than three objects is a challenging Task and Motion Planning (TAMP) problem. The assembly planner shall consider not only the pose constraints of objects and robots, but also the gravitational constraints that may break the finished part. This paper proposes a planner to plan the dual-arm assembly of more than three objects. It automatically generates the grasp configurations and assembly poses, and simultaneously searches and backtracks the grasp space and assembly space to accelerate the motion planning of robot arms. Meanwhile, the proposed method considers gravitational constraints during robot motion planning to avoid breaking the finished part. In the experiments and analysis section, the time cost of each process and the influence of different parameters used in the proposed planner are compared and analyzed. The optimal values are used to perform real-world executions of various robotic assembly tasks. The planner is proved to be robust and efficient through the experiments.

Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a complex feature space into rewards. Second, the reward function has to be manually tuned by an expert. Manually tuning this reward function becomes a tedious task. In this paper, we propose an approach that relies on human driving demonstrations to automatically tune reward functions. This study offers important insights into the driving style optimization of general-purpose planners with maximum entropy inverse reinforcement learning. We evaluate our approach based on the expected value difference between learned and demonstrated policies. Furthermore, we compare the similarity of human driven trajectories with optimal policies of our planner under learned and expert-tuned reward functions. Our experiments show that we are able to learn reward functions exceeding the level of manual expert tuning without prior domain knowledge.

Monocular depth estimation is the task of obtaining a measure of distance for each pixel using a single image. It is an important problem in computer vision and is usually solved using neural networks. Though recent works in this area have shown significant improvement in accuracy, the state-of-the-art methods tend to require massive amounts of memory and time to process an image. The main purpose of this work is to improve the performance of the latest solutions with no decrease in accuracy. To this end, we introduce the Double Refinement Network architecture. The proposed method achieves state-of-the-art results on the standard benchmark RGB-D dataset NYU Depth v2, while its frames per second rate is significantly higher (up to 18 times speedup per image at batch size 1) and the RAM usage is lower.

Robotic systems are ever more capable of automation and fulfilment of complex tasks, particularly with reliance on recent advances in intelligent systems, deep learning and artificial intelligence in general. However, as robots and humans come closer together in their interactions, the matter of interpretability, or explainability of robot decision-making processes for the human grows in importance. A successful interaction and collaboration would only be possible through mutual understanding of underlying representations of the environment and the task at hand. This is currently a challenge in deep learning systems. We present a hierarchical deep reinforcement learning system, consisting of a low-level agent handling the large actions/states space of a robotic system efficiently, by following the directives of a high-level agent which is learning the high-level dynamics of the environment and task. This high-level agent forms a representation of the world and task at hand that is interpretable for a human operator. The method, which we call Dot-to-Dot, is tested on a MuJoCo-based model of the Fetch Robotics Manipulator, as well as a Shadow Hand, to test its performance. Results show efficient learning of complex actions/states spaces by the low-level agent, and an interpretable representation of the task and decision-making process learned by the high-level agent.

One of the main challenges in the vision-based grasping is the selection of feasible grasp regions while interacting with novel objects. Recent approaches exploit the power of convolutional neural network (CNN) to achieve accurate grasping at the cost of high computational power and time. In this paper, we present a novel unsupervised learning based algorithm for the selection of feasible grasp regions. Unsupervised learning infers the pattern in dataset without any external labels. We applied k-means clustering at every sampling stage in image plane to identify the grasp regions, followed by axes assignment method. We define a novel concept of Grasp Decide Index (GDI) to select the best grasp pose in the image plane. We have conducted several experiments in clutter or isolated environment on standard objects of Amazon Robotics Challenge 2017 and Amazon Picking Challenge 2016. We compared the results with prior learning based approaches to validate the robustness and adaptive nature of our algorithm for a variety of novel objects in different domains.

Many multirotor Unmanned Aerial Systems applications have a critical need for precise position control in environments with strong dynamic external disturbances such as wind gusts or ground and wall effects. Moreover, to maximize flight time, small multirotor platforms have to operate within strict constraints on payload and thus computational performance. In this paper, we present the design and experimental comparison of Model Predictive and PID multirotor position controllers augmented with a disturbance estimator to reject strong wind gusts up to 12 m/s and ground effect. For disturbance estimation, we compare Extended and Unscented Kalman filtering. In extensive in- and outdoor flight tests, we evaluate the suitability of the developed control and estimation algorithms to run on a computationally constrained platform. This allows to draw a conclusion on whether potential performance improvements justify the increased computational complexity of MPC for multirotor position control and UKF for disturbance estimation.

Distributed scalable algorithms are sought in many multi-robot contexts. In this work we address the dynamic optimal linear assignment problem, exemplified as a target tracking mission in which mobile robots visually track mobile targets in a one-to-one capacity. We adapt our previous work on formation achievement by means of a distributed simplex variant, which results in a conceptually simple consensus solution, asynchronous in nature and requiring only local broadcast communications. This approach seamlessly tackles dynamic changes in both costs and network topology. Improvements designed to accelerate the global convergence in the face of dynamically evolving task rewards are described and evaluated with simulations that highlight the efficiency and scalability of the proposal. Experiments with a team of three Turtlebot robots are finally shown to validate the applicability of the algorithm.

Shared control for human-robot teams - where both the human and the robot's autonomy provide commands to the hardware - offers advantages over fully teleoperated or fully autonomous systems by utilizing the unique skill sets of both the human and robot's autonomy simultaneously. However, the mechanism by which control is shared is often static and many teams could benefit from adjusting this mechanism, such that the human or autonomy alternatively receive more control authority in different scenarios. The question then is: how do we know when these scenarios occur? In this paper, we present a method to estimate the performance of human-robot teams using a novel metric called Discrete N-Dimensional Entropy of Behavior (DNDEB). DNDEB utilizes knowledge of a high-performing human-robot team to build a model of how the team should operate. The model is used to predict the human's command. The error between the prediction and actual command is tracked and after a certain number of samples, entropy is estimated. A higher level of entropy corresponds to deviations from the high-performance model, which can be interpreted as poor performance by the human-robot team (e.g., long task time or a collision). Our formulation offers several advantages: it (1) accepts discrete inputs of any size, (2) does not require additional sensors, and (3) is tunable to the specific application. To validate this, we conduct a 15person study where subjects operated a powered wheelchair under three different shared-control paradigms. We find that entropy is higher for cases with longer task durations and cases where there is a collision. Moreover, we use DNDEB thresholds as a mechanism to predict the performance of the human-robot team online and find an average accuracy of 91% with a prescience rate of 72%.

For building a disaster response robot, WAREC-l's fully-automated system for manipulating a valve, this paper proposes a method for (1) detecting a valve which is far away from the robot, (2) estimating the position and orientation for grasping the valve by the robot at a closer position. Our methods do not need any prior information about a valve for the above-mentioned detection and estimation for grasping. In addition, our estimation for grasping provides useful information, by which WAREC-I can rotate a valve autonomously. The method (1) uses the RGB image and the point cloud data captured by Multisense SL as the input, and estimate the position and orientation of a valve far away from the robot. The method (2) uses both the RGB and depth images captured by KinectV2 as input and estimate information for grasping the valve. Our experiments are conducted using a real disaster response robot. Our experimental results show the error of the estimation by the (a) two methods are small enough to achieve a fully-automated system for detecting and rotating the valve by WAREC-I.

Real-time 3D reconstruction from RGB-D sensor data plays an important role in many robotic applications, such as object modeling and mapping. The popular method of fusing depth information into a truncated signed distance function (TSDF) and applying the marching cubes algorithm for mesh extraction has severe issues with thin structures: not only does it lead to loss of accuracy, but it can generate completely wrong surfaces. To address this, we propose the directional TSDF-a novel representation that stores opposite surfaces separate from each other. The marching cubes algorithm is modified accordingly to retrieve a coherent mesh representation. We further increase the accuracy by using surface gradient-based ray casting for fusing new measurements. We show that our method outperforms state-of-the-art TSDF reconstruction algorithms in mesh accuracy.

Object detection is an integral part of an autonomous vehicle for its safety-critical and navigational purposes. Traffic sign as an object plays a vital role in guiding such systems. However, if the vehicle fails to locate any critical sign, it might make a catastrophic failure. In this paper, we are proposing an approach to identify traffic signs that have been mistakenly discarded by the object detector. The proposed method raises an alarm when it discovers a failure by the object detector to detect a traffic sign. This approach can be useful to evaluate the performance of the detector during the deployment phase. We trained a single shot multi-box object detector to detect traffic signs and used its internal features to train a separate false negative detector (FND). During deployment, FND decides whether the traffic sign detector (TSD) has missed a sign or not. We are using precision and recall to measure the accuracy of FND in two different datasets. For 80% recall, FND has achieved 89.9% precision in Belgium Traffic Sign Detection dataset and 90.8% precision in German Traffic Sign Recognition Benchmark dataset respectively. To the best of our knowledge, our method is the first to tackle this critical aspect of false negative detection in robotic vision. Such a fail-safe mechanism for object detection can improve the engagement of robotic vision systems in our daily life.

In this paper, a new control method for a hexapod robot walking on irregular terrain based on a human operator's foot-placement navigation is proposed and evaluated. The control method is based on the Follow-the-contact-point (FCP) gait control that operates on the principle that each leg follows the contact point of its foreleg. Hence, planning the contact points for all the legs are summarized to one of the front legs. To dedicate the FCP gait control to a hexapod robot, three control architectures are added. First, new constraints in the transition from a stance phase to a swing phase are added to maintain static stability when a leg leaves the ground. Second, a real-time posture control system for contacting legs is added. The third is an adaptive control to adjust the time elapsed in each control mode, by which specifications of deadlock-free and static stability are satisfied. The proposed control architecture is installed to a small hexapod robot, and its performance is evaluated through experiments wherein the robot walks on uneven terrain.

Large demand for robotics and automation has been reflected in the sanding works, as current manual operations are labor-intensive, without consistent quality, and also subject to safety and health issues. While several machines have been developed to automate one or two steps in the sanding works, the autonomous capability of existing solutions is relatively low, and the human assistance or supervision is still heavily required in the calibration of target objects or the planning of robot motion and tasks. This paper presents the development of an autonomous sanding robot, which is able to perform the sanding works on an unknown object automatically, without any prior calibration or human intervention. The developed robot works as follows. First, the target object is scanned then modeled with the structured-light camera. Second, the robot motion is planned to cover all the surfaces of the object with an optimized transition sequence. Third, the robot is controlled to perform the sanding on the object under the desired impedance model. A prototype of the sanding robot is fabricated and its performance is validated in the task of sanding a batch of wooden boxes. With sufficient degrees of freedom (DOFs) and the customization of the end effector, the developed robot is able to provide a general solution to the autonomous sanding on many other different objects.

A method to improve quality of life of elderly people is to increase muscle strength. The purpose of this study is to develop a machine to improve muscle strength effectively. In this paper, we proposed a muscle strength training machine that exerts variable resistance using pneumatic artificial rubber muscle. In addition, we observed the effect on muscle activity by using the proposed machine with predictable and unpredictable load change patterns. Experimental results showed that there was a difference between the rising time of muscle activity and the maximum amplitude of electromyogram depending on predictability of the load change and showed the possibility of effective muscle strength improvement against unpredictable load using the arm curl machine.