IROS 2018 Papers

Traditional Simultaneous Localization and Mapping (SLAM) approaches build maps based on points, lines or planes. These maps visually resemble the environment but without any semantic or information about the objects in the environment. Recent advancements in machine learning have made object detection highly accurate and reliable with large set of objects. Object detection can effectively help SLAM to incorporate semantics in the mapping process. One of the main obstacles is data association between detected objects over time. We demonstrate a nonparametric statistical approach to solve the data association between detected objects over consecutive frames. Then we use an unsupervised clustering method to identify the existence of objects in the map. The complete process can be run in parallel with SLAM. The performance of our algorithm is demonstrated on several public datasets, which shows promising results in locating objects in SLAM.

Progress in soft robotics depends on the integration of electronics for sensing, power regulation, and signal processing. Commercially available microelectronics satisfy these functions and are small enough to preserve the natural mechanics of the host system. Here, we present a method for incorporating microelectronic sensors and integrated circuits (ICs) into the elastomeric skin of a soft robot. The thin stretchable skin contains various solid-state electronics for orientation, pressure, proximity, and temperature sensing, and a microprocessor. The components are connected by thin-film copper traces wetted with eutectic gallium indium (EGaIn), a room temperature liquid metal alloy that allows the circuit to maintain conductivity as it deforms under mechanical loading. In this paper, we characterize the function of the individual sensors in air and water, discuss the integration of the microelectronic skin with a shape-memory actuated soft gripper, and demonstrate the sensorized soft gripper in conjunction with a 4 degree-of-freedom (DOF) robot arm.

The proximity between the multirotor blades and the environmental obstacles restricts the application of aerial manipulators in inspection tasks due to the risk of impacts, the limitation in the reach of the arm, and the physical interactions. This paper presents a long reach aerial manipulator consisting of a hexarotor platform equipped with a 2-DOF compliant joint arm attached at the tip of a one-meter-length link in passive pendulum configuration. The arm integrates magnetic encoders for force/torque estimation-control based on joint deflection, a range sensor in the forearm link for measuring the distance to the contact point, and a camera for visual inspection. A 2-DOF wearable exoskeleton interface has been developed, allowing the teleoperation of the arm with visual feedback in a more intuitive way. The paper also covers the kinematics and dynamics of the aerial manipulator, including the dynamics of the flexible long reach link. The developed system has been evaluated in test-bench and in outdoor flight tests.

One of the safest and most reliable strategies for vehicle's collision avoidance is embedded control at low level to guarantee safe motion in all situations using on-board sensors. In this paper, we propose a novel lightweight collision avoidance strategy that can be implemented as a low level motion control to achieve safe motion while simultaneously tracking the robot's reference control input. This strategy is designed to be general so that it can be easily integrated with most control designs, with the primary target of resource-constrained robot swarms that act in real-time, dynamic environments. The main advantages of our approach are a very simple structure and low computational requirements. We verified the effectiveness of the proposed collision avoidance strategy through two simulated scenarios and with physical robots. We believe our design can be directly used in many areas, such as autonomous driving, intelligent transportation and planetary exploration.

Most of the current solutions for autonomous flights in indoor environments rely on purely geometric maps (e.g., point clouds). There has been, however, a growing interest in supplementing such maps with semantic information (e.g., object detections) using computer vision algorithms. Unfortunately, there is a disconnect between the relatively heavy computational requirements of these computer vision solutions, and the limited computation capacity available on mobile autonomous platforms. In this paper, we propose to bridge this gap with a novel Markov Decision Process framework that adapts the parameters of the vision algorithms to the incoming video data rather than fixing them a priori. As a concrete example, we test our framework on a object detection and tracking task, showing significant benefits in terms of energy consumption without considerable loss in accuracy, using a combination of publicly available and novel datasets.

Reliable object perception is a vital requirement for automated driving. Despite the availability of precise contour measurements, most state-of-the-art tracking systems still represent object geometry as bounding boxes. However, there are objects operating in public traffic for which the box assumption is highly inappropriate. We therefore propose to represent object contours using 2D polylines. Taking into account the mutual dependence of object poses and shape, our tracking framework targets at a simultaneous estimation of both states. Moreover, we propose to augment scan segments with free-space information at their boundaries and show how this knowledge can be incorporated into the tracking framework and beyond. Evaluation with real scan data shows that our method produces accurate dynamic estimates and consistent shape reconstructions.

This paper proposes a targetless and automatic camera-LiDAR calibration method. Our approach extends the hand-eye calibration framework to 2D-3D calibration. The scaled camera motions are accurately calculated using a sensor-fusion odometry method. We also clarify the suitable motions for our calibration method. Whereas other calibrations require the LiDAR reflectance data and an initial extrinsic parameter, the proposed method requires only the three-dimensional point cloud and the camera image. The effectiveness of the method is demonstrated in experiments using several sensor configurations in indoor and outdoor scenes. Our method achieved higher accuracy than comparable state-of-the-art methods.

Motion planning through optimization is largely based on locally improving the cost of a trajectory until an optimal solution is found. Choosing the initial trajectory has therefore a significant effect on the performance of the motion planner, especially when the cost landscape contains local minima. While multiple heuristics and approximations may be used to efficiently compute an initialization online, they are based on generic assumptions that do not always match the task at hand. In this paper, we exploit the fact that repeated tasks are similar according to some metric. We store solutions of the problem as a library of initial seed trajectories offline and employ a problem encoding to retrieve near-optimal warm-start initializations on-the-fly. We compare how different initialization strategies affect the global convergence and runtime of quasi-Newton and probabilistic inference solvers. Our analysis on the 38-DoF NASA Valkyrie robot shows that efficient and optimal planning in high-dimensional state spaces is possible despite the presence of globally non-smooth and discontinuous constraints, such as the ones imposed by collisions.

We propose a head pose estimation framework that leverages on a recent keypoint detection model. More specifically, we apply the convolutional pose machines (CPMs) to input images, extract different types of facial keypoint features capturing appearance information and keypoint relationships, and train multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) for head pose estimation. The benefit of leveraging on the CPMs (which we apply anyway for other purposes like tracking) is that we can design highly efficient models for practical usage. We evaluate our approach on the Annotated Facial Landmarks in the Wild (AFLW) dataset and achieve competitive results with the state-of-the-art.

The design process for leg morphology has taken much of its inspiration from the manipulator community, including the concept of maximizing the workspace of a design. In this paper, we define the concept of Effective Dynamic Workspace, which examines the subset of the overall workspace capable of achieving the desired template dynamics. With this new design tool, the leg configuration of a new multi-modal platform BOBCAT is examined and refined. With the refined design, BOBCAT is able to achieve speeds of 2m/s while running and 0.17m/s while climbing a vertical wall.

Lower exoskeleton has gained considerable interests in walking assistance applications for both paraplegia and hemiplegia patients. In walking assistance of hemiplegia patients, the exoskeleton should have the ability to control the affected leg to follow the unaffected leg's motion naturally. One critical issue of walking assistance for hemiplegia patients is how to adapt the controller of both lower limbs with different patients. This paper presents a novel learning-based walking assistance control strategy for lower exoskeleton with hemiplegia patients. In the proposed control strategy, we modeled the control system of lower exoskeleton with hemiplegia patient as a Leader-Follower Multi-Agent System (LF -MAS). In order to adapt different patients with different conditions, reinforcement learning framework is utilized to adapt controllers online. In reinforcement learning framework with LF-MAS, we employed a Policy Iteration Adaptive Dynamic Programming (PI-ADP) algorithm, which aims to achieve better tracking control performance for lower exoskeleton with hemiplegia patient. We demonstrate the efficiency of proposed learning-based walking assistance control strategy in an exoskeleton system with healthy subjects who simulate hemiplegia patients. Experimental results indicate that the proposed control strategy can adapt different pilots with good tracking performance.

Flapping-wing micro air vehicles (FWMAVs) become promising research platforms due to their advantages such as various maneuverability, and concealment. However, unsteady flow at low Reynolds number around the wings makes their dynamics time-varying and highly non-linear. It makes autonomous flight of FWMAV as a big challenge. In this paper, we suggest a model-based control strategy for FWMAV using learning architecture. For this task, we construct a ground station for logging flight data and control inputs, and train dynamics with a neural network. Then, we apply model predictive control (MPC) to the trained model. We validate our method by hardware experiments.

Assessing grasp stability is essential to prevent the failure of robotic manipulation tasks due to sensory data and object uncertainties. Learning-based approaches are widely deployed to infer the success of a grasp. Typically, the underlying model used to estimate the grasp stability is trained for a specific task, such as lifting, hand-over, or pouring. Since every task has individual stability demands, it is important to adapt the trained model to new manipulation actions. If the same trained model is directly applied to a new task, unnecessary grasp adaptations might be triggered, or in the worst case, the manipulation might fail. To address this issue, we divide the manipulation task used for training into seven sub-tasks, defined as modular tasks. We deploy a learning-based approach and assess the stability for each modular task separately. We further propose analytical features to reduce the dimensionality and the redundancy of the tactile sensor readings. A main task can thereby be represented as a sequence of relevant modular tasks. The stability prediction of the main task is computed based on the inferred success labels of the modular tasks. Our experimental evaluation shows that the proposed feature set lowers the prediction error up to 5.69% compared to other sets used in state-of-the-art methods. Robotic experiments demonstrate that our modular task-oriented stability assessment avoids unnecessary grasp force adaptations and regrasps for various manipulation tasks.

Learning complex behaviors through reinforcement learning is particularly challenging when reward is only available upon successful completion of the full behavior. In manipulation robotics, so-called shaping rewards are often used to overcome this problem. However, these usually require human engineering or (partial)world models describing, e.g., the kinematics of the robot or high-level modules for perception. Here we propose an alternative method to learn an object palm-touching task through a weakly-supervised and stagewise learning of simpler tasks. First, the robot learns to fixate the object with its cameras. Second, the robot learns eye-hand coordination by learning to fixate its end effector. Third, using the previously acquired skills an informative shaping reward can be computed which facilitates efficient learning of the object palm-touching task. We demonstrate in simulation that learning the full task with this shaping reward is comparable to learning with an informative supervised reward.

Pouring is a fundamental skill for robots in both domestic and industrial environments. Ideally, a robot should be able to pour with high accuracy to specific, pre-defined heights and without spilling. However, due to the complex dynamics of liquids, it is difficult to learn how to pour to achieve these goals. In this paper we present an approach to learn a policy for pouring using Deep Deterministic Policy Gradients (DDPG). We remove the need for collecting training experiences on a real robot, by using a state-of-the-art liquid simulator, which allows for learning the liquid dynamics. We show through our experiments, performed with a PR2 robot, that it is possible to successfully transfer the learned policy to a real robot and even apply it to different liquids.

We present a robot model of early reach and grasp learning, inspired by infant learning without prior knowledge of the geometry, kinematics, or dynamics of the arm. Human infants at reach onset are capable of using a sequence of jerky submotions to bring the hand to the position of a nearby object. A robotic learning agent can produce qualitatively similar behavior by using a graph representation to encode a set of safe, potentially useful arm states and feasible moves between them. These observations show that the Peri-Personal Space (PPS) Graph model is sufficient for early reaching and suggest that infants may use analogous models during this phase. In this paper, we show that the PPS Graph, with a simulated Palmar reflex (a reflex in infants that closes the fingers when the palm is touched), allows accidental grasps to occur during continued reaching practice. Given these occasional events, the agent can bootstrap to a simple deliberate grasp action. In particular, the agent must learn three new necessary conditions for a grasp: the hand should be open as the grasp begins, the final motion of the hand should be led by the gripper opening so that it reaches the target first, and the wrist must be oriented such that the gripper fingers may close around the target object, often requiring the opening to be perpendicular to the object's major axis. Combined with the existing capability to reach and interact with target objects, knowledge of these conditions allows the agent to learn increasingly reliable purposeful grasps. The first two conditions are addressed in this paper, and allow 45% of grasps to succeed. This work contributes toward the larger goal of foundational robot learning after the model of infant learning, with minimal prior knowledge of its own anatomy or its environment. The ability to grasp will allow the agent to control the motion and position of objects, providing a richer representation for its environment and new experiences to learn from. Show More

Nowadays, Unmanned Aerial Vehicles (UAVs)are becoming increasingly popular facilitated by their extensive availability. Autonomous navigation methods can act as an enabler for the safe deployment of drones on a wide range of real-world civilian applications. In this work, we introduce a self-supervised CNN-based approach for indoor robot navigation. Our method addresses the problem of real-time obstacle avoidance, by employing a regression CNN that predicts the agent's distance-to-collision in view of the raw visual input of its on-board monocular camera. The proposed CNN is trained on our custom indoor-flight dataset which is collected and annotated with real-distance labels, in a self-supervised manner using external sensors mounted on an UAV. By simultaneously processing the current and previous input frame, the proposed CNN extracts spatio-temporal features that encapsulate both static appearance and motion information to estimate the robot's distance to its closest obstacle towards multiple directions. These predictions are used to modulate the yaw and linear velocity of the UAV, in order to navigate autonomously and avoid collisions. Experimental evaluation demonstrates that the proposed approach learns a navigation policy that achieves high accuracy on real-world indoor flights, outperforming previously proposed methods from the literature.

Recent physics-based models of concentric tube continuum robots are able to describe pose of the tip, given the preformed translation and rotation in joint space of the robot. However, such model-based approaches are associated with high computational load and highly non-linear modeling effort. A data-driven approach for computationally fast estimation of the kinematics without requiring the knowledge and the uncertainties in the physics-based model would be an asset. This paper introduces an approach to solve the forward kinematics as well as the inverse kinematics of concentric tube continuum robots with 6-DOF in three dimensional space SE(3). Two artificial neural networks with ReLU (rectified linear unit) activation functions are designed in order to approximate the respective kinematics. Measured data from a robot prototype are used in order to train, validate, and test the proposed approach. We introduce a representation of the rotatory joints by trigonometric functions that improves the accuracy of the approximation. The results with experimental measurements show higher accuracy for the forward kinematics compared to the state of the art mechanics modeling. The tip error is less then 2.3 mm w.r.t. position (1 % of total robot length) and 1.1° w.r.t. orientation. The single artificial neural network for the inverse kinematics approximation achieves a translation and rotation actuator error of 4.0 mm and 8.3 0, respectively.

This paper presents, for the first time, a method for learning in-contact tasks from a teleoperated demonstration with a hydraulic manipulator. Due to the use of extremely powerful hydraulic manipulator, a force-reflected bilateral teleoperation is the most reasonable method of giving a human demonstration. An advanced subsystem-dynamic-based control design framework, virtual decomposition control (VDC), is used to design a stability-guaranteed controller for the teleoperation system, while taking into account the full nonlinear dynamics of the master and slave manipulators. The use of fragile force/torque sensor at the tip of the hydraulic slave manipulator is avoided by estimating the contact forces from the manipulator actuators' chamber pressures. In the proposed learning method, it is observed that a surface-sliding tool has a friction-dependent range of directions (between the actual direction of motion and the contact force) from which the manipulator can apply force to produce the sliding motion. By this intuition, an intersection of these ranges can be taken over a motion to robustly find a desired direction for the motion from one or more demonstrations. The compliant axes required to reproduce the motion can be found by assuming that all motions outside the desired direction is caused by the environment, signalling the need for compliance. Finally, the learning method is incorporated to a novel VDC-based impedance control method to learn compliant behaviour from teleoperated human demonstrations. Experiments with 2-DOF hydraulic manipulator with a 475kg payload demonstrate the suitability and effectiveness of the proposed method to perform learning from demonstration (LfD) with heavy-duty hydraulic manipulators.