ICRA 2019 Papers

Soft robots undergo large nonlinear spatial deformations due to both inherent actuation and external loading. The physics underlying these deformations is complex, and often requires intricate analytical and numerical models. The complexity of these models may render traditional model-based control difficult and unsuitable. Model-free methods offer an alternative for analyzing the behavior of such complex systems without the need for elaborate modeling techniques. In this paper, we present a model-free approach for open loop position control of a soft spatial continuum arm, based on deep reinforcement learning. The continuum arm is pneumatically actuated and attains a spatial work-space by a combination of unidirectional bending and bidirectional torsional deformation. We use Deep-Q Learning with experience replay to train the system in simulation. The efficacy and robustness of the control policy obtained from the system is validated both in simulation and on the continuum arm prototype for varying external loading conditions.

One of the key challenges in realizing a robot that is capable of completing a variety of manipulation tasks in the real world is the need to utilize sufficiently compact and rich world models. If the assumed prediction model does not match real observations, planning systems are unable to perform properly. We propose a system that corrects the models based on information collected from the robot's sensors. We encode prior experiences in a neural network to generate possible parameters of the models for a physics engine from real observations. An online POMDP solver is used to plan actions to complete the task while progressively validating and improving the models. We perform experiments in simulations and on a real robot. The results show that this approach appropriately clarifies observed environments, can handle dynamics with discontinuities, and with increasing domain complexity achieves a better success rate than baseline methods.

The compliant motion of humanoid robots is one of their most important characteristics for interacting with humans and various environments in the real world. During walking, compliant motion ensures stable contact between the foot and ground, but walking stability is degraded by position tracking performance and unknown disturbances. To address the issue of instability of humanoid robot walking with compliant motion control, this paper proposes a model for real-time walking pattern generation considering the motion control performance of a robot. The dynamic model of a robot with a motion controller is described as a second-order system approximating position tracking performance with a linear inverted pendulum model to determine the relationship between the zero-moment point and center of mass (CoM). The CoM trajectory is calculated using preview control based on the dynamics model and current state of the robot. Therefore, even if the robot has the low tracking performance due to compliant motion control, the walking stability can be ensured. The proposed method was implemented on our humanoid robot, DYROS-JET, and its performance was demonstrated through improved stability during walking.

In this paper, we present an online two-level vehicle trajectory prediction framework for urban autonomous driving where there are complex contextual factors, such as lane geometries, road constructions, traffic regulations and moving agents. Our method combines high-level policy anticipation with low-level context reasoning. We leverage a long short-term memory (LSTM) network to anticipate the vehicle's driving policy (e.g., forward, yield, turn left, turn right, etc.) using its sequential history observations. The policy is then used to guide a low-level optimization-based context reasoning process. We show that it is essential to incorporate the prior policy anticipation due to the multimodal nature of the future trajectory. Moreover, contrary to existing regression-based trajectory prediction methods, our optimization-based reasoning process can cope with complex contextual factors. The final output of the two-level reasoning process is a continuous trajectory that automatically adapts to different traffic configurations and accurately predicts future vehicle motions. The performance of the proposed framework is analyzed and validated in an emerging autonomous driving simulation platform (CARLA).

This paper considers the problem of online optimal trajectory design under time-varying environments. Of particular interest is the design of energy-efficient trajectories under strong and uncertain disturbances in ocean environments and time-varying goal location. We formulate the problem within the constrained online convex optimization formalism, and a modified online gradient descent algorithm is motivated. The mobility constraints are met using a carefully chosen stepsize, and the proposed algorithm is shown to incur sublinear regret. Different from the state-of-the-art algorithms that entail planning and re-planning the full trajectory using forecast data at each time instant, the proposed algorithm is entirely online and relies mostly on the current ocean velocity measurements at the vehicle locations. The trade-off between excess delay incurred in reaching the goal and the overall energy consumption is examined via numerical tests carried out on real data obtained from the regional ocean modelling system. As compared to the state-of-the-art algorithms, the proposed algorithm is not only energy-efficient but also several orders of magnitude computationally efficient.

The problem of finding and grasping a target object in a cluttered, uncertain environment, target object search, is a common and important problem in robotics. One key challenge is the uncertainty of locating and recognizing each object in a cluttered environment due to noisy perception and occlusions. Furthermore, the uncertainty in localization makes manipulation difficult and uncertain. To cope with these challenges, we formulate the target object search task as a partially observable Markov decision process (POMDP), enabling the robot to reason about perceptual and manipulation uncertainty while searching. To further address the manipulation difficulty, we propose Parameterized Action Partially Observable Monte-Carlo Planning (PA-POMCP), an algorithm that evaluates manipulation actions by taking into account the effect of the robot's current belief on the success of the action execution. In addition, a novel run-time initial belief generator and a state value estimator are introduced in this paper to facilitate the PA-POMCP algorithm. Our experiments show that our methods solve the target object search task in settings where simpler methods either take more object movements or fail.

This paper addresses the problem of multi-robot coordination in scenarios where the robots may experience unexpected delays in their movements. Prior work by Čáp, Gregoire, and Frazzołi introduced a control law, called RMTRACK, which enables robots in such scenarios to execute preplanned paths in spite of disturbances in the execution speed of each robot, while guaranteeing that each robot can reach its goal without collisions and without deadlocks. We extend that approach to handle scenarios in which the disturbance probabilities are unknown at the start and non-uniform across the environment. The key idea is to `repair' a plan on-the-fly, by swapping the order in which a pair of robots passes through a mutual collision region (i.e. a coordination space obstacle), when making such a change can be estimated to improve the overall performance of the system. We introduce a technique based on Gaussian Processes to estimate future disturbances, and propose two algorithms for testing, at appropriate times, whether a swap of a given obstacle would be beneficial. Tests in simulation demonstrate that our algorithm achieves significantly smaller average travel time than RMTRACK at only a modest computational expense.

This work describes the development of a robotic system that acquires knowledge incrementally through human interaction where new objects and motions are taught on the fly. The robotic system developed was one of the five finalists in the KUKA Innovation Award competition and demonstrated during the Hanover Messe 2018 in Germany. The main contributions of the system are i) a novel incremental object learning module - a deep learning based localization and recognition system - that allows a human to teach new objects to the robot, ii) an intuitive user interface for specifying 3D motion task associated with the new object, and iii) a hybrid force-vision control module for performing compliant motion on an unstructured surface. This paper describes the implementation and integration of the main modules of the system and summarizes the lessons learned from the competition.

Underwater robots are subject to complex hydro-dynamic forces. These forces define how the vehicle moves, so it is important to consider them when planning trajectories. However, performing motion planning considering the dynamics on the robot’s onboard computer is challenging due to the limited computational resources available. In this paper an efficient motion planning framework for autonomous underwater vehicles (AUVs) is presented. By introducing a loosely coupled multilayered planning design, our framework is able to generate dynamically feasible trajectories while keeping the planning time low enough for online planning. First, a fast path planner operating in a lower-dimensional projected space computes a lead path from the start to the goal configuration. Then, the lead path is used to bias the sampling of a second motion planner, which takes into account all the dynamic constraints. Furthermore, we propose a strategy for online planning that saves computational resources by generating the final trajectory only up to a finite horizon. By using the finite horizon strategy together with the multilayered approach, the sampling of the second planner focuses on regions where good quality solutions are more likely to be found, significantly reducing the planning time. To provide strong safety guarantees our framework also incorporates the conservative approximations of inevitable collision states (icss). finally, we present simulations and experiments using a real underwater robot to demonstrate the capabilities of our framework.

Avoiding obstacles poses a significant challenge for amputees using mechanically-passive transfemoral prosthetic limbs due to their lack of direct knee control. In contrast, powered prostheses can potentially improve obstacle avoidance via their ability to add energy to the system. In past work, researchers have proposed stumble recovery systems for powered prosthetic limbs that provide assistance in the event of a trip. However, these systems only aid recovery after an obstacle has disrupted the user's gait and do not proactively help the amputee avoid obstacles. To address this problem, we designed an adaptive system that learns online to use kinematic data from the prosthetic limb to detect the user's obstacle avoidance intent in early swing. When the system detects an obstacle, it alters the planned swing trajectory to help avoid trips. Additionally, the system uses a regression model to predict the required knee flexion angle for the trip response. We validated the system by comparing obstacle avoidance success rates with and without the obstacle avoidance system. For a non-amputee subject wearing the prosthesis through an adapter, the trip avoidance system improved the obstacle negotiation success rate from 37% to 89%, while an amputee subject improved his success rate from 35% to 71% when compared to utilizing minimum jerk trajectories for the knee and ankle joints.

Underwater robots are subject to position drift due to the effect of ocean currents and the lack of accurate localisation while submerged. We are interested in exploiting such position drift to estimate the ocean current in the surrounding area, thereby assisting navigation and planning. We present a Gaussian process (GP)-based expectation-maximisation (EM) algorithm that estimates the underlying ocean current using sparse GPS data obtained on the surface and dead-reckoned position estimates. We first develop a specialised GP regression scheme that exploits the incompressibility of ocean currents to counteract the underdetermined nature of the problem. We then use the proposed regression scheme in an EM algorithm that estimates the best-fitting ocean current in between each GPS fix. The proposed algorithm is validated in simulation and on a real dataset, and is shown to be capable of reconstructing the underlying ocean current field. We expect to use this algorithm to close the loop between planning and estimation for underwater navigation in unknown ocean currents.

Accurate knowledge of geometric and inertia parameters are a necessity for precise and robust control of aerial vehicles. We propose a novel filter that is able to fuse motor speed, inertia, and pose measurements to estimate the vehicle's key dynamic properties online. The presented framework is able to estimate the multirotor's moment of inertia, mass, center of mass and each sensor module's relative position. Obtaining these estimates in-flight allow the multirotor to be precisely controlled even during tasks such as load transportation or after configuration changes on scene. We provide a nonlinear observability analysis, proving that the presented model is locally weakly observable. Experimental results validate the proposed approach, showing the ability to estimate the dynamic properties accurately and demonstrate its capability to do so even while additional loads are added. The framework is flexible and can easily be adapted to a wide range of applications, including self-calibration, object grasping, and single robot or multi-robot payload transportation.

This work presents an online learning-based control method for improved trajectory tracking of unmanned aerial vehicles using both deep learning and expert knowledge. The proposed method does not require the exact model of the system to be controlled, and it is robust against variations in system dynamics as well as operational uncertainties. The learning is divided into two phases: offline (pre-)training and online (post-)training. In the former, a conventional controller performs a set of trajectories and, based on the input-output dataset, the deep neural network (DNN)-based controller is trained. In the latter, the trained DNN, which mimics the conventional controller, controls the system. Unlike the existing papers in the literature, the network is still being trained for different sets of trajectories which are not used in the training phase of DNN. Thanks to the rule-base, which contains the expert knowledge, the proposed framework learns the system dynamics and operational uncertainties in real-time. The experimental results show that the proposed online learning-based approach gives better trajectory tracking performance when compared to the only offline trained network.

The representation of the environment strongly affects how robots can move and interact with it. This paper presents an online approach for continuous mapping using Gaussian Process Implicit Surfaces (GPISs). Compared with grid-based methods, GPIS better utilizes sparse measurements to represent the world seamlessly. It provides direct access to the signed-distance function (SDF) and its derivatives which are invaluable for other robotic tasks and it incorporates uncertainty in the sensor measurements. Our approach incrementally and efficiently updates GPIS by employing a regressor on observations and a spatial tree structure. The effectiveness of the suggested approach is demonstrated using simulations and real world 2D/3D data.

In this paper, we study the NP-Hard problem of multi-robot coalition formation for task allocation. To tackle this notoriously difficult problem, we model it as a variant of classical bipartite matching, which we call One-To-Many Bipartite Matching (OTMaM). Unlike the classical bipartite matching techniques used for matching a unique robot to a unique task, in the OTMaM problem, we let multiple robots to be matched to a single task while restricting the opposite. To this end, we propose a novel heuristic algorithm that allocates robots to tasks by finding mutually best robot-task pairs. Our algorithm provides a similar theoretical worst-case approximation ratio and guarantees a better worst-case time complexity than a comparable algorithm from the literature. The proposed approach in this paper is proved to be deterministic and the resultant matching is perfect. Simulation results also demonstrate the scalability of the presented algorithm (taking less than 1 millisecond with 100 robots and 10 tasks).

Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots. Demonstrations are inherently ambiguous and incomplete, making correct generalization to unseen situations difficult without a large number of demonstrations in varying conditions. By contrast, humans are often able to learn complex tasks from a single demonstration (typically observations without action labels) by leveraging context learned over a lifetime. Inspired by this capability, our goal is to enable robots to perform one-shot learning of multi-step tasks from observation by leveraging auxiliary video data as context. Our primary contribution is a novel system that achieves this goal by: (1) using a single user-segmented demonstration to define the primitive actions that comprise a task, (2) localizing additional examples of these actions in unsegmented auxiliary videos via a metalearning-based approach, (3) using these additional examples to learn a reward function for each action, and (4) performing reinforcement learning on top of the inferred reward functions to learn action policies that can be combined to accomplish the task. We empirically demonstrate that a robot can learn multi-step tasks more effectively when provided auxiliary video, and that performance greatly improves when localizing individual actions, compared to learning from unsegmented videos.

In this paper, we present an on-line active pose-graph simultaneous localization and mapping (SLAM) frame-work for robots in three-dimensional (3D) environments using graph topology and sub-maps. This framework aims to find the best trajectory for loop-closure by re-visiting old poses based on the T-optimality and D-optimality metrics of the Fisher information matrix (FIM) in pose-graph SLAM. In order to reduce computational complexity, graph topologies are introduced, including weighted node degree (T-optimality metric) and weighted tree-connectivity (D-optimality metric), to choose a candidate trajectory and several key poses. With the help of the key poses, a sampling-based path planning method and a continuous-time trajectory optimization method are combined hierarchically and applied in the whole framework. So as to further improve the real-time capability of the method, the sub-map joining method is used in the estimation and planning process for large-scale active SLAM problems. In simulations and experiments, we validate our approach by comparing against existing methods, and we demonstrate the on-line planning part using a quad-rotor unmanned aerial vehicle (UAV).

Robotic teleoperation enables humans to safely complete exploratory procedures in remote locations for applications such as deep sea exploration or building assessments following natural disasters. Successful task completion requires meaningful dual arm robotic coordination and proper understanding of the environment. While these capabilities are inherent to humans via impedance regulation and haptic interactions, they can be challenging to achieve in telerobotic systems. Teleimpedance control has allowed impedance regulation in such applications, and bilateral teleoperation systems aim to restore haptic sensation to the operator, though often at the expense of stability or workspace size. Wearable haptic devices have the potential to apprise the operator of key forces during task completion while maintaining stability and transparency. In this paper, we evaluate the impact of wearable haptics for force feedback in teleimpedance control for dual-arm robotic teleoperation. Participants completed a peg-in-hole, box placement task, aiming to seat as many boxes as possible within the trial period. Experiments were conducted both transparent and opaque boxes. With the opaque box, participants achieved a higher number of successful placements with haptic feedback, and we saw higher mean interaction forces. Results suggest that the provision of wearable haptic feedback may increase confidence when visual cues are obscured.

We consider the problem of planning paths on graphs with some edges whose traversability is uncertain; for each uncertain edge, we are given a probability of being traversable (e.g., by a learned classifier). We categorize different interpretations of the problem that are meaningful for mobile robots navigating partially-known environments, each of which yields a different formalization; we then focus on the case in which the true traversability of an edge is revealed only when the agent visits one of its endpoints (Canadian Traveller Problem). In this context, we design a large simulation campaign on synthetic and real-world maps to study the impact of two different factors: the planning strategy, and the amount of uncertainty (which could depend on the quality of the classifier producing traversability estimates).