ICRA 2020 Papers

We address goal-based imitation learning, where the aim is to output the symbolic goal from a third-person video demonstration. This enables the robot to plan for execution and reproduce the same goal in a completely different environment. The key challenge is that the goal of a video demonstration is often ambiguous at the level of semantic actions. The human demonstrators might unintentionally achieve certain subgoals in the demonstrations with their actions. Our main contribution is to propose a motion reasoning framework that combines task and motion planning to disambiguate the true intention of the demonstrator in the video demonstration. This allows us to recognize the goals that cannot be disambiguated by previous action-based approaches. We evaluate our approach on a new dataset of 96 video demonstrations in a mockup kitchen environment. We show that our motion reasoning plays an important role in recognizing the actual goal of the demonstrator and improves the success rate by over 20%. We further show that by using the automatically inferred goal from the video demonstration, our robot is able to reproduce the same task in a real kitchen environment.

This paper presents a novel path planning strategy for fast and agile exploration using aerial robots. Tailored to the combined need for large-scale exploration of challenging and confined environments, despite the limited endurance of micro aerial vehicles, the proposed planner employs motion primitives to identify admissible paths that search the configuration space, while exploiting the dynamic flight properties of small aerial robots. Utilizing a computationally efficient volumetric representation of the environment, the planner provides fast collision-free and future-safe paths that maximize the expected exploration gain and ensure continuous fast navigation through the unknown environment. The new method is field-verified in a set of deployments relating to subterranean exploration and specifically, in both modern and abandoned underground mines in Northern Nevada utilizing a 0.55m-wide collision-tolerant flying robot exploring with a speed of up to 2m/s and navigating sections with width as small as 0.8m.

Underactuated adaptive hands simplify grasping tasks but it is difficult to model their interactions with objects during in-hand manipulation. Learned data-driven models have been recently shown to be efficient in motion planning and control of such hands. Still, the accuracy of the models is limited even with the addition of more data. This becomes important for long horizon predictions, where errors are accumulated along the length of a path. Instead of throwing more data into learning the transition model, this work proposes to rather invest a portion of the training data in a critic model. The critic is trained to estimate the error of the transition model given a state and a sequence of future actions, along with information of past actions. The critic is used to reformulate the cost function of an asymptotically optimal motion planner. Given the critic, the planner directs planned paths to less erroneous regions in the state space. The approach is evaluated against standard motion planning on simulated and real hands. The results show that it outperforms an alternative where all the available data is used for training the transition model without a critic.

This paper presents a cooperative robotic team composed of unmanned ground vehicles (UGVs) with hybrid operational modes to tackle the multiple traveling salesman problem (mTSP) with obstacles. The hybrid operational modes allow every UGV in the team to not only travel on a ground surface but also jump over obstacles. We name these UGVs jumping rovers. The jumping capability provides a flexible form of locomotion by leaping and landing on top of obstacles instead of navigating around obstacles. To solve the mTSP, an optimal path between any two objective points in an mTSP is determined by the optimized rapidly-exploring random tree method, named RRT*, and is further improved through a refined RRT* algorithm to find a smoother path between targets. We then formulate the mTSP as a mixed-integer linear programming (MILP) problem to search for the most cost-effective combination of paths for multiple UGVs. The effectiveness of the hybrid operational modes and optimized motion with assigned tasks is verified in an indoor, physical experimental environment using the customized jumping rovers.

Surface electromyography contains muscular information representing gestures and corresponding forces. However, conventional sEMG-based motion recognition methods, such as pattern classification and regression, have intrinsic limitations due to the complex characteristics of sEMG signals. In this paper, motion intensity, a highly selective sEMG feature proportional to the level of muscle contraction, is proposed. The motion intensity feature allows proportional and simultaneous recognition of multiple degrees of freedom. The proposed method was demonstrated in terms of simultaneous recognition of wrist/hand motions. The result shows that the proposed method can successfully decompose sEMG signals into highly selective signals to target motions. In future works, the proposed method will be adapted for more subjects and to sEMG applications for practical evaluation considering various grasping motions.

In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent the scene in a bird's eye view, where each grid cell contains the occupancy prob-ability and the two dimensional velocity. As input data, our approach relies on measurement grid maps, which contain occupancy probabilities, generated with lidar measurements. Given this configuration, we propose a recurrent neural net-work architecture to predict a dynamic occupancy grid map, i.e. filtered occupancy and velocity of each cell, by using a sequence of measurement grid maps. Our network architecture contains convolutional long-short term memories in order to sequentially process the input, makes use of spatial context, and captures motion. In the evaluation, we quantify improvements in estimating the velocity of braking and turning vehicles compared to the state-of-the-art. Additionally, we demonstrate that our approach provides more consistent velocity estimates for dynamic objects, as well as, less erroneous velocity estimates in static area.

A method for designing the motion of a snake robot negotiating complicated pipe structures having a constant diameter is presented. For such robots moving inside pipes, there are various "obstacles" such as junctions, bends, shears, and blockages. To surmount these obstacles, we propose a method that enables the robot to adapt to multiple pipe structures of a constant diameter. We designed the target form of the snake robot of two helices connected with an arbitrary shape. This method is applicable to various obstacles by designing a part of the target form conforming to the obstacle. The robot negotiates obstacles under shift control by employing a rolling motion. We demonstrated the effectiveness of the proposed method in various experiments.

State of the art visual-inertial odometry approaches suffer from the requirement of high gradients and sufficient visual texture. Even direct photometric approaches select a subset of the image with high-gradient areas and ignore smooth gradients or generally low-textured areas. In this work, we show that taking all image information (i.e. every single pixel) enables visual-inertial odometry even on areas with very low texture and smooth gradients, inherently interpolating and estimating the scene with no texture based on its informative surrounding. This information propagation is only possible as we estimate all states and their uncertainties (robot pose, extrinsic sensor calibration, and scene depth) jointly in a fully dense filter framework. Our complexity reduction approach enables real-time execution despite the large size of the state vector. Compared to our previous basic feasibility study on this topic, this work includes higher order covariance propagation and improved state handling for a significant performance gain, thorough comparisons to state-of-the-art algorithms, larger mapping components with uncertainty, self-calibration capability, and real-data tests.

Robustness and accuracy for monocular visual odometry (VO) under challenging environments are widely concerned. In this paper, we present a monocular VO system leveraging learned repeatability and description. In a hybrid scheme, the camera pose is initially tracked on the predicted repeatability maps in a direct manner and then refined with the patch-wise 3D-2D association. The local feature parameterization and the adapted mapping module further boost different functionalities in the system. Extensive evaluations on challenging public datasets are performed. The competitive performance on camera pose estimation demonstrates the effectiveness of our method. Additional studies on the local reconstruction accuracy and running time exhibit that our system is capable of maintaining a robust and lightweight backend.

In this paper, we introduce an approach to tracking the pose of a monocular camera in a prior surfel map. By rendering vertex and normal maps from the prior surfel map, the global planar information for the sparse tracked points in the image frame is obtained. The tracked points with and without the global planar information involve both global and local constraints of frames to the system. Our approach formulates all constraints in the form of direct photometric errors within a local window of the frames. The final optimization utilizes these constraints to provide the accurate estimation of global 6-DoF camera poses with the absolute scale. The extensive simulation and real-world experiments demonstrate that our monocular method can provide accurate camera localization results under various conditions.

In this paper, we formulate and solve the intermittent deployment problem, which yields strategies that couple when heterogeneous robots should sense an environmental process, with where a deployed team should sense in the environment. As a motivation, suppose that a spatiotemporal process is slowly evolving and must be monitored by a multi-robot team, e.g., unmanned aerial vehicles monitoring pasturelands in a precision agriculture context. In such a case, an intermittent deployment strategy is necessary as persistent deployment or monitoring is not cost-efficient for a slowly evolving process. At the same time, the problem of where to sense once deployed must be solved as process observations yield useful feedback for determining effective future deployment and monitoring decisions. In this context, we model the environmental process to be monitored as a spatiotemporal Gaussian process with mutual information as a criterion to measure our understanding of the environment. To make the sensing resource-efficient, we demonstrate how to use matroid constraints to impose a diverse set of homogeneous and heterogeneous constraints. In addition, to reflect the cost-sensitive nature of real-world applications, we apply budgets on the cost of deployed heterogeneous robot teams. To solve the resulting problem, we exploit the theories of submodular optimization and matroids and present a greedy algorithm with bounds on sub-optimality. Finally, Monte Carlo simulations demonstrate the correctness of the proposed method.

Restoring healthy kinematics is a critical component of assisting and rehabilitating impaired locomotion. Here we tested whether spatiotemporal gait patterns can be modulated by applying mechanical impedance to hip joints. Using the Samsung GEMS-H exoskeleton, we emulated a virtual spring (positive and negative) between the user's legs. We found that applying positive stiffness with the exoskeleton decreased stride time and hip range of motion for healthy subjects during treadmill walking. Conversely, the application of negative stiffness increased stride time and hip range of motion. These effects did not vary over long nor short repeated exposures to applied stiffness. In addition, minimal transient behavior was observed in spatiotemporal measures of gait when the stiffness controller transitioned between on and off states. These results suggest that changes in gait behavior induced by applying hip stiffness were purely a mechanical effect. Together, our findings indicate that applying mechanical impedance using lower-limb assistive devices may be an effective, minimally-encumbering intervention to restore healthy gait patterns.

Soft robots apply compliant materials to perform motions and behaviors not typically achievable by rigid robots. An underwater, compliant, multi-segment continuum manipulator that can bend, swallow, disgorge is developed in this study. The manipulator is driven by McKibben water hydraulic artificial muscle (WHAM). The mechanical properties of the WHAM are tested and analyzed experimentally. The kinematics model, which concerns about the variable diameter structure of the soft grippers, are established to simulate the behaviors of the manipulator among the bending, swallowing and disgorging procedure. A mouth-tongue collaborative soft robot assembled with another single-segment soft robot arm is presented. And its functions are experimentally testified. The distinctive functions were verified according to the experimental results.

This paper proposes a muscle-like SMA (Shape Memory Alloy) actuator with an active cooling system for efficient response. An SMA coil spring is embedded into a stretchable coolant vascular for soften structure of robots. In order to design a flexible, lightweight, and fast-response soft actuator with the SMA coil spring and coolant circulation system, a modeling based approach has been conducted. Analysis of coolant effects has been conducted in aspects of heating speed, cooling speed, and energy consumption based on both theoretical and empirical studies. From thermomechanical and heat transfer model between SMA and coolant, the actuation times in the case of heating and cooling phase have been estimated. From experimental results, Mineral oil is selected as the optimal coolant, and the maximum actuation frequency was measured as 0.5Hz for 40% contraction lifting 1kg.

Electromagnetic Navigation Systems (eMNS) can be used to control a variety of multiscale devices within the human body for remote surgery. Accurate modeling of the magnetic fields generated by the electromagnets of an eMNS is crucial for the precise control of these devices. Existing methods assume a linear behavior of these systems, leading to significant modeling errors within nonlinear regions exhibited at higher magnetic fields, preventing these systems from operating at full capacity. In this paper, we use a random forest (RF) and an artificial neural network (ANN) to model the nonlinear behavior of the magnetic fields generated by an eMNS. Both machine learning methods outperformed the state-of-the-art linear multipole electromagnet model (MPEM). The RF and the ANN model reduced the root mean squared error (RMSE) of the MPEM when predicting the field magnitude by approximately 40% and 87%, respectively, over the entire current range of the eMNS. At high current regions, especially between 30 and 35 A, the field-magnitude RMSE improvement of the ANN model over the MPEM was 37 mT, equivalent to 90% error reduction. This study demonstrates the feasibility of using machine learning to model an eMNS for medical applications, and its ability to account for complex nonlinear behavior at high currents. The use of machine learning thus shows promise in developing accurate field predicting models, and ultimately improving surgical procedures that use magnetic navigation.

Carrying payloads in air is a major mission for multirotor aerial robot. However, the presence of payloads on multirotor aerial robot has a risk of degrading the performance of the flight controller. This concern becomes obvious especially when carrying objects not securely attached to the body or performing aerial manipulation. Therefore, controller with the ability to adapt itself to the effects of payloads on flight stability is needed. This paper proposes a novel nonlinear multiple-input and multiple-output (MIMO) model reference adaptive control (MRAC) system for attitude control of multirotor aerial robots which can dynamically compensate change in the position of center of gravity and inertia caused by payloads. Stability and robustness of the controller are experimentally confirmed in quadrotor and transformable multirotor, and experiments modeling practical applications are conducted for each aerial robot system, proving the utility of the controller.

Robots are more and more often designed in order to perform tasks in synergy with human operators. In this context, a current research focus for collaborative robotics lies in the design of high-performance control solutions, which ensure security in spite of unmodeled external forces. The present work provides a method based on Model Predictive Control (MPC) to allow compliant behavior when interacting with an environment, while respecting practical robotic constraints. The study shows in particular how to define the impedance control problem as a MPC problem. The approach is validated with an experimental setup including a collaborative robot. The obtained results emphasize the ability of this control strategy to solve constraints like speed, energy or jerk limits, which have a direct impact on the operator's security during human-robot compliant interactions.

Functional electrical stimulation (FES) is a promising technique for restoring reaching ability to individuals with tetraplegia. To this point, the complexities of goal-directed reaching motions and the shoulder-arm complex have prevented the realization of this potential in full-arm 3D reaching tasks. We trained a Gaussian process regression model to form the basis of a feedforward-feedback control structure capable of achieving reaching motions with a paralyzed upper limb. Over a series of 95 reaches of at least 10 cm in length, the controller achieved an average accuracy (measured by the Euclidean distance of the wrist to the final target position) of 3.8 cm and an average error along the path of 3.5 cm. This controller is the first demonstration of an accurate, complete-arm, FES-driven 3D reaching controller to be implemented with an individual with tetraplegia.

In this work we introduce ModQuad-DoF, a modular flying robotic structure with enhanced capabilities for yaw actuation. We propose a new module design that allows a one degree of freedom relative motion between the flying robot and the cage, with a docking mechanism allowing rigid connections between cages. A novel method of yaw actuation that increases the structure control authority is also presented. Our new method for the structure yaw control relies on the independent roll angles of each one of the modules, instead of the traditional drag moments from the propellers. In this paper, we propose a controller that allows the ModQuad-DoF to control its position and attitude. In our experiments, we tested a different number of modules flying in cooperation and validated the novel yaw actuation method.