ICRA 2020 Papers

Is it possible to learn policies for robotic assembly that can generalize to new objects? We explore this idea in the context of the kit assembly task. Since classic methods rely heavily on object pose estimation, they often struggle to generalize to new objects without 3D CAD models or task-specific training data. In this work, we propose to formulate the kit assembly task as a shape matching problem, where the goal is to learn a shape descriptor that establishes geometric correspondences between object surfaces and their target placement locations from visual input. This formulation enables the model to acquire a broader understanding of how shapes and surfaces fit together for assembly - allowing it to generalize to new objects and kits. To obtain training data for our model, we present a self-supervised data-collection pipeline that obtains ground truth object-to-placement correspondences by disassembling complete kits. Our resulting real-world system, Form2Fit, learns effective pick and place strategies for assembling objects into a variety of kits - achieving 90% average success rates under different initial conditions (e.g. varying object and kit poses), 94% success under new configurations of multiple kits, and over 86% success with completely new objects and kits. Code, videos, and supplemental material are available at https://form2fit.github.io.

Out in the field, bipedal robots need to travel on terrain that is uneven, non-rigid, and sometimes moving beneath their feet. We present a force-based double support balancing controller for such dynamic terrain scenarios for bipedal robots, and test it on the robotic bipedal platform "Tallahassee Cassie." The presented controller relies on minimal information about the robot model, requiring its kinematics and overall weight, but not inertias of individual links or components. The controller is pelvis-centric, commanding pelvis positions in Cartesian space, which a model-free PD controller converts to motor torques in joint space. By commanding forces, torques, and a frontal center of pressure in this fashion, Tallahassee Cassie is capable of balancing on a variety of scenarios, from a lifting/sliding platform, to soft foam, to a sudden drop. These results show the potential for bipedal control to balance successfully despite minimal model information, the presence of large dynamic impacts-e.g., falling through trap door, and soft series-spring deflections. These results motivate future work for walking and running controllers on dynamic terrain with relatively low reliance on modeling information.

In many tasks such as finishing operations, achieving accurate force tracking is essential. However, uncertainties in the robot dynamics and the environment limit the force tracking accuracy. Learning a compensation model for these uncertainties to reduce the force error is an effective approach to overcome this limitation. However, this approach requires an adaptive and robust framework for motion and force generation. In this paper, we use the time-invariant Dynamical System (DS) framework for force adaptation in contact tasks. We propose to improve force tracking accuracy through online adaptation of a state-dependent force correction model encoded with Radial Basis Functions (RBFs). We evaluate our method with a KUKA LWR IV+ robotic arm. We show its efficiency to reduce the force error to a negligible amount with different target forces and robot velocities. Furthermore, we study the effect of the hyper-parameters and provide a guideline for their selection. We showcase a collaborative cleaning task with a human by integrating our method to previous works to achieve force, motion, and task adaptation at the same time. Thereby, we highlight the benefits of using adaptive force control in real-world environments where we need reactive and adaptive behaviours in response to interactions with the environment.

In this paper, we present FootTile, a foot sensor for reaction force and center of pressure sensing in challenging terrain. We compare our sensor design to standard biomechanical devices, force plates and pressure plates. We show that FootTile can accurately estimate force and pressure distribution during legged locomotion. FootTile weighs 0.9 g, has a sampling rate of 330 Hz, a footprint of 10×10 mm and can easily be adapted in sensor range to the required load case. In three experiments, we validate: first, the performance of the individual sensor, second an array of FootTiles for center of pressure sensing and third the ground reaction force estimation during locomotion in granular substrate. We then go on to show the accurate sensing capabilities of the waterproof sensor in liquid mud, as a showcase for real world rough terrain use.

For robots using motion planning algorithms such as RRT and RRT*, the computational load can vary by orders of magnitude as the complexity of the local environment changes. To adaptively provide such computation, we propose Fog Robotics algorithms in which cloud-based serverless lambda computing provides parallel computation on demand. To use this parallelism, we propose novel motion planning algorithms that scale effectively with an increasing number of serverless computers. However, given that the allocation of computing is typically bounded by both monetary and time constraints, we show how prior learning can be used to efficiently allocate resources at runtime. We demonstrate the algorithms and application of learned parallel allocation in both simulation and with the Fetch commercial mobile manipulator using Amazon Lambda to complete a sequence of sporadically computationally intensive motion planning tasks.

We present a novel approach to increase the flight time of a multirotor via mid-air docking and in-flight battery switching. A main quadcopter flying using a primary battery has a docking platform attached to it. A `flying battery' - a small quadcopter carrying a secondary battery - is equipped with docking legs that can mate with the main quadcopter's platform. Connectors between the legs and the platform establish electrical contact on docking, and enable power transfer from the secondary battery to the main quadcopter. A custom-designed circuit allows arbitrary switching between the primary battery and secondary battery. We demonstrate the concept in a flight experiment involving repeated docking, battery switching, and undocking. This is shown in the video attachment. The experiment increases the flight time of the main quadcopter by a factor of 4.7× compared to solo flight, and 2.2× a theoretical limit for that given multirotor. Importantly, this increase in flight time is not associated with a large increase in overall vehicle mass or size, leaving the main quadcopter in fundamentally the same safety class.

In this paper, the high angular rate estimation for simultaneous localisation and mapping (SLAM) of a Flying Li-DAR (Flydar) is presented. The proposed EKF-based algorithm exploits the sinusoidal magnetometer measurement generated by the continuously rotating airframe for estimation of the robot hovering angular velocity. Significantly, the proposed method does not rely on additional sensors other than existing IMU sensors already being used for flight stabilization. The gyro measurement and the gyro bias are incorporated as a control input and a filter state respectively to enable estimation even under gyro saturation condition. Additionally, this work proposes leveraging on the inherently rotating locomotion to generate a planar lidar scan using only a single-point laser for possible lightweight autonomy. The proposed estimation method was experimentally evaluated on a ground rotating rig up to twice the gyro saturation limit with an effective rms error of 0.0045Hz; and on the proposed aerial platform - Flydar - hovering beyond the saturation limit with a rms error of 0.0056Hz. Lastly, the proposed method for SLAM using the rotating dynamics of Flydar was demonstrated with a localisation accuracy of 0.11m.

Many approaches have been proposed to estimate camera poses by directly minimizing photometric error. However, due to the non-convex property of direct alignment, proper initialization is still required for these methods. Many robust norms (e.g. Huber norm) have been proposed to deal with the outlier terms caused by incorrect initializations. These robust norms are solely defined on the magnitude of each error term. In this paper, we propose a novel robust norm, named FlowNorm, that exploits the information from both the local error term and the global image registration information. While the local information is defined on patch alignments, the global information is estimated using a learning-based network. Using both the local and global information, we achieve a large convergence range in which images can be aligned given large view angle changes or small overlaps. We further demonstrate the usability of the proposed robust norm by integrating it into the direct methods DSO and BA-Net, and generate more robust and accurate results in real-time.

Dynamic environments are challenging for visual SLAM since the moving objects occlude the static environment features and lead to wrong camera motion estimation. In this paper, we present a novel dense RGB-D SLAM solution that simultaneously accomplishes the dynamic/static segmentation and camera ego-motion estimation as well as the static background reconstructions. Our novelty is using optical flow residuals to highlight the dynamic semantics in the RGB-D point clouds and provide more accurate and efficient dynamic/static segmentation for camera tracking and background reconstruction. The dense reconstruction results on public datasets and real dynamic scenes indicate that the proposed approach achieved accurate and efficient performances in both dynamic and static environments compared to state-of-the-art approaches.

Biped robots require massive power on each leg while walking, hopping, and running. We have developed a flow-based control system-called hydraulic direct drive system- that can achieve high output while avoiding spatial limitations. To implement the proposed system with simple equipment configuration, a pump and single-rod cylinder are connected in a closed loop. However, because compensation for flow rate is impossible in a completely closed loop, owing to the difference in the pressure receiving area caused by the rod, a passive flow compensation valve is employed. This valve has a simple structure and is easy to implement. Further, an additional sensor is required to detect the open/close state because the valve state will cause an error in flow control. Therefore, we implemented a model in the controller to predict the state of the flow compensation valve and formulated a method of switching from flow control to pressure control according to the predicted state. Experimental results indicate that the error of the joint angle is reduced to less than 1.6 degrees for walking patterns, and stable walking is realized when the system is installed in biped humanoid robots.

The human thumb's state contribution to grasping and dexterous manipulation of objects is a function of the kinematic multiplicity of joints and structure of the bones, joints, and ligaments. This paper looks at the design and evaluation of a human-like thumb for use in a robotic hand, where the thumb's state contribution to grasping and dexterous manipulation is a function of a simplified kinematic model based on that of the human thumb, but also on empirical trials of surgical techniques to retain functionality while reducing the number of joints in the thumb. Motion Capture Data of the End Effector is analyzed with the measured excursion of the tendons to determine the relationship between tendon velocities and task-space velocities. After validating the procedure experimentally, a simplified metric is proposed to represent this data, and shows that our prototype is predicted to have a relatively smooth mapping between tendon excursion velocity and end effector velocity.

Flexible adaptation of learning strategy depending on online changes of the user's current intents have a high relevance in human-robot collaboration. In our previous study, we proposed an intrinsic interactive reinforcement learning approach for human-robot interaction, in which a robot learns his/her action strategy based on intrinsic human feedback that is generated in the human's brain as neural signature of the human's implicit evaluation of the robot's actions. Our approach has an inherent property that allows robots to adapt their behavior depending on online changes of the human's current intents. Such flexible adaptation is possible, since robot learning is updated in real time by human's online feedback. In this paper, the adaptivity of robot learning is tested on eight subjects who change their current control strategy by adding a new gesture to the previous used gestures. This paper evaluates the learning progress by analyzing learning phases (before and after adding a new gesture for control). The results show that the robot can adapt the previously learned policy depending on online changes of the user's intents. Especially, learning progress is interrelated with the classification performance of electroencephalograms (EEGs), which are used to measure the human's implicit evaluation of the robot's actions.

Fisheye cameras are commonly used in applications like autonomous driving and surveillance to provide a large field of view (> 180o). However, they come at the cost of strong non-linear distortions which require more complex algorithms. In this paper, we explore Euclidean distance estimation on fisheye cameras for automotive scenes. Obtaining accurate and dense depth supervision is difficult in practice, but self-supervised learning approaches show promising results and could potentially overcome the problem. We present a novel self-supervised scale-aware framework for learning Euclidean distance and ego-motion from raw monocular fisheye videos without applying rectification. While it is possible to perform piece-wise linear approximation of fisheye projection surface and apply standard rectilinear models, it has its own set of issues like re-sampling distortion and discontinuities in transition regions. To encourage further research in this area, we will release our dataset as part of the WoodScape project [1]. We further evaluated the proposed algorithm on the KITTI dataset and obtained state-of-the-art results comparable to other self-supervised monocular methods. Qualitative results on an unseen fisheye video demonstrate impressive performance1.

Research on advanced driver assistance systems for reducing risks to vulnerable road users (VRUs) has recently gained popularity because the traffic accident reduction rate for VRUs is still small. Dealing with unexpected VRU movements on residential roads requires proficient acceleration and deceleration. Although fine-grained prediction of driving behavior through inverse reinforcement learning (IRL) has been reported with promising results in recent years, learning of a precise model fails when driving strategies vary with contextual factors, i.e., weather, time of day, road width, and traffic direction. In this work, we propose a novel multi-task IRL approach with a multilinear reward function to incorporate contextual information into the model. This approach can provide precise long-term prediction of fine-grained driving behavior while adjusting to context. Experimental results using actual driving data over 141 km with various contexts and roads confirm the success of this approach in terms of predicting defensive driving strategy even in unknown situations.

Recently, Linear Temporal Logic (LTL) has been used as a formalism for defining high-level robot tasks, and LTL synthesis has been used to automatically create correct-by-construction robot control. The underlying premise of this approach is that the robot has a set of actions, or skills, that can be composed to achieve the high- level task. In this paper we consider LTL specifications that cannot be synthesized into robot control due to lack of appropriate skills; we present algorithms for automatically suggesting new or modified skills for the robot that will guarantee the task will be achieved. We demonstrate our approach with a physical Baxter robot and a simulated KUKA IIWA arm.

We present a method that finds locomanipulation plans that perform simultaneous locomotion and manipulation of objects for a desired end-effector trajectory. Key to our approach is to consider an injective locomotion constraint manifold that defines the locomotion scheme of the robot and then using this constraint manifold to search for admissible manipulation trajectories. The problem is formulated as a weighted-A* graph search whose planner output is a sequence of contact transitions and a path progression trajectory to construct the whole-body kinodynamic locomanipulation plan. We also provide a method for computing, visualizing, and learning the locomanipulability region, which is used to efficiently evaluate the edge transition feasibility during the graph search. Numerical simulations are performed with the NASA Valkyrie robot platform that utilizes a dynamic locomotion approach, called the divergent-component-of-motion (DCM), on two example locomanipulation scenarios.

We present a novel approach to control design for nonlinear systems which leverages model-free policy optimization techniques to learn a linearizing controller for a physical plant with unknown dynamics. Feedback linearization is a technique from nonlinear control which renders the input-output dynamics of a nonlinear plant linear under application of an appropriate feedback controller. Once a linearizing controller has been constructed, desired output trajectories for the nonlinear plant can be tracked using a variety of linear control techniques. However, the calculation of a linearizing controller requires a precise dynamics model for the system. As a result, model-based approaches for learning exact linearizing controllers generally require a simple, highly structured model of the system with easily identifiable parameters. In contrast, the model-free approach presented in this paper is able to approximate the linearizing controller for the plant using general function approximation architectures. Specifically, we formulate a continuous-time optimization problem over the parameters of a learned linearizing controller whose optima are the set of parameters which best linearize the plant. We derive conditions under which the learning problem is (strongly) convex and provide guarantees which ensure the true linearizing controller for the plant is recovered. We then discuss how model-free policy optimization algorithms can be used to solve a discrete-time approximation to the problem using data collected from the real-world plant. The utility of the framework is demonstrated in simulation and on a real-world robotic platform.

Tilted rotors in multirotor vehicles have shown to be useful for different practical reasons. For instance, increasing yaw maneuverability or enabling full position and attitude control of hexarotor vehicles. It has also been proven that a hexagon-shaped multirotor is capable of complete attitude and altitude control under failures of one of its rotors. However, when a rotor fails, the torque that can be reached in the worst- case direction decreases considerably.This work proposes to actively change the tilt angle of the rotors when a failure occurs. This rotor reconfiguration increases the maximum torque that can be achieved in the most stressful direction, reducing maneuverability limitations. Experimental validations are shown, where the proposed reconfigurable tilted rotor is used in order to control a hexarotor vehicle when a failure appears mid-flight. The impact of the delay in the reconfiguration when a failure occurs is also addressed.

It is known that the interior of the human body is one of the most adverse environments for a foreign body, such as an in-vivo robot, and vice-versa. As robots operating in-vivo are increasingly recognized for their capabilities and potential for improved therapies, it is important to ensure their safety, especially for long term treatments when little supervision can be provided. We introduce an implantable robot that is flexible, extendable and symmetric, thus changing shape and size. This design allows the implementation of an effective fault tolerant control, with features such as physical polling for fault diagnosis, retraction and redundancy-based control switching at fault. We demonstrate the fault-tolerant capabilities for an implantable robot that elongates tubular tissues by applying tension to the tissue. In benchtop tests, we show a reduction of the fault risks by at least 83%. The study provides a valuable methodology to enhance safety and efficacy of implantable and surgical robots, and thus to accelerate their adoption.