ICRA 2019 论文列表

We present a navigation system that combines ideas from hierarchical planning and machine learning. The system uses a traditional global planner to compute optimal paths towards a goal, and a deep local trajectory planner and velocity controller to compute motion commands. The latter components of the system adjust the behavior of the robot through attention mechanisms such that it moves towards the goal, avoids obstacles, and respects the space of nearby pedestrians. Both the structure of the proposed deep models and the use of attention mechanisms make the system's execution interpretable. Our simulation experiments suggest that the proposed architecture outperforms baselines that try to map global plan information and sensor data directly to velocity commands. In comparison to a hand-designed traditional navigation system, the proposed approach showed more consistent performance.

The synchronization of the movement between exoskeleton robot and human arm is crucial for Robot-assisted training (RAT) in upper limb rehabilitation. In this paper, we propose a deep learning based motion prediction model which is applied to our recently developed 8 degrees-of-freedom (DoFs) upper limb rehabilitation exoskeleton, named NTUH-II. The human arm dynamics and surface electromyography (sEMG) can be first measured by two wireless sensors and used as input of deep learning model to predict user's motion. Then, the prediction can be used as desired motion trajectory of the exoskeleton. As a result, the robot arm can follow the movement on either side of the user's arm in real-time. Various experiments have been conducted to verify the performance of the proposed motion prediction model, and the results show that the proposed motion prediction implementation can reduce the mean absolute error and the average delay time of movement between human arm and robot arm.

In contrast to underactuated robotic hands the DLR AWIWI II hand of the David robot is fully controllable because each finger with 4 joints is actuated by 6 or 8 tendons respectively. For such fingers all joint angles (generalized positions) or joint torques (generalized forces) can be controlled independently. Usually, the specifications in joint space are converted to desired tendon forces or motor torques, which are regulated by an inner loop impedance controller. However, this conversion typically exhibits couplings between the components of the joint angle vector or the joint torque vector respectively, which arise when using the well known equations. Therefore the usual force control and position control schemes are reviewed and a generic computation of the desired tendon forces is presented. This is also done for the control of the Cartesian position and force at the finger endpoint. Thus the main contribution of the paper is the inhibition of couplings in joint space or at the Cartesian endpoint. This is demonstrated in simulations of the index finger of the DLR David hand.

Small unmanned aerial vehicles (UAVs) have generally little capacity to carry payloads. Through collaboration, the UAVs can increase their joint payload capacity and carry more significant loads. For maximum flexibility to dynamic and unstructured environments and task demands, we propose a fully decentralized control infrastructure based on a swarm-specific scripting language, Buzz. In this paper, we describe the control infrastructure and use it to compare two algorithms for collaborative transport: field potentials and spring-damper. We test the performance of our approach with a fleet of micro-UAVs, demonstrating the potential of decentralized control for collaborative transport.

This paper addresses the problem of decentralized, outdoor formation coordination with multiple quadcopters. The problem is formulated as a receding horizon, mixed-integer non-linear program (RH-MINLP). Each quadcopter solves this RH-MINLP to generate its time-optimal speed profile along a minimum snap spline path while coordinating its position in a desired formation with other quadcopters. Constraints on quadcopter kinematics, dynamics, collision avoidance, wireless communication connectivity, and geometric formations are modeled. Communication connectivity is modeled as a constraint on maximum separation distance based on a minimum viable received signal strength in the presence of path loss attenuation. The resulting RH-MINLP is non-convex, and is solved using an outer-approximation branch and bound solver with a warm-starting scheme. The framework is validated via Hardware-in-the-Loop (HITL) and outdoor flight test with up to 6 quadcopters. Results demonstrate the effect of number of quadcopters and formation type on total transit time. Average radio packet loss statistics during transit indicate robust network performance for a round robin communication scheduling scheme.

Effective communication is required for teams of robots to solve sophisticated collaborative tasks. In practice it is typical for both the encoding and semantics of communication to be manually defined by an expert; this is true regardless of whether the behaviors themselves are bespoke, optimization based, or learned. We present an agent architecture and training methodology using neural networks to learn task-oriented communication semantics based on the example of a communication-unaware expert policy. A perimeter defense game illustrates the system's ability to handle dynamically changing numbers of agents and its graceful degradation in performance as communication constraints are tightened or the expert's observability assumptions are broken.

Robot design is often a slow and difficult process requiring the iterative construction and testing of prototypes, with the goal of sequentially optimizing the design. For most robots, this process is further complicated by the need, when validating the capabilities of the hardware to solve the desired task, to already have an appropriate controller, which is in turn designed and tuned for the specific hardware. In this paper, we propose a novel approach, HPC-BBO, to efficiently and automatically design hardware configurations, and evaluate them by also automatically tuning the corresponding controller. HPC-BBO is based on a hierarchical Bayesian optimization process which iteratively optimizes morphology configurations (based on the performance of the previous designs during the controller learning process) and subsequently learns the corresponding controllers (exploiting the knowledge collected from optimizing for previous morphologies). Moreover, HPC-BBO can select a “batch” of multiple morphology designs at once, thus parallelizing hardware validation and reducing the number of time-consuming production cycles. We validate HPC-BBO on the design of the morphology and controller for a simulated 6-legged microrobot. Experimental results show that HPC-BBO outperforms multiple competitive baselines, and yields a 360% reduction in production cycles over standard Bayesian optimization, thus reducing the hypothetical manufacturing time of our microrobot from 21 to 4 months.

Hybrid systems, such as bipedal walkers, are challenging to control because of discontinuities in their nonlinear dynamics. Little can be predicted about the systems' evolution without modeling the guard conditions that govern transitions between hybrid modes, so even systems with reliable state sensing can be difficult to control. We propose an algorithm that allows for determining the hybrid mode of a system in real-time using data-driven analysis. The algorithm is used with data-driven dynamics identification to enable model predictive control based entirely on data. Two examples-a simulated hopper and experimental data from a bipedal walker-are used. In the context of the first example, we are able to closely approximate the dynamics of a hybrid SLIP model and then successfully use them for control in simulation. In the second example, we demonstrate gait partitioning of human walking data, accurately differentiating between stance and swing, as well as selected subphases of swing. We identify contact events, such as heel strike and toe-off, without a contact sensor using only kinematics data from the knee and hip joints, which could be particularly useful in providing online assistance during walking. Our algorithm does not assume a predefined gait structure or gait phase transitions, lending itself to segmentation of both healthy and pathological gaits. With this flexibility, impairment-specific rehabilitation strategies or assistance could be designed.

We propose a novel data-driven learning-based contact clustering (i.e., of contact points and contact normals) framework for rigid-body robot simulation, with its accuracy established/verified by real experimental data. We first construct an experimental robotic setup with force/torque (F/T) sensors to collect real contact motion/force data. We then design a multilayer perceptron (MLP) network for the contact clustering based on the full motion and force/torque information of the contacts. We also adopt the constraint-based optimization contact solver to facilitate the learning of our MLP network during the training. Our proposed data-driven/learning-based contact clustering framework is then verified against the experimental setup, compared with other techniques/simulators and shown to significantly (or meaningfully) enhance the accuracy of contact simulation as compared to them.

Soft robots are advantageous in terms of flexibility, safety and adaptability. It is challenging to find efficient computational approaches for planning and controlling their motion. This work takes a direct data-driven approach to learn the kinematics of the three-dimensional shape of a soft robot, by using visual markers. No prior information about the robot at hand is required. The model is oblivious to the design of the robot and type of actuation system. This allows adaptation to erroneous manufacturing. We present a highly versatile and inexpensive learning cube environment for collecting and analysing data. We prove that using multiple, lower order models of data opposed to one global, higher order model, will reduce the required data quantity, time complexity and memory complexity significantly without compromising accuracy. Further, our approach allows for embarrassingly parallelism. Yielding an overall much more simple and efficient approach.

Recently, research works have attempted the joint prediction of scene semantics and optical flow estimation, which demonstrate the mutual improvement between both tasks. Besides, the depth information is also indispensable for the scene understanding, and disparity estimation is necessary for outputting dense depth maps. Such task shares a great similarity with the optical flow estimation since they can all be cast into a problem of capturing the difference at a location of two image frames. However, as far as we know, currently there are few networks for the joint learning of semantic and disparity. Moreover, since deep semantic information and disparity feature maps can learn from each other, we find it unnecessary with two independent encoding modules to separately extract semantic and disparity features. Therefore, we propose a unified multi-tasking architecture DSNet, for the simultaneous estimation of semantic and disparity information. In our model, semantic features, extracted by the encoding module ResNet from the left and right images, are used to obtain the deep disparity features via a novel matching module which performs pixel-to-pixel matching. In addition, we also use the disparity map to perform warp operation on deep features of the right image to deal with the problem of lacking of semantic labels. The effectiveness of our method is demonstrated by extensive experiments.

Dynamic Movement Primitive (DMP) which is popular for motion planning of a robot manipulator, has been adapted for a nonholonomic mobile robot to track the desired trajectory. DMP is a simple damped spring model with a forcing function, which learns the trajectory. The damped spring model attracts the robot towards the goal position, and the forcing function forces the robot to follow the given trajectory. Two Radial Basis Function Networks (RBFNs) have been used to learn the forcing function associated with the DMP model. Weight update laws are derived using the gradient descent approach to train the RBFNs. Fuzzy logic based steering angle dynamics is proposed to handle the asymmetric nature of an obstacle. The proposed scheme is capable enough to generate a smooth trajectory in the presence of an obstacle even when start and goal positions are altered, without losing the spatial information embedded while training. The convergence of the robot goal position has been shown using Lyapunov stability theory-based analysis. The approach has been extended to multiple static and dynamic obstacles for the successful convergence of the robot at the goal position. Both simulation and experimental results are provided to confirm the efficacy of the proposed scheme.

For the domain of self-driving and automatic parking, perception is a basic and critical technique, moreover, the detection of lane markings and parking slots is an important part of visual perception. Compared with front sight images, panoramic images(PI) can capture more comprehensive pavement information. However, the imbalance of different classes in PI is even more serious. Additionally, the judgment of boundary information between areas is a hard problem in deep models. Therefore, we propose a new model named DFNet to solve these problems. The proposed model has two main contributions, one is dynamic loss weights, and the other is residual fusion block(RFB). DFNet use dynamic loss weights to overcome the negative effect of imbalance dataset, which are calculated according to the pixel number of each class in a batch. RFB is composed of several convolutional layers, a pooling layer, and a fusion layer to combine the feature maps by pixel multiplication, which can reduce boundary information loss. We evaluate our method on PSV dataset, and the achieved advanced results demonstrate the effectiveness of the proposed model.

Object detection models based on convolutional neural networks (CNNs) demonstrate impressive performance when trained on large-scale labeled datasets. While a generic object detector trained on such a dataset performs adequately in applications where the input data is similar to user photographs, the detector performs poorly on small objects, particularly ones with limited training data or imaged from uncommon viewpoints. Also, a specific room will have many objects that are missed by standard object detectors, frustrating a robot that continually operates in the same indoor environment.This paper describes a system for rapidly creating customized object detectors. Data is collected from a quadcopter that is teleoperated with an interactive interface. Once an object is selected, the quadcopter autonomously photographs the object from multiple viewpoints to collect data to train DUNet (Dense Upscaled Network), our proposed model for learning customized object detectors from scratch given limited data. Our experiments compare the performance of learning models from scratch with DUNet vs. fine tuning existing state of the art object detectors, both on our indoor robotics domain and on standard datasets.

There are two difficulties to utilize state-of-the-art object recognition/detection/segmentation methods to robotic applications. First, most of the deep learning models heavily depend on large amounts of labeled training data, which are expensive to obtain for each individual application. Second, the object categories must be pre-defined in the dataset, thus not practical to scenarios with varying object categories. To alleviate the reliance on pre-defined big data, this paper proposes a customized object recognition and segmentation method. It aims to recognize and segment any object defined by the user, given only one annotation. There are three steps in the proposed method. First, the user takes an exemplar video of the target object with the robot, defines its name, and mask its boundary on only one frame. Then the robot automatically propagates the annotation through the exemplar video based on a proposed data generation method. In the meantime, a segmentation model continuously updates itself on the generated data. Finally, only a lightweight segmentation net is required at testing stage, to recognize and segment the user-defined object in any scenes.

Mobility in an effective and socially-compliant manner is an essential yet challenging task for robots operating in crowded spaces. Recent works have shown the power of deep reinforcement learning techniques to learn socially cooperative policies. However, their cooperation ability deteriorates as the crowd grows since they typically relax the problem as a one-way Human-Robot interaction problem. In this work, we want to go beyond first-order Human-Robot interaction and more explicitly model Crowd-Robot Interaction (CRI). We propose to (i) rethink pairwise interactions with a self-attention mechanism, and (ii) jointly model Human-Robot as well as Human-Human interactions in the deep reinforcement learning framework. Our model captures the Human-Human interactions occurring in dense crowds that indirectly affects the robot's anticipation capability. Our proposed attentive pooling mechanism learns the collective importance of neighboring humans with respect to their future states. Various experiments demonstrate that our model can anticipate human dynamics and navigate in crowds with time efficiency, outperforming state-of-the-art methods.

Fast robotics pick-and-place with suction cups is a crucial component in the current development of automation in logistics (factory lines, e-commerce, etc.). By “critically fast” we mean the fastest possible movement for transporting an object such that it does not slip or fall from the suction cup. The main difficulties are: (i) handling the contact between the suction cup and the object, which fundamentally involves kinodynamic constraints; and (ii) doing so at a low computational cost, typically a few hundreds of milliseconds. To address these difficulties, we propose (a) a model for suction cup contacts, (b) a procedure to identify the contact stability constraint based on that model, and (c) a pipeline to parameterize, in a time-optimal manner, arbitrary geometric paths under the identified contact stability constraint. We experimentally validate the proposed pipeline on a physical robot system: the cycle time for a typical pick-and-place task was less than 5 seconds, planning and execution times included. The full pipeline is released as opensource for the robotics community.

We study the problem of covering an environment using an Unmanned Aerial Vehicle (UAV) with limited battery capacity. We consider a scenario where the UAV can land on an Unmanned Ground Vehicle (UGV) and recharge the onboard battery. The UGV can also recharge the UAV while transporting the UAV to the next take-off site. We present an algorithm to solve a new variant of the area coverage problem that takes into account this symbiotic UAV and UGV system. The input consists of a set of boustrophedon cells - rectangular strips whose width is equal to the field-of-view of the sensor on the UAV. The goal is to find a tour for the UAV that visits and covers all cells in minimum time. This includes flight time for visiting and covering all cells, recharging time, as well as the take-off and landing times. We show how to reduce this problem to a known NP-hard problem, Generalized Traveling Salesperson Problem (GTSP). Given an optimal GTSP solver, our approach finds the optimal coverage paths for the UAV and UGV. We evaluate our algorithm through simulations and proof-of-concept experiments.

For safety reasons, robotic lawn mowers and similar devices are required to stay within a predefined working area. Keeping the robot within its workspace is typically achieved by special safeguards such as a wire installed in the ground. In the case of robotic lawn mowers, this causes a certain customer reluctance. It is more desirable to fulfill those safety-critical tasks by safe navigation and path planning. In this paper, we tackle the problem of planning a coverage path composed of parallel lanes that maximizes robot safety under the constraints of cheap, low range sensors and thus substantial uncertainty in the robot's belief and ability to execute actions. Our approach uses a map of the environment to estimate localizability at all locations, and it uses these estimates to search for an uncertainty-aware coverage path while avoiding collisions. We implemented our approach using C++ and ROS and thoroughly tested it on real garden data. The experiment shows that our approach leads to safer meander patterns for the lawn mower and takes expected localizability information into account.