ICRA 2021 论文列表

Safety is a critical property in applications including robotics, transportation, and energy. Safety is especially challenging in reinforcement learning (RL) settings, in which uncertainty of the system dynamics may cause safety violations during exploration. Control Barrier Functions (CBFs), which enforce safety by constraining the control actions at each time step, are a promising approach for safety-critical control. This technique has been applied to ensure the safety of model-free RL, however, it has not been integrated into model-based RL. In this paper, we propose Uncertainty-Tolerant Control Barrier Functions (UTCBFs), a new class of CBFs to incorporate model uncertainty and provide provable safety guarantees with desired probability. Furthermore, we introduce an algorithm for model-based RL to guarantee safety by integrating CBFs with gradient-based policy search. Our approach is verified through a numerical study of a cart-pole system and an inverted pendulum system with comparison to state-of-the-art RL algorithms.

Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we introduce model-based dynamics and policy learning for efficient DR. A Bayesian model of locally linear embedding is designed to fit the stochastic dynamics in DR. By virtue of locally linear dynamics, model-based optimal control is substituted for the policy optimization. Unlike previous works, our proposed Bayesian model with a MNIW prior allows the locally linear embedding to capture the dynamics in DR as a stochastic model. We show that a training method that combines variational and adversarial approaches is adequate for Bayesian embedding. Finally, a model-based controller is designed on our Bayesian locally linear embedding, and it shows better performance in DR environments compared with the non-Bayesian model of locally linear embedding.

The da Vinci Research Kit (dVRK) was introduced in 2012 to provide an affordable, open-source platform for research in robotic minimally-invasive surgery. It provides access to all levels of control but, until now, has relied on an analog controller for the motor current, which cannot easily be customized to improve performance. This paper aims to implement the low-level control digitally and to improve the overall control performance. To enable model-based controller design, the system is first identified using measurements provided by the encoders and internal electronics. The digital current controller is then implemented on the existing field programmable gate array (FPGA). Experiments demonstrate that the new digital current controller yields superior performance compared to the original analog design. In addition, the identified system model is used to design an improved position controller that is also implemented on the FPGA and provides better trajectory tracking than the position controller currently implemented on the control PC. The comparison between simulation and measurement, for both the current and position control, verifies the validity of the model based on the system identification, enabling utilization for future adaptations. The improved low-level control enlarges the possibilities for more accurate operation and the achieved digital implementation enables researchers worldwide to easily adapt the low-level control in future versions of the dVRK.

Recent developments in surgical robotics have led to new advancements in the automation of surgical sub-tasks such as suturing, soft tissue manipulation, tissue tensioning and cutting. However, integration of dynamics to optimize these control policies for the variety of scenes encountered in surgery remains unsolved. Towards this effort, we investigate the integration of differentiable fluid dynamics to optimizing a suction tool’s trajectory to clear the surgical field from blood as fast as possible. The fully differentiable fluid dynamics is integrated with a novel suction model for effective model predictive control of the tool. The differentiability of the fluid model is crucial because we utilize the gradients of the fluid states with respect to the suction tool position to optimize the trajectory. Through a series of experiments, we demonstrate how, by incorporating fluid models, the trajectories generated by our method can perform as good as or better than handcrafted human-intuitive suction policies. We also show that our method is adaptable and can work in different cavity conditions while using a single handcrafted strategy fails.

We study synthesis of control strategies from linear temporal logic (LTL) objectives in unknown environments. We model this problem as a turn-based zero-sum stochastic game between the controller and the environment, where the transition probabilities and the model topology are fully unknown. The winning condition for the controller in this game is the satisfaction of the given LTL specification, which can be captured by the acceptance condition of a deterministic Rabin automaton (DRA) directly derived from the LTL specification. We introduce a model-free reinforcement learning (RL) methodology to find a strategy that maximizes the probability of satisfying a given LTL specification when the Rabin condition of the derived DRA has a single accepting pair. We then generalize this approach to any LTL formulas, for which the Rabin accepting condition may have more than one pairs, providing a lower bound on the satisfaction probability. Finally, we show applicability of our RL method on two planning case studies.

Current lower-limb prosthesis control methods are primarily model-independent — lacking formal guarantees of stability, relying largely on heuristic tuning parameters for good performance, and neglecting use of the natural dynamics of the system. Model-dependence for prosthesis controllers is difficult to achieve due to the unknown human dynamics. We build upon previous work which synthesized provably stable prosthesis walking through the use of rapidly exponentially stabilizing control Lyapunov functions (RES-CLFs). This paper utilizes RES-CLFs together with force estimation to construct model-based optimization-based controllers for the prosthesis. These are experimentally realized on hardware with onboard sensing and computation. This hardware demonstration has formal guarantees of stability, utilizes the natural dynamics of the system, and achieves superior tracking to other prosthesis trajectory tracking control methods.

The interaction between an exoskeleton and its human user is complex, and needs to conform to various requirements related to safety, comfort and adaptability. It is however impractical to test a large number of prototype variations against a large number of user variations, especially in the initial design and testing phases. Model based methods can help at this design stage by providing a virtual testbed. In this study, we develop a MATLAB-based toolbox that can simulate the interaction between a human model and a lower limb exoskeleton during the sit to stand motion. We present results for different scales of human users as well as variation in levels of exoskeleton assistance. Our results show that large reductions in human joint torques upto 57Nm are possible, while also transmitting large forces upto 300N to the human model. Additionally, by varying the human body sizes by 15% we found that the interaction forces also changed by as much as 29.2%. Therefore, careful consideration of the human user and its limitation should be made in the exoskeleton design and concept phases.

Soft robots are known for their ability to comply and having superior extensibility. However, one of the limitations of most of these robots is that they can stand only a limited amount of load before buckling, and they feature a non-negligible initial height. Hybrid soft-rigid actuators seem to offer a trade-off between compliance and the amount of load they can withstand, but only a few simple models have been proposed to describe the behavior of these actuators. In this paper, we propose a design, model and experimental validation of a soft actuator based on stackable Hyperelastic Ballooning Membranes (HBMA). This actuator shows an extensibility higher than 179%, as well as an ability to stand more than 20 times its own weight at a pressure as low as 35 kPa. Two models, giving the dynamic behavior of the HBMA in terms of displacement and pressure, have been derived from different hyperelastic models (Neo-Hookean and Mooney-Rivlin) and compared in terms of accuracy and robustness. Finally, an example of a hybrid soft-rigid continuum ballooning robot built with HBMAs is presented and characterized experimentally.

Modern, torque-controlled service robots can regulate contact forces when interacting with their environment. Model Predictive Control (MPC) is a powerful method to solve the underlying control problem, allowing to plan for whole-body motions while including different constraints imposed by the robot dynamics or its environment. However, an accurate model of the robot-environment is needed to achieve a satisfying closed-loop performance. Currently, this necessity undermines the performance and generality of MPC in manipulation tasks. In this work, we combine an MPC-based whole-body controller with two adaptive schemes, derived from online system identification and adaptive control. As a result, we enable a general mobile manipulator to interact with unknown environments, without any need for re-tuning parameters or pre-modeling the interacting objects. In combination with the MPC controller, the two adaptive approaches are validated and benchmarked with a ball-balancing manipulator in door opening and object lifting tasks.

Control of nonlinear uncertain systems is a common challenge in the robotics field. Nonlinear latent force models, which incorporate latent uncertainty characterized as Gaussian processes, carry the promise of representing such systems effectively, and we focus on the control design for them in this work. To enable the design, we adopt the state-space representation of a Gaussian process to recast the nonlinear latent force model and thus build the ability to predict the future state and uncertainty concurrently. Using this feature, a stochastic model predictive control problem is formulated. To derive a computational algorithm for the problem, we use the scenario-based approach to formulate a deterministic approximation of the stochastic optimization. We evaluate the resultant scenario-based model predictive control approach through a simulation study based on motion planning of an autonomous vehicle, which shows much effectiveness. The proposed approach can find prospective use in various other robotics applications.

Formation control of multi-agent systems deals with groups of robots forming specific spatial geometries. Combined with the advancements of unmanned aerial vehicles (UAVs) in the past decade, formation control may potentially be applied to tasks such as search-and-rescue, surveillance, even collaborative manipulation. A key challenge is the decentralization of formation control, where each agent behaves independently using onboard sensors and computation, improving the scaleability and robustness of the system.This paper proposes a decentralized controller based on model predictive control (MPC), for the control of formations of quadrotor UAVs defined by inter-agent bearings. The use of MPC allows the controller to account for attitude kinematics, improving upon the results of existing bearing formation control methods based on rigidity and visual servoing approaches, which typically only consider the quadrotor as a single or double integrator. Furthermore the near-optimality of MPC permits a more optimal use of the quadrotors dynamic capabilities for faster maneuvering. Extensive simulations are performed to demonstrate the improved transient formation convergence and fast maneuvering permitted by this controller. Experiments show that it is indeed a real-time feasible solution for bearing formation control.

This paper studies the cooperative hunting problem, where a group of agents encircle a target while avoiding collisions with each other and with obstacles in the environment. The paper deals with obstacle rich environments and dynamic (moving obstacle) environments by formulating the problem as both a control problem and a planning problem. A model predictive control (MPC) method is proposed which integrates a multi-agent planner with the cooperative hunting objective while also accounting for UAV dynamics. The effectiveness of the proposed method is verified through a comparative analysis with optimal reciprocal collision avoidance (ORCA), and then validated through experiments with quadrotor UAVs. Using the proposed method, agents no longer get stuck in local minima for obstacle rich environments and capture the target faster with shorter trajectories in moving obstacle environments.

Substantial advancements to model-based reinforcement learning algorithms have been impeded by the model-bias induced by the collected data, which generally hurts performance. Meanwhile, their inherent sample efficiency warrants utility for most robot applications, limiting potential damage to the robot and its environment during training. Inspired by information theoretic model predictive control and advances in deep reinforcement learning, we introduce Model Predictive Actor-Critic (MoPAC)†, a hybrid model-based/model-free method that combines model predictive rollouts with policy optimization as to mitigate model bias. MoPAC leverages optimal trajectories to guide policy learning, but explores via its model-free method, allowing the algorithm to learn more expressive dynamics models. This combination guarantees optimal skill learning up to an approximation error and reduces necessary physical interaction with the environment, making it suitable for real-robot training. We provide extensive results showcasing how our proposed method generally outperforms current state-of-the-art and conclude by evaluating MoPAC for learning on a physical robotic hand performing valve rotation and finger gaiting–a task that requires grasping, manipulation, and then regrasping of an object.

A long short-term memory neural network is used to provide a system model that captures the temporal-dynamics of a holonomic, fully-actuated aquatic surface vehicle. As is true in many fields, new developments in robotics often are made in simulation first before being applied to real systems. To simulate an aquatic or aerial robot, a dynamic system model of the robot is required. The more representative the dynamic model is of the real robot, the smaller the simulation-to-reality gap becomes. The performance of the neural network is compared against a classical parametric model, where coefficients of the parametric model were identified using the same data that was used to train the neural network. The results show that the neural network consistently outperforms the classical parametric model and significantly reduces the error between real velocities and estimated velocities. The neural network also demonstrated the ability to capture complex hydrodynamic effects that were not captured in the parametric model. In addition to the performance improvements, the neural network method can be easily adapted to similarly actuated aquatic vehicles by simply retraining, whereas the classical approach would require manual selection of new equation terms. The neural network model that was created has been used in a vehicle simulation and is presently being used as a research tool.

This study presents a novel dynamic modelling method for analyzing the modal properties of a curved extensible continuum manipulator (ECM). Considering the variable bending angle and the length, the kinematic and static models are firstly established for descripting the geometric posture deformation and deflection generated by gravity. Then, the modal dynamic model is developed along the centerline direction of the ECM in the polar coordinates, by which the modal properties, include modal frequency and mode shape of curved continuum manipulator with different posture parameters are analyzed. The modal response is further analyzed based on the generalized coordinate method. Finally, the modal properties of ECM with variable posture are experimentally tested and compared with the theoretical value using the in-house developed ECM physical prototype. In the experiments, the bending, length varying and circular tracking motion are performed for the comprehensive validation of the proposed dynamics modelling method.

Recent work in the multi-agent domain has shown the promise of Graph Neural Networks (GNNs) to learn complex coordination strategies. However, most current approaches use minor variants of a Graph Convolutional Network (GCN), which applies a convolution to the communication graph formed by the multi-agent system. In this paper, we investigate whether the performance and generalization of GCNs can be improved upon. We introduce ModGNN, a decentralized framework which serves as a generalization of GCNs, providing more flexibility. To test our hypothesis, we evaluate an implementation of ModGNN against several baselines in the multi-agent flocking problem. We perform an ablation analysis to show that the most important component of our framework is one that does not exist in a GCN. By varying the number of agents, we also demonstrate that an application-agnostic implementation of ModGNN possesses an improved ability to generalize to new environments.

The flexibility and adaptability of modular and re-configurable robots opens up new opportunities for on-demand robot morphology optimization for varying tasks. In particular, multi-arm robotic systems can expand the solution space for any given task. In this paper, we present a novel approach to exploit this feature for generating optimal fit-to-task robot structures with respect to a minimum-effort objective. By describing the task in terms of relative poses between the end-effector and the constraint frame, and making use of the relative Jacobian, the minimum effort optimization problem can be equally expressed for single-arm or multi-arm robots. We test our approach for a peg-in-hole and a contour-following task and compare the performance of the optimal solution obtained with that of a standard manipulator configuration.

A robot’s dynamics – especially the degree and location of compliance – can significantly affect performance and control complexity. Passive dynamics can be designed with good regions of attraction or limit cycles for a specific task, but achieving flexibility on a range of tasks requires co-design of control. This paper takes an information perspective: the robot dynamics should reduce the amount of information required for a controller to achieve a threshold of performance in a range of tasks. Towards this goal, an iterative method is proposed to minimize the directed information from state to control on discrete-time nonlinear systems. iLQG is used to find a controller and value of information, then the design parameters of the dynamics (e.g. stiffness of end-effector or joint) are optimized to reduce directed information while maintaining a minimum bound on performance. The approach is validated in simulation, on a two-mass system in contact with an uncertain wall position and a high-DOF door opening task, and shown to improve noise robustness and reduce time variance of control gains.

Multi-task IRL recognizes that expert(s) could be switching between multiple ways of solving the same problem, or interleaving demonstrations of multiple tasks. The learner aims to learn the reward functions that individually guide these distinct ways. We present a new method for multi-task IRL that generalizes the well-known maximum entropy approach by combining it with a Dirichlet process based minimum entropy clustering of the observed data. This yields a single nonlinear optimization problem, called MinMaxEnt Multi-task IRL (MME-MTIRL), which can be solved using the Lagrangian relaxation and gradient descent methods. We evaluate MME-MTIRL on the robotic task of sorting onions on a processing line where the expert utilizes multiple ways of detecting and removing blemished onions. The method is able to learn the underlying reward functions to a high level of accuracy and it improves on the previous approaches.