ICRA 2021 论文列表

Ellipsoid-based manipulability measures are often used to characterize the force/velocity task-space capabilities of robots. While computationally simple, this approach largely approximate and underestimate the true capabilities. Force/velocity polytopes appear to be a more appropriate representation to characterize the robot’s task-space capabilities. However, due to the computational complexity of the associated vertex search problem, the polytope approach is mostly restricted to offline use, e.g. as a tool aiding robot mechanical design, robot placement in work-space and offline trajectory planning. In this paper, a novel on-line polytope vertex search algorithm is proposed. It exploits the parallelotope geometry of actuator constraints. The proposed algorithm significantly reduces the complexity and computation time of the vertex search problem in comparison to commonly used algorithms. In order to highlight the on-line capability of the proposed algorithm and its potential for robot control, a challenging experiment with two collaborating Franka Emika Panda robots, carrying a load of 12 kilograms, is proposed. In this experiment, the load distribution is adapted on-line, as a function of the configuration dependant task-space force capability of each robot, in order to avoid, as much as possible, the saturation of their capacity.

In this paper, we study the effect of alternative leg designs on energy consumption in legged locomotion. Focusing on gaits with constant horizontal velocity and constant height we introduce models of two simplified parallel and serial designs with realistic mechanical and actuation parameters. The analysis yields the distribution of power demands in the leg workspace, leading to useful conclusions related to mechanical power antagonism and actuator electric losses. Mechanical antagonism occurs not only in parallel but also in serial legs causing extensive power waste, since one actuator contributes to the locomotion task and the other consumes power with no contribution to it. Based on the analysis, we propose a new leg design that minimizes the total actuation power consumption criterion given a nominal robot toe trajectory.

Twisted string actuators (TSA) are efficient, compliant cable actuators with high power density that have found their way into many engineering and robotics applications in recent years. They are generally used in the scenarios involving comparatively slow positioning, since quick motions in TSAs often result in large overshoot values, undesired oscillations and loss of cable tension. This work discusses time optimal trajectory generation for point-to-point transitions in TSA. We propose a method to generate smooth trajectories by directly solving an optimal control problem while respecting constraints on motor torque and speed and preventing tension loss on cables. We have conducted an experimental evaluation demonstrating that, in comparison to classical, constrained ‘bangbang’ transitions, the proposed trajectories result in much faster settling time, nearly-zero overshoot and require almost 60% less motor power for tracking. The proposed method can help TSAs find their way into highly-dynamical applications that have been previously deemed to be too demanding, which include parallel manipulators and antagonistically-controlled systems.

In this paper, we explore whether a robot can learn to hang arbitrary objects onto a diverse set of supporting items such as racks or hooks. Endowing robots with such an ability has applications in many domains such as domestic services, logistics, or manufacturing. Yet, it is a challenging manipulation task due to the large diversity of geometry and topology of everyday objects. In this paper, we propose a system that takes partial point clouds of an object and a supporting item as input and learns to decide where and how to hang the object stably. Our system learns to estimate the contact point correspondences between the object and supporting item to get an estimated stable pose. We then run a deep reinforcement learning algorithm to refine the predicted stable pose. Then, the robot needs to find a collision-free path to move the object from its initial pose to stable hanging pose. To this end, we train a neural network based collision estimator that takes as input partial point clouds of the object and supporting item. We generate a new and challenging, large-scale, synthetic dataset annotated with stable poses of objects hung on various supporting items and their contact point correspondences. In this dataset, we show that our system is able to achieve a 68.3% success rate of predicting stable object poses and has a 52.1% F1 score in terms of finding feasible paths. Supplemental material and videos are available on our project webpage https://sites.google.com/view/hangingobject.

In this work, we focus on mobile robot navigation in indoor environments where occlusions and field-of-view limitations hinder onboard sensing capabilities. We show that the footprint of a camera mounted on a robot can be drastically improved using learning-based approaches. Specifically, we consider the task of building an occupancy map for autonomous navigation of a robot equipped with a depth camera. In our approach, a local occupancy map is first computed using measurements from the camera directly. Afterwards, an inpainting network adds further information, the occupancy probabilities of unseen grid cells, to the map. A novel aspect of our approach is that rather than direct supervision from ground truth, we combine the information from a second camera with a better field-of-view for supervision. The training focuses on predicting extensions of the sensed data. To test the effectiveness of our approach, we use a robot setup with a single camera placed at 0.5m above the ground. We compare the navigation performance using raw maps from only this camera’s input (baseline) versus using inpainted maps augmented with our network. Our method outperforms the baseline approach even in completely new environments not included in the training set and can yield 21% shorter paths than the baseline approach. A real-time implementation of our method on a mobile robot is also tested in home and office environments.

This paper presents Kinetic Energy Difference (KED) as a metric for collision proximity. The calculation of KED for differentially driven robots is explained, along with an example obstacle avoidance algorithm that utilizes it. This example algorithm is computationally efficient and simulations show that it is capable of guiding robots with slow dynamics through narrow corridors.

Recent advances in deep reinforcement learning (deep RL) enable researchers to solve challenging control problems, from simulated environments to real-world robotic tasks. However, deep RL algorithms are known to be sensitive to the problem formulation, including observation spaces, action spaces, and reward functions. There exist numerous choices for observation spaces but they are often designed solely based on prior knowledge due to the lack of established principles. In this work, we conduct benchmark experiments to verify common design choices for observation spaces, such as Cartesian transformation, binary contact flags, a short history, or global positions. Then we propose a search algorithm to find the optimal observation spaces, which examines various candidate observation spaces and removes unnecessary observation channels with a Dropout-Permutation test. We demonstrate that our algorithm significantly improves learning speed compared to manually designed observation spaces. We also analyze the proposed algorithm by evaluating different hyperparameters.

In order to interact with the world, agents must be able to predict the results of the world’s dynamics. A natural approach to learn about these dynamics is through video prediction, as cameras are ubiquitous and powerful sensors. Direct pixel-to-pixel video prediction is difficult, does not take advantage of known priors, and does not provide an easy interface to utilize the learned dynamics. Object-centric video prediction offers a solution to these problems by taking advantage of the simple prior that the world is made of objects and by providing a more natural interface for control. However, existing object-centric video prediction pipelines require dense object annotations in training video sequences. In this work, we present Object-centric Prediction without Annotation (OPA), an object-centric video prediction method that takes advantage of priors from powerful computer vision models. We validate our method on a dataset comprised of video sequences of stacked objects falling, and demonstrate how to adapt a perception model in an environment through end-to-end video prediction training.

Robotic object rearrangement combines the skills of picking and placing objects. When object models are unavailable, typical collision-checking models may be unable to predict collisions in partial point clouds with occlusions, making generation of collision-free grasping or placement trajectories challenging. We propose a learned collision model that accepts scene and query object point clouds and predicts collisions for 6DOF object poses within the scene. We train the model on a synthetic set of 1 million scene/object point cloud pairs and 2 billion collision queries. We leverage the learned collision model as part of a model predictive path integral (MPPI) policy in a tabletop rearrangement task and show that the policy can plan collision-free grasps and placements for objects unseen in training in both simulated and physical cluttered scenes with a Franka Panda robot. The learned model outperforms both traditional pipelines and learned ablations by 9.8% in accuracy on a dataset of simulated collision queries and is 75x faster than the best-performing baseline. Videos and supplementary material are available at https://research.nvidia.com/publication/2021-03_Object-Rearrangement-Using.

Accurate and fast inventory management algorithms are essential in the modern distribution industry. However, the configuration process of inventory management algorithms is very expensive, and the direct comprehensive management of inventory procedures is labor intensive and inaccurate. Therefore, in this paper, we propose an optical character recognition (OCR)-based inventory management algorithm to resolve these practical issues. The main purpose of our inventory management algorithm is to automatically inspect whether a list of items and the actual items match. To this end, our method consists of three steps, namely, text detection, text recognition, and text matching. In addition, to expand our algorithm to real-world applications, we propose adversarial training to ensure robustness against various damaged images, including corruption, blur, and inappropriate viewpoints. To train the network, we construct a new inventory management dataset (IMD) consisting of 10,000 sheets in real retail store environments. We verify our algorithm on the public dataset and our new IMD. As a result, we experimentally demonstrate that our method is not only robust against various damaged images but is also easily applicable in both large and small scale distributions stores at a low cost. Our code and dataset is available at https://blog.airlab.re.kr/Deform-and-Recover/.

This paper proposes a novel force controller that serially combines admittance and impedance controllers. The proposed controller is adaptable to an unknown changeable environment in terms of stiffness, and it is able to achieve high control accuracy and stable operation. First, conventional admittance and impedance controllers are recalled, and based on them, a new force controller is designed. Next, the proposed controller is applied to a one DoF system in contact with an external environment in the case where the contact stiffness is changeable, and compare the behavior of the proposed controller with that of the conventional simple admittance and impedance controllers through numerical simulations. Additionally, the proposed controller is applied to a two DoFs system including some nonlinearities, and proposes a design of the desired anisotropic admittance and impedance parameters to the proposed controller. This effectiveness is also demonstrated through numerical simulation results.

Simulating the swimming of soft underwater robot remains challenging due to the absence of an efficient numerical framework that can effectively capture the geometrically nonlinear deformation of soft materials and structures when interacting with a liquid environment. Here, we address this by introducing a discrete differential geometry-based model that incorporates an implicit treatment of the elasticity of soft limbs and a fluid model with three different components: hydrodynamic drag, jetting, and virtual added mass. The physical engine can run faster than real-time on a single thread desktop processor. We experimentally validate this numerical simulation tool by performing tests using an untethered omni-directional star-shaped swimming soft robot that is capable of moving with multiple swimming gaits. Quantitative agreement between experiment and simulation indicates the potential application of such a numerical framework for robot design and for model-based control schemes.

Derivatives of equations of motion describing the rigid body dynamics are becoming increasingly relevant for the robotics community and find many applications in design and control of robotic systems. Controlling robots, and multibody systems comprising elastic components in particular, not only requires smooth trajectories but also the time derivatives of the control forces/torques, hence of the equations of motion (EOM). This paper presents novel nth order time derivatives of the EOM in both closed and recursive forms. While the former provides a direct insight into the structure of these derivatives, the latter leads to their highly efficient implementation for large degree of freedom robotic system.

For 25 years the Robotics Toolbox for MATLAB® has been used for teaching and research worldwide. This paper describes its successor – the Robotics Toolbox for Python. More than just a port, it takes advantage of popular open-source packages and resources to provide platform portability, fast browser-based 3D graphics, quality documentation, fast numerical and symbolic operations, powerful IDEs, shareable and web-browseable notebooks all powered by GitHub and the open-source community. The new Toolbox provides well-known functionality for spatial mathematics (homogeneous transformations, quaternions, triple angles and twists), trajectories, kinematics (zeroth to second order), dynamics and a rich assortment of robot models. In addition, we’ve taken the opportunity to add new capabilities such as branched mechanisms, collision checking, URDF import, and interfaces to ROS. With familiar, simple yet powerful functions; the clarity of Python syntax; but without the complexity of ROS; users from beginner to advanced will find this a powerful open-source toolset for ongoing robotics education and research.

Template models are frequently used to simplify the control dynamics for robot hopping or running. Passive limit cycles can emerge for such systems and be exploited for energy-efficient control. A grand challenge in locomotion is trunk stabilization when the hip is offset from the center of mass (CoM). The swing phase plays a major role in this process due to the moment of inertia of the leg; however, many template models ignore the leg mass. In this work, the authors consider a robot hopper model (RHM) with a rigid trunk and leg plus a hip that is displaced from the CoM. It has been previously shown that no passive limit cycle exists for such a model given a linear hip spring. In this work, we show that passive limit cycles can be found when a nonlinear hip spring is used instead. To the authors’ knowledge, this is the first time that a passive limit cycle has been found for this type of system.

Motion control of multi-joint Series Elastic Actuator (SEA)-driven robots still faces challenges including intrinsic oscillatory dynamics, high-order robotic dynamics, low-bandwidth inner loop, and dynamic nonlinearities. In this letter, a nonlinear disturbance observer (NDOB)-based robust controller with the singular perturbation theory is proposed to perform stable and precise motion control of multi-joint SEA-driven robots. First, a fast-time control term is designed according to the singular perturbation theory to stabilize the SEA-level dynamics. Then, for the robot-level dynamics, a NDOB is designed to estimate the effects of unmodeled dynamics and external disturbance. The NDOB is combined with a baseline computed torque controller (CTC) to construct a composite controller NDOB-CTC. In addition, bounded stability is achieved with Lyapunov-type analysis. Finally, the proposed controller was implemented on a 2 DOFs SEA-driven robot. Comparative experiments were conducted for validations.

This paper considers the problem of planning trajectories for a team of sensor-equipped robots to reduce uncertainty about a dynamical process. Optimizing the trade-off between information gain and energy cost (e.g., control effort, distance travelled) is desirable but leads to a non-monotone objective function in the set of robot trajectories. Therefore, common multi-robot planning algorithms based on techniques such as coordinate descent lose their performance guarantees. Methods based on local search provide performance guarantees for optimizing a non-monotone submodular function, but require access to all robots’ trajectories, making it not suitable for distributed execution. This work proposes a distributed planning approach based on local search and shows how lazy/greedy methods can be adopted to reduce the computation and communication of the approach. We demonstrate the efficacy of the proposed method by coordinating robot teams composed of both ground and aerial vehicles with different sensing/control profiles and evaluate the algorithm’s performance in two target tracking scenarios. Compared to the naive distributed execution of local search, our approach saves up to 60% communication and 80–92% computation on average when coordinating up to 10 robots, while outperforming the coordinate descent based algorithm in achieving a desirable trade-off between sensing and energy cost.

In this paper, we address the task of interacting with dynamic environments where the changes in the environment are independent of the agent. We study this through the context of trapping a moving ball with a UR5 robotic arm. Our key contribution is an approach to utilize a static planner for dynamic tasks using a Dynamic Planning add-on; that is, if we can successfully solve a task with a static target, then our approach can solve the same task when the target is moving. Our approach has three key components: an off-the-shelf static planner, a trajectory forecasting network, and a network to predict robot’s estimated time of arrival at any location. We demonstrate the generalization of our approach across environments. More information and videos at https://mlevy2525.github.io/DynamicAddOn/.

Interactions with either environments or expert policies during training are needed for most of the current imitation learning (IL) algorithms. For IL problems with no interactions, a typical approach is Behavior Cloning (BC). However, BC-like methods tend to be affected by distribution shift. To mitigate this problem, we come up with a Robust Model-Based Imitation Learning (RMBIL) framework that casts imitation learning as an end-to-end differentiable nonlinear closed-loop tracking problem. RMBIL applies Neural ODE to learn a precise multi-step dynamics and a robust tracking controller via Nonlinear Dynamics Inversion (NDI) algorithm. Then, the learned NDI controller will be combined with a trajectory generator, a conditional VAE, to imitate an expert’s behavior. Theoretical derivation shows that the controller network can approximate an NDI when minimizing the training loss of Neural ODE. Experiments on Mujoco tasks also demonstrate that RMBIL is competitive to the state-of-the-art generative adversarial method (GAIL) and achieves at least 30% performance gain over BC in uneven surfaces.