ICRA 2020 Papers

Motion retargeting between heterogeneous polymorphs with different sizes and kinematic configurations requires a comprehensive knowledge of (inverse) kinematics. Moreover, it is non-trivial to provide a kinematic independent general solution. In this study, we developed a cyclic three-phase optimization method based on deep reinforcement learning for human-robot motion retargeting. The motion retargeting learning is performed using refined data in a latent space by the cyclic and filtering paths of our method. In addition, the human- in-the-loop based three-phase approach provides a framework for the improvement of the motion retargeting policy by both quantitative and qualitative manners. Using the proposed C- 3PO method, we were successfully able to learn the motion retargeting skill between the human skeleton and motion of the multiple robots such as NAO, Pepper, Baxter and C-3PO.

Fall-related injury is a significant health problem on a global scale and is expected to grow with the aging population. Laboratory-based perturbation systems have the capability of simulating various modes of fall-inducing perturbations in a repeatable way. These systems enable fundamental research on human gait and balance and facilitate the development of devices to assist human balance. We present a robotic, rope-driven system capable of rendering bumps and force-fields at a person's pelvis in any direction in the transverse plane with forces up to 200 N, and a 90% rise time of as little as 44 ms, which is faster than a human's ability to sense and respond to the force. These capabilities enable experiments that require stabilizing or destabilizing subjects as they stand or walk on a treadmill. To facilitate use by researchers from all backgrounds, we designed both a configuration with simpler open-loop force control, and another with higher-performance, closed-loop force control. Both configurations are modular, and the open-loop system is made entirely from 3D-printed and catalog components. The design files and assembly instructions for both are freely available in an online repository.

Autonomous driving is a dynamically growing field of research, where quality and amount of experimental data is critical. Although several rich datasets are available these days, the demands of researchers and technical possibilities are evolving. Through this paper, we bring a new dataset recorded in Brno - Czech Republic. It offers data from four WUXGA cameras, two 3D LiDARs, inertial measurement unit, infrared camera and especially differential RTK GNSS receiver with centimetre accuracy which, to the best knowledge of the authors, is not available from any other public dataset so far. In addition, all the data are precisely timestamped with submillisecond precision to allow wider range of applications. At the time of publishing of this paper, recordings of more than 350 km of rides in varying environment are shared at: https://github.com/RoboticsBUT/Brno-Urban-Dataset.

This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian impedance controllers cascaded by a null space projector. The contact forces acting on supporting feet are optimized by quadratic programming (QP). The foot's Cartesian impedance controller may also estimate contact forces from trajectory tracking errors, and relay the force-feedback to the operator. A 7D haptic joystick, Sigma.7, transmits motion commands to the quadruped robot ANYmal, and renders the force feedback. Furthermore, the joystick's motion is bounded by mapping the foot's feasible force polytope constrained by the friction cones and torque limits in order to prevent the operator from driving the robot to slipping or falling over. Experimental results demonstrate the efficiency of the proposed framework.

The variable-height inverted pendulum (VHIP) model enables a new balancing strategy by height variations of the center of mass, in addition to the well-known ankle strategy. We propose a biped stabilizer based on linear feedback of the VHIP that is simple to implement, coincides with the state-of-the-art for small perturbations and is able to recover from larger perturbations thanks to this new strategy. This solution is based on "best-effort" pole placement of a 4D divergent component of motion for the VHIP under input feasibility and state viability constraints. We complement it with a suitable whole-body admittance control law and test the resulting stabilizer on the HRP-4 humanoid robot.

All animals and robots that move in the world must navigate to a goal while clearing obstacles. Using vision to accomplish such task has several advantages in cost and payload, which explains the prevalence of biological visual guidance. However, the computational overhead has been an obvious concern when increasing number of pixels and frames that need to be analyzed in real-time for a machine vision system. The use of motion vision and optic flow has been a popular bio-inspired solution for this problem. However, many early-stage motion detection approaches rely on special hardware (e.g. event-cameras) or extensive computation (e.g. dense optic flow map). Here we demonstrate a method to combine an insect vision inspired object motion filter model with simple visual guidance rules to fly through a cluttered environment. We have implemented a complete feedback control loop in a micro racing drone and achieved proximaldistal object separation through only two object motion filters. We discuss the key constraints and the scalability of this approach for future development.

This paper presents a method to create fish-like robots with tensegrity systems and describes a prototype modeled on the body shape of the rainbow trout with a length of 400 mm and a mass of 102 g that is driven by a waterproof servomotor. The structure of the tensegrity robot consists of rigid body segments and elastic cables that represent bone/tissue and muscles of fish, respectively. This structural configuration employing the tensegrity class 2 is much simpler than other tensegrity-based underwater robots. It also allows the tuning of the mechanical stiffness, which is often said to be an important factor in fish swimming. In our robot, the body stiffness can be tuned by changing the cross-section of the cables and their pre-stretch ratio. We characterize the robot in terms of body stiffness, swimming speed, and thrust force while varying the body stiffness i.e., the cross-section of the elastic cables. The results show that the body stiffness of the robot can be designed to approximate that of the real fish and modulate its performance characteristics. The measured swimming speed of the robot is 0.23 m/s (0.58 BL/s), which is comparable to other fish robots of the same type. Strouhal number of the robot 0.54 is close to that of the natural counterpart, suggesting that the presented method is an effective engineering approach to realize the swimming characteristics of real fish.

In this paper we propose an algorithm to actively control the distance of a motor-propeller system (MPS) to a large obstacle using data from a single microphone. The method is based upon a broadband constructive/destructive interference pattern across the audible frequency band that is present when the MPS is near an obstacle. By taking the difference between the power spectrum in the obstacle-free case and the spectrum when recording near an obstacle, a broadband oscillation with respect to frequency is revealed. The frequency of this oscillation is linearly-related to the distance from the microphone to the wall. We present both static and dynamic experiments showcasing the ability of the proposed method to estimate the distance to a wall as well as actively control it.

Driveable area detection is a key component for various applications in the field of autonomous driving (AD), such as ground-plane detection, obstacle detection and maneuver planning. Additionally, bulky and over-parameterized networks can be easily forgone and replaced with smaller networks for faster inference on embedded systems. The driveable area detection, posed as a two class segmentation task, can be efficiently modeled with slim binary networks. This paper proposes a novel binarized driveable area detection network (binary DAD-Net), which uses only binary weights and activations in the encoder, the bottleneck, and the decoder part. The latent space of the bottleneck is efficiently increased (×32→×16 downsampling) through binary dilated convolutions, learning more complex features. Along with automatically generated training data, the binary DAD-Net outperforms state-of-the-art semantic segmentation networks on public datasets. In comparison to a full-precision model, our approach has a ×14.3 reduced compute complexity on an FPGA and it requires only 0.9MB memory resources. Therefore, commodity SIMD-based AD-hardware is capable of accelerating the binary DAD-Net.

In this paper, a bilateral teleoperation control of a serial robot manipulator, which guarantees a Remote Center of Motion (RCM) constraint in its kinematic level, is developed. A two-layered approach based on the energy tank model is proposed to achieve haptic feedback on the end effector with a pedal switch. The redundancy of the manipulator is exploited to maintain the RCM constraint using the decoupled Cartesian Admittance Control. Transparency and stability of the proposed bilateral teleoperation are demonstrated using a KUKA LWR4+ serial robot and a Sigma 7 haptic manipulator with an RCM constraint in augmented reality. The results prove that the control can achieve not only the bilateral teleoperation but also maintain the RCM constraint.

Battery life, reliability, and localization are prominent challenges in the design of autonomous underwater vehicles (AUVs). This work aims to address facets of these challenges using a single system. We describe the design of a bidirectional resonant pump that uses a single electromagnetic voice coil motor (VCM) capable of rotation around a central two degree-of-freedom flexure stage axis. This actuator design produces highly efficient resonant motion that drives two orthogonally oriented diaphragms simultaneously. The operation of this diaphragm pump mechanism produces both adjustable thrust vectors at the aft surface of the AUV and a monotonic relationship between thrust vectors and operating frequency. We propose using the unique frequency to thrust relationship to enhance AUV localization capabilities. We construct a prototype and use it to experimentally demonstrate the feasibility of the directionally-tunable resonance concept.

Correlation filters (CFs) have shown excellent performance in unmanned aerial vehicle (UAV) tracking scenarios due to their high computational efficiency. During the UAV tracking process, viewpoint variations are usually accompanied by changes in the object and background appearance, which poses a unique challenge to CF-based trackers. Since the appearance is gradually changing over time, an ideal tracker can not only forward predict the object position but also backtrack to locate its position in the previous frame. There exist response-based errors in the reversibility of the tracking process containing the information on the changes in appearance. However, some existing methods do not consider the forward and backward errors based on while using only the current training sample to learn the filter. For other ones, the applicants of considerable historical training samples impose a computational burden on the UAV. In this work, a novel bidirectional incongruity-aware correlation filter (BiCF) is proposed. By integrating the response-based bidirectional incongruity error into the CF, BiCF can Efficiently learn the changes in appearance and suppress the inconsistent error. Extensive experiments on 243 challenging sequences from three UAV datasets (UAV123, UAVDT, and DTB70) are conducted to demonstrate that BiCF favorably outperforms other 25 state-of-the-art trackers and achieves a real-time speed of 45.4 FPS on a single CPU, which can be applied in UAV Efficiently.

Autonomous cars and fixed-wing aerial vehicles have the so-called non-holonomic kinematics which non-linearly maps control input to states. As a result, trajectory optimization with such a motion model becomes highly non-linear and non-convex. In this paper, we improve the computational tractability of non-holonomic trajectory optimization by reformulating it in terms of a set of bi-convex cost and constraint functions along with a non-linear penalty. The bi-convex part acts as a relaxation for the non-holonomic trajectory optimization while the residual of the penalty dictates how well its output obeys the non-holonomic behavior. We adopt an alternating minimization approach for solving the reformulated problem and show that it naturally leads to the replacement of the challenging non-linear penalty with a globally valid convex surrogate. Along with the common cost functions modeling goal-reaching, trajectory smoothness, etc., the proposed optimizer can also accommodate a class of non-linear costs for modeling goal-sets, while retaining the bi-convex structure. We benchmark the proposed optimizer against off-the-shelf solvers implementing sequential quadratic programming and interior-point methods and show that it produces solutions with similar or better cost as the former while significantly outperforming the latter. Furthermore, as compared to both off-the-shelf solvers, the proposed optimizer achieves more than 20x reduction in computation time.

Current end-to-end grasp planning methods propose grasps in the order of seconds that attain high grasp success rates on a diverse set of objects, but often by constraining the workspace to top-grasps. In this work, we present a method that allows end-to-end top-grasp planning methods to generate full six-degree-of-freedom grasps using a single RGBD view as input. This is achieved by estimating the complete shape of the object to be grasped, then simulating different viewpoints of the object, passing the simulated viewpoints to an end-to-end grasp generation method, and finally executing the overall best grasp. The method was experimentally validated on a Franka Emika Panda by comparing 429 grasps generated by the state-of-the-art Fully Convolutional Grasp Quality CNN, both on simulated and real camera images. The results show statistically significant improvements in terms of grasp success rate when using simulated images over real camera images, especially when the real camera viewpoint is angled. Code and video are available at https://irobotics.aalto.fi/beyond-topgrasps-through-scene-completion/.

Pose estimation and map building are central ingredients of autonomous robots and typically rely on the registration of sensor data. In this paper, we investigate a new metric for registering images that builds upon on the idea of the photometric error. Our approach combines a gradient orientation-based metric with a magnitude-dependent scaling term. We integrate both into stereo estimation as well as visual odometry systems and show clear benefits for typical disparity and direct image registration tasks when using our proposed metric. Our experimental evaluation indicate that our metric leads to more robust and more accurate estimates of the scene depth as well as camera trajectory. Thus, the metric improves camera pose estimation and in turn the mapping capabilities of mobile robots. We believe that a series of existing visual odometry and visual SLAM systems can benefit from the findings reported in this paper.

We contribute a study benchmarking the performance of multiple motion-based learning from demonstration approaches. Given the number and diversity of existing methods, it is critical that comprehensive empirical studies be performed comparing the relative strengths of these techniques. In particular, we evaluate four approaches based on properties an end user may desire for real-world tasks. To perform this evaluation, we collected data from nine participants, across four manipulation tasks. The resulting demonstrations were used to train 180 task models and evaluated on 720 task reproductions on a physical robot. Our results detail how i) complexity of the task, ii) the expertise of the human demonstrator, and iii) the starting configuration of the robot affect task performance. The collected dataset of demonstrations, robot executions, and evaluations are publicly available. Research insights and guidelines are also provided to guide future research and deployment choices about these approaches.

In recent years, graph neural networks have been successfully applied for learning the dynamics of complex and partially observable physical systems. However, their use in the robotics domain is, to date, still limited. In this paper, we introduce Belief Regulated Dual Propagation Networks (BRDPN), a general-purpose learnable physics engine, which enables a robot to predict the effects of its actions in scenes containing groups of articulated multi-part objects. Specifically, our framework extends recently proposed propagation networks (PropNets) and consists of two complementary components, a physics predictor and a belief regulator. While the former predicts the future states of the object(s) manipulated by the robot, the latter constantly corrects the robot's knowledge regarding the objects and their relations. Our results showed that after training in a simulator, the robot can reliably predict the consequences of its actions in object trajectory level and exploit its own interaction experience to correct its belief about the state of the environment, enabling better predictions in partially observable environments. Furthermore, the trained model was transferred to the real world and verified in predicting trajectories of pushed interacting objects whose joint relations were initially unknown. We compared BRDPN against PropNets, and showed that BRDPN performs consistently well. Moreover, BRDPN can adapt its physic predictions, since the relations can be predicted online.

In many cases the multi-robot systems are desired to execute simultaneously multiple behaviors with different controllers, and sequences of behaviors in real time, which we call behavior mixing. Behavior mixing is accomplished when different subgroups of the overall robot team change their controllers to collectively achieve given tasks while maintaining connectivity within and across subgroups in one connected communication graph. In this paper, we present a provably minimum connectivity maintenance framework to ensure the subgroups and overall robot team stay connected at all times while providing the highest freedom for behavior mixing. In particular, we propose a real-time distributed Minimum Connectivity Constraint Spanning Tree (MCCST) algorithm to select the minimum inter-robot connectivity constraints preserving subgroup and global connectivity that are least likely to be violated by the original controllers. With the employed safety and connectivity barrier certificates for the activated connectivity constraints and collision avoidance, the behavior mixing controllers are thus minimally modified from the original controllers. We demonstrate the effectiveness and scalability of our approach via simulations of up to 100 robots with multiple behaviors.

Deep learning has enjoyed much recent success, and applying state-of-the-art model learning methods to controls is an exciting prospect. However, there is a strong reluctance to use these methods on safety-critical systems, which have constraints on safety, stability, and real-time performance. We propose a framework which satisfies these constraints while allowing the use of deep neural networks for learning model uncertainties. Central to our method is the use of Bayesian model learning, which provides an avenue for maintaining appropriate degrees of caution in the face of the unknown. In the proposed approach, we develop an adaptive control framework leveraging the theory of stochastic CLFs (Control Lyapunov Functions) and stochastic CBFs (Control Barrier Functions) along with tractable Bayesian model learning via Gaussian Processes or Bayesian neural networks. Under reasonable assumptions, we guarantee stability and safety while adapting to unknown dynamics with probability 1. We demonstrate this architecture for high-speed terrestrial mobility targeting potential applications in safety-critical high-speed Mars rover missions.