ICRA 2020 Papers

Correlation filter-based tracking has been widely applied in unmanned aerial vehicle (UAV) with high efficiency. However, it has two imperfections, i.e., boundary effect and filter corruption. Several methods enlarging the search area can mitigate boundary effect, yet introducing undesired background distraction. Existing frame-by-frame context learning strategies for repressing background distraction nevertheless lower the tracking speed. Inspired by keyframe-based simultaneous localization and mapping, keyfilter is proposed in visual tracking for the first time, in order to handle the above issues efficiently and effectively. Keyfilters generated by periodically selected keyframes learn the context intermittently and are used to restrain the learning of filters, so that 1) context awareness can be transmitted to all the filters via keyfilter restriction, and 2) filter corruption can be repressed. Compared to the state-of-the-art results, our tracker performs better on two challenging benchmarks, with enough speed for UAV real-time applications.

We aim to develop an algorithm for robots to manipulate novel objects as tools for completing different task goals. An efficient and informative representation would facilitate the effectiveness and generalization of such algorithms. For this purpose, we present KETO, a framework of learning keypoint representations of tool-based manipulation. For each task, a set of task-specific keypoints is jointly predicted from 3D point clouds of the tool object by a deep neural network. These keypoints offer a concise and informative description of the object to determine grasps and subsequent manipulation actions. The model is learned from self-supervised robot interactions in the task environment without the need for explicit human annotations. We evaluate our framework in three manipulation tasks with tool use. Our model consistently outperforms state-of-the-art methods in terms of task success rates. Qualitative results of keypoint prediction and tool generation are shown to visualize the learned representations.

Joint torque feedback is a new and promising means of kinesthetic feedback for providing information to a person or guiding them during a motion task. However, little work has been done in determining the psychophysical parameters of how well humans can detect external torques. In this study, we determine the human perceptual ability to detect kinesthetic feedback at the elbow during all possible combinations of preload torques and test stimulus torques, with the elbow in a static posture. To accomplish this, we constructed an exoskeleton for the elbow providing joint torque feedback. The device is designed to convey 0.54 Nm of stall torque for up to 120 seconds via a semi-rigid sleeve structure. Using this device, we assessed perception capability using the Interweaving Staircase Method. We found that users could detect average torques of 0.14-0.18 Nm in the extension or flexion directions with no preload. When a preload of 1.27 Nm was applied, this increased to 0.25-0.27 Nm for when flexion stimuli were applied, and 0.180.3 Nm when extension stimuli were applied, depending on the preload direction.

This paper proposes a novel algorithm for joint space position/torque hybrid control of a mammal-type quadruped robot. With this control algorithm, the robot demonstrated both dynamic locomotion and push reaction abilities without the need for torque control in the ab/ad joints. Based on the tipping and slipping condition of the legged robot, we showed that reaction to a typical push in the horizontal direction does not require full contact-force-control in the frontal plane. Furthermore, we showed that position/torque hybrid control in Cartesian space is directly applicable to joint space hybrid control due to the joint configuration of the quadruped robot. We conducted experiments on our legged robot platform to verify the performance of our hybrid control algorithm. With this approach, the robot displayed stability while walking and reacting to external push disturbances.

Accurate motion control of surgical robots is critical for the efficiency and safety of both state-of-the-art teleoperated robotic surgery and the ultimate autonomous robotic surgery. However, fine motion control for a flexible endoscopic surgical robot is highly challenging because of the shape-dependent and speed-dependent motion hysteresis of tendon-sheath mechanisms (TSMs) in the long, tortuous, and dynamically shape-changing robot body. Aiming to achieve precise closed-loop motion control, we propose a small and flexible sensor to directly sense the large and sharp rotations of the articulated joints of a flexible endoscopic surgical robot. The sensor-a Fiber Bragg Grating (FBG) eccentrically embedded in a thin and flexible epoxy substrate-can be significantly bent with a large bending angle range of [-62.9°, 75.5°] and small bending radius of 6.9 mm. Mounted in-between the two pivot-connected links of a joint, the sensor will bend once the joint is actuated, resulting in the wavelength shift of the FBG. In this study, the relationship between the wavelength shift and the rotation angle of the joint was theoretically modeled and then experimentally verified before and after the installation of the sensor in a robotic endoscopic grasper. The sensor, with the calibrated model, can track the rotation of the robotic joint with an RMSE of 3.34°. This small and flexible sensor has good repeatability, high sensitivity (around 147.5 pm/degree), and low hysteresis (7.72%). It is suitable for surgical robots and manipulators whose articulated joints have a large rotation angle and small bending radius.

The paper presents a pedestrian near-miss detector with temporal analysis that provides both pedestrian detection and risk-level predictions which are demonstrated on a self-collected database. Our work makes three primary contributions: (i) The framework of pedestrian near-miss detection is proposed by providing both a pedestrian detection and risk-level assignment. Specifically, we have created a Pedestrian Near-Miss (PNM) dataset that categorizes traffic near-miss incidents based on their risk levels (high-, low-, and no-risk). Unlike existing databases, our dataset also includes manually localized pedestrian labels as well as a large number of incident-related videos. (ii) Single-Shot MultiBox Detector with Motion Representation (SSD-MR) is implemented to effectively extract motion-based features in a detected pedestrian. (iii) Using the self-collected PNM dataset and SSD-MR, our proposed method achieved +19.38% (on risk-level prediction) and +13.00% (on joint pedestrian detection and risk-level prediction) higher scores than that of the baseline SSD and LSTM. Additionally, the running time of our system is over 50 fps on a graphics processing unit (GPU).

Aiming to understand how human (false-)belief— a core socio-cognitive ability—would affect human interactions with robots, this paper proposes to adopt a graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs. Specifically, a parse graph (pg) is learned from a single-view spatiotemporal parsing by aggregating various object states along the time; such a learned representation is accumulated as the robot’s knowledge. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning and inference capability to overcome the errors originated from a single view. In the experiments, through the joint inference over pgs, the system correctly recognizes human (false-)belief in various settings and achieves better cross-view accuracy on a challenging small object tracking dataset.

We present an end-to-end trainable Neural Network architecture for stereo imaging that jointly locates and estimates human body poses in 3D. Our method defines a 2D pose for each human in a stereo pair of images and uses a correlation layer with a composite field to associate each left-right pair of joints. In absence of a stereo pose dataset, we show that we can train our method with synthetic data only and test it on real-world images (i.e., our training stage is domain invariant). Our method is particularly suitable for autonomous vehicles. We achieve state-of-the-art results for the 3D localization task on the challenging real-world KITTI dataset while running four times faster.

Temporal logic task planning for robotic systems suffers from state explosion when specifications involve large numbers of discrete locations. We provide a novel approach, particularly suited for task specifications with universally quantified locations, that has constant time with respect to the number of locations, enabling synthesis of plans for an arbitrary number of them. We propose a hybrid control framework that uses an iterator to manage the discretised workspace hiding it from a plan enacted by a discrete event controller. A downside of our approach is that it incurs in increased overhead when executing a synthesised plan. We demonstrate that the overhead is reasonable for missions of a fixed-wing Unmanned Aerial Vehicle in simulated and real scenarios for up to 700000 locations.

This paper provides a real time on-board algorithm for a biplane-quadrotor to iteratively learn a forward transition maneuver via repeated flight trials. The maneuver is controlled by regulating the pitch angle and propeller thrust according to feedforward control laws that are parameterized by polynomials. Based on a nominal model with simplified aerodynamics, the optimal coefficients of the polynomials are chosen through simulation such that the maneuver is completed with specified terminal conditions on altitude and air speed. In order to compensate for modeling errors, repeated flight trials are performed by updating the feedforward control parameters according to an iterative learning algorithm until the maneuver is perfected. A geometric attitude controller, valid for all flight modes is employed in order to track the pitch angle according to the feedforward law. Further, a high-fidelity thrust model of the propeller for varying advance-ratio and orientation angle is obtained from wind tunnel data which is captured using a neural network model. This facilitates accurate application of feedforward thrust for varying flow conditions during transition. Experimental flight trials are performed to demonstrate the robustness and rapid convergence of the proposed learning algorithm.

For robots to exhibit a high level of intelligence in the real world, they must be able to assess objects for which they have no prior knowledge. Therefore, it is crucial for robots to perceive object affordances by reasoning about physical interactions with the object. In this paper, we propose a novel method to provide robots with an ability to imagine object affordances using physical simulations. The class of chair is chosen here as an initial category of objects to illustrate a more general paradigm. In our method, the robot "imagines" the affordance of an arbitrarily oriented object as a chair by simulating a physical sitting interaction between an articulated human body and the object. This object affordance reasoning is used as a cue for object classification (chair vs non-chair). Moreover, if an object is classified as a chair, the affordance reasoning can also predict the upright pose of the object which allows the sitting interaction to take place. We call this type of poses the functional pose. We demonstrate our method in chair classification on synthetic 3D CAD models. Although our method uses only 30 models for training, it outperforms appearance-based deep learning methods, which require a large amount of training data, when the upright orientation is not assumed to be known a priori. In addition, we showcase that the functional pose predictions of our method align well with human judgments on both synthetic models and real objects scanned by a depth camera.

This paper describes the development of a tilting locomotion system based on a compliant tensegrity structure with multiple stable equilibrium configurations. A tensegrity structure featuring 4 stable equilibrium states is considered. The mechanical model of the structure is presented and the according equations of motion are derived. The variation of the length of selected structural members allows to influence the prestress state and the corresponding shape of the tensegrity structure. Based on bifurcation analyses a reliable actuation strategy to control the current equilibrium state is designed. In this work, the tensegrity structure is assumed to be in contact with a horizontal plane due to gravity. The derived actuation strategy is utilized to generate tilting locomotion by successively changing the equilibrium state. Numerical simulations are evaluated considering the locomotion characteristics. In order to validate this theoretical approach a prototype is developed. Experiments regarding to the equilibrium configurations, the actuation strategy and the locomotion characteristics are evaluated using image processing tools and motion capturing. The results verify the theoretical data and confirm the working principle of the investigated tilting locomotion system. This approach represents a feasible actuation strategy to realize a reliable tilting locomotion utilizing the multistability of compliant tensegrity structures.

Inverse kinematics is a fundamental challenge for articulated robots: fast and accurate algorithms are needed for translating task-related workspace constraints and goals into feasible joint configurations. In general, inverse kinematics for serial kinematic chains is a difficult nonlinear problem, for which closed form solutions cannot easily be obtained. Therefore, computationally efficient numerical methods that can be adapted to a general class of manipulators are of great importance. In this paper, we use convex optimization techniques to solve the inverse kinematics problem with joint limit constraints for highly redundant serial kinematic chains with spherical joints in two and three dimensions. This is accomplished through a novel formulation of inverse kinematics as a nearest point problem, and with a fast sum of squares solver that exploits the sparsity of kinematic constraints for serial manipulators. Our method has the advantages of post-hoc certification of global optimality and a runtime that scales polynomially with the number of degrees of freedom. Additionally, we prove that our convex relaxation leads to a globally optimal solution when certain conditions are met, and demonstrate empirically that these conditions are common and represent many practical instances. Finally, we provide an open source implementation of our algorithm.

We present an approach for controlling the position of a quadrotor in 3D space using pointing gestures; the task is difficult because it is in general ambiguous to infer where, along the pointing ray, the robot should go. We propose and validate a pragmatic solution based on a push button acting as a simple additional input device which switches between different virtual workspace surfaces. Results of a study involving ten subjects show that the approach performs well on a challenging 3D piloting task, where it compares favorably with joystick control.

In this paper, a new method is proposed to solve the inverse kinematics problem of redundant manipulators. This method demonstrates superior performance on continuous motion by combining interval search genetic algorithm based on trajectory which we propose with parametric joint angle method. In this method, population continuity strategy is utilized to improve search speed and reduce evolutionary generation, interval search strategy is introduced to enhance the search ability and overcome the influence of singularity, and reference point strategy is used to avoid sudden changes of joint variables. By introducing those three strategies, this method is especially suitable for redundant manipulators that perform continuous motion. It can not only obtain solutions of inverse kinematics quickly, but also ensure the motion continuity of manipulator and accuracy of the end effector. Moreover, this algorithm can also perform multi-objective tasks by adjusting the fitness function. Finally, this algorithm is applied to an 8 degree of freedom tunnel shotcrete robot. Field experiments and data analysis show that the algorithm can solve the problem quickly in industrial field, and ensure the motion continuity and accuracy.

In this paper, an Internet of Things-based human-robot collaborative control scheme is developed in Robot-assisted Minimally Invasive Surgery scenario. A hierarchical operational space formulation is designed to exploit the redundancies of the 7-DoFs redundant manipulator to handle multiple operational tasks based on their priority levels, such as guaranteeing a remote center of motion constraint and avoiding collision with a swivel motion without influencing the undergoing surgical operation. Furthermore, the concept of the Internet of Robotic Things is exploited to facilitate the best action of the robot in human-robot interaction. Instead of utilizing compliant swivel motion, HTC VIVE PRO controllers, used as the Internet of Things technology, is adopted to detect the collision. A virtual force is applied to the robot elbow, enabling a smooth swivel motion for human-robot interaction. The effectiveness of the proposed strategy is validated using experiments performed on a patient phantom in a lab setup environment, with a KUKA LWR4+ slave robot and a SIGMA 7 master manipulator. By comparison with previous works, the results show improved performances in terms of the accuracy of the RCM constraint and surgical tip.

To detach a permanent magnet using a control force much smaller than its original attractive force, the internally-balanced magnetic unit (IB Magnet) was invented. It has been applied to magnetic devices such as wall-climbing robots, ceiling-dangling drones, and modular swarm robots. In contrast to its significant reduction rate with regard to the control force, the IB Magnet has two major problems in its nonlinear spring, which serves the purpose of cancelling out the internal force on the magnet. These problems include the complicated design procedure and the trade-off relationship between balancing the precision and the volume of the mechanism. This paper proposes a principle for a new balancing method for the IB Magnet. This method uses a like-pole pair of magnets as a magnetic spring, whose repulsive force should equal the attractive force of an unlike-pole pair. To verify the proposed principle, a prototype of the IB Magnet was designed using a magnetic spring and verified through experiments such that its reduction rate is comparable to those of conventional IB Magnets. Moreover, a robotic clamp was developed as an application example that contains the proposed IB Magnets as its internal mechanism.

In this paper, we present an efficient and robust GPS-aided visual inertial odometry (GPS-VIO) system that fuses IMU-camera data with intermittent GPS measurements. To perform sensor fusion, spatiotemporal sensor calibration and initialization of the transform between the sensor reference frames are required. We propose an online calibration method for both the GPS-IMU extrinsics and time offset as well as a reference frame initialization procedure that is robust to GPS sensor noise. In addition, we prove the existence of four unobservable directions of the GPS-VIO system when estimating in the VIO reference frame, and advocate a state transformation to the GPS reference frame for full observability. We extensively evaluate the proposed approach in Monte-Carlo simulations where we investigate the system's robustness to different levels of GPS noise and loss of GPS signal, and additionally study the hyper-parameters used in the initialization procedure. Finally, the proposed system is validated in a large-scale real-world experiment.

A compass-like biped robot can go down a gentle slope without the need of actuation through a proper choice of its dynamic parameter and starting from a suitable initial condition. Addition of control actions is requested to generate additional gaits and robustify the existing one. This paper designs an interconnection and damping assignment passivity-based control, rooted within the port-Hamiltonian framework, to generate further gaits with respect to state-of-the-art methodologies, enlarge the basin of attraction of existing gaits, and further robustify the system against controller discretization and parametric uncertainties. The performance of the proposed algorithm is validated through numerical simulations and comparison with existing passivity-based techniques.