ICRA 2020 论文列表

In table tennis, the rotation (spin) of the ball plays a crucial role. A table tennis match will feature a variety of strokes. Each generates different amounts and types of spin. To develop a robot that can compete with a human player, the robot needs to detect spin, so it can plan an appropriate return stroke. In this paper we compare three methods to estimate spin. The first two approaches use a high-speed camera that captures the ball in flight at a frame rate of 380 Hz. This camera allows the movement of the circular brand logo printed on the ball to be seen. The first approach uses background difference to determine the position of the logo. In a second alternative, we train a CNN to predict the orientation of the logo. The third method evaluates the trajectory of the ball and derives the rotation from the effect of the Magnus force. This method gives the highest accuracy and is used for a demonstration. Our robot successfully copes with different spin types in a real table tennis rally against a human opponent.

Learning robot behaviors in unstructured environments often requires handcrafting the features for a given task. In this paper, we present and evaluate an unsupervised representation learning architecture, Layered Spatiotemporal Memory Long Short-Term Memory (LSTM-LSTM), that learns the underlying representation without knowledge of the task. The goal of this architecture is to learn the dynamics of the environment from high-dimensional raw video inputs. Using a Generative Adversarial Network (GAN) framework with the proposed network, this architecture is able to learn a spatiotemporal representation in its lower-dimensional latent space directly from raw input sequences. We show that our approach learns the spatial and temporal information simultaneously as opposed to a two-stage learning approach of alternating between training a Convolutional Neural Network (ConvNet) and a Long Short-Term Network (LSTM). Furthermore, by using LSTM-LSTM cells that shrink in size with the increase in the number of layers, the network learns a hierarchical representation with a low-dimensional representation at the top layer. We show that this architecture achieves state-of-the-art results with a substantially lower-dimensional representation than existing methods. We evaluate our approach on a video prediction task with standard benchmark datasets like Moving MNIST and KTH Action, as well as a simulated robot dataset.

Analyzing sensor data of plants and monitoring plant performance is a central element in different agricultural robotics applications. In plant science, phenotyping refers to analyzing plant traits for monitoring growth, for describing plant properties, or characterizing the plant's overall performance. It plays a critical role in the agricultural tasks and in plant breeding. Recently, there is a rising interest in using 3D data obtained from laser scanners and 3D cameras to develop automated non-intrusive techniques for estimating plant traits. In this paper, we address the problem of registering 3D point clouds of the plants over time, which is a backbone of applications interested in tracking spatio-temporal traits of individual plants. Registering plants over time is challenging due to its changing topology, anisotropic growth, and non-rigid motion in between scans. We propose a novel approach that exploits the skeletal structure of the plant and determines correspondences over time and drives the registration process. Our approach explicitly accounts for the non-rigidity and the growth of the plant over time in the registration. We tested our approach on a challenging dataset acquired over the course of two weeks and successfully registered the 3D plant point clouds recorded with a laser scanner forming a basis for developing systems for automated temporal plant-trait analysis.

In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.

High resolution tactile sensors are often bulky and have shape profiles that make them awkward for use in manipulation. This becomes important when using such sensors as fingertips for dexterous multi-fingered hands, where boxy or planar fingertips limit the available set of smooth manipulation strategies. High resolution optical based sensors such as GelSight have until now been constrained to relatively flat geometries due to constraints on illumination geometry. Here, we show how to construct a rounded fingertip that utilizes a form of light piping for directional illumination. Our sensors can replace the standard rounded fingertips of the Allegro hand. They can capture high resolution maps of the contact surfaces, and can be used to support various dexterous manipulation tasks.

Soft strain sensors have been explored as an unobtrusive approach for wearable motion tracking. However, accurate tracking of multi degree-of-freedom (DOF) noncyclic joint movements remains a challenge. This paper presents a soft sensing shirt for tracking shoulder kinematics of both cyclic and random arm movements in 3 DOFs: adduction/abduction, horizontal flexion/extension, and internal/external rotation. The sensing shirt consists of 8 textile-based capacitive strain sensors sewn around the shoulder joint that communicate to a customized readout electronics board through sewn micro-coaxial cables. An optimized sensor design includes passive shielding and demonstrates high linearity and low hysteresis, making it suitable for wearable motion tracking. In a study with a single human subject, we evaluated the tracking capability of the integrated shirt in comparison with a ground truth optical motion capture system. An ensemble-based regression algorithm was implemented in post-processing to estimate joint angles and angular velocities from the strain sensor data. Results demonstrated root mean square errors (RMSEs) less than 4.5° for joint angle estimation and normalized root mean square errors (NRMSEs) less than 4% for joint velocity estimation. Furthermore, we applied a recursive feature elimination (RFE)-based sensor selection analysis to down select the number of sensors for future shirt designs. This sensor selection analysis found that 5 sensors out of 8 were sufficient to generate comparable accuracies.

This paper discusses the design and preliminary evaluation of a soft pneumatic socket (SPS) with real-time pressure regulation and an automated underactuated donning mechanism (UDM). The ability to modulate the pressure at the human-socket interface of a prosthesis or wearable device to accommodate user's activities has the potential to make the user more comfortable. Furthermore, a hands-free, underactuated donning mechanism designed to reliably and safely don the socket onto the user may increase the convenience of prostheses and wearable devices. The pneumatic socket and donning mechanism are evaluated on synthetic forearm model designed to closely match the mechanical properties of the human forearm. The pneumatic socket was tested to determine the maximum loads it can withstand before slipping and the displacement of the socket after loading. The donning mechanism was able to successfully don the socket on to the replica forearm with a 100% success rate for the 30 trials that were tested. Both devices were also tested to determine the pressures they impart on the user. The highest pressures the socket can impart on the user is 4psi and the maximum pressure the donning mechanism imparts on the user is 0.83psi. These pressures were found to be lower than the reported pressures that cause pain and tissue damage.

Soft robotics technology creates new ways for legged robots to interact with and adapt to their environment. In this paper we develop i) a new 2-degree-of-freedom soft pneumatic actuator, and ii) a novel soft robotic hexapedal robot called SoRX that leverages the new actuators. Simulation and physical testing confirm that the proposed actuator can generate cyclic foot trajectories that are appropriate for legged locomotion. Consistent with other hexapedal robots (and animals), SoRX employs an alternating tripod gait to propel itself forward. Experiments reveal that SoRX can reach forward speeds of up to 0.44 body lengths per second, or equivalently 101 mm/s. With a size of 230 mm length, 140 mm width and 100 mm height, and weight of 650 grams, SoRX is among the fastest tethered soft pneumatically-actuated legged robots to date. The motion capabilities of SoRX are evaluated through five experiments: running, step climbing, and traversing rough terrain, steep terrain, and unstable terrain. Experimental results show that SoRX is able to operate over challenging terrains in open-loop control and by following the same alternating tripod gait across all experimental cases.

Learning-based visual navigation still remains a challenging problem in robotics, with two overarching issues: how to transfer the learnt policy to unseen scenarios, and how to deploy the system on real robots. In this paper, we propose a deep neural network based visual navigation system, SnapNav. Unlike map-based navigation or Visual-Teach-and-Repeat (VT&R), SnapNav only receives a few snapshots of the environment combined with directional guidance to allow it to execute the navigation task. Additionally, SnapNav can be easily deployed on real robots due to a two-level hierarchy: a high level commander that provides directional commands and a low level controller that provides real-time control and obstacle avoidance. This also allows us to effectively use simulated and real data to train the different layers of the hierarchy, facilitating robust control. Extensive experimental results show that SnapNav achieves a highly autonomous navigation ability compared to baseline models, enabling sparse, map-less navigation in previously unseen environments.

In this paper, a slip-limiting controller for redundant line-suspended robots is presented. This kind of robot is usually equipped with v-shaped wheels, which brings uncertainty about the effective wheel radius, particularly when crossing obstacles. The proposed algorithm is able to estimate and limit wheel slippage in the presence of such uncertainty, relying only on wheel angular velocity measurements. Slip limitation occurs in the control allocation algorithm and hence is decoupled from the high-level velocity controller, allowing a broad applicability in centralized control approaches. Experimental results on Line Ranger show that it effectively reduces wheel slippage compared to traditional centralized control while being more energy efficient than traditional decentralized control approaches.

Path following accuracy and error convergence with graceful motion in vehicle steering control is challenging due to the competing nature of these requirements, especially across a range of operating speeds. This work is founded upon slip-based kinematic and dynamic models, which allow derivation of controllers considering error due to sideslip and the mapping between steering commands and graceful lateral motion. A novel recursive backstepping steering controller is proposed that better couples yaw-rate based path following commands to steering angle and rate. Observer based sideslip estimates are combined with heading error in the kinematic controller to provide feedforward slip compensation. Path following error is compensated by a Variable Structure Controller (VSC) to balance graceful motion, path following error, and robustness. Yaw rate commands are used by a backstepping dynamic controller to generate robust steering commands. A High Gain Observer (HGO) estimates sideslip and yaw rate for output feedback control. Stability analysis is provided and peaking is addressed. Field experimental results evaluate the work and provide comparisons to MPC.

Motion control of underactuated systems through the inverse dynamics contains configuration singularities. These limitations in configuration space mainly stem from the inertial coupling that passive joints/bodies create. In this study, we present a model that is free from singularity while the trajectory of the rotating mass has a small-amplitude sine wave around its circle. First, we derive the modified non-linear dynamics for a rolling system. Also, the singularity regions for this underactuated system is demonstrated. Then, the wave parameters are designed under certain conditions to remove the coupling singularities. We obtain these conditions from the positive definiteness of the inertia matrix in the inverse dynamics. Finally, the simulation results are confirmed by using a prescribed Beta function on the specified states of the rolling carrier. Because our algebraic method is integrated into the non-linear dynamics, the proposed solution has a great potential to be extended to the Lagrangian mechanics with multiple degrees-of-freedom.

The 3-CRS manipulator is an original parallel mechanism having 6 degrees of freedom (DOFs) with only 3 limbs. This mechanism uses a motorized cylindrical joint per limb. This new paradigm of actuation opens research fields on new families of robots that should particularly interest the parallel robotics community. According to its dimensional synthesis, this mechanism can have remarkable kinematic properties such as a large orientation workspace or reconfiguration capabilities. In this paper, we introduce this mechanism and we study its singularities by using a geometric approach. This approach simplifies considerably singularity analysis problem by considering the relative geometric configurations of three planes defined by the distal links of the limbs. Thanks to that, a reconfiguration mode of the 3-CRS, that doubles its reachable workspace, is highlighted. This property is illustrated on a physical prototype of the robot.

Surgical robots are used to perform minimally invasive surgery and alleviate much of the burden imposed on surgeons. Our group has developed a surgical robot to aid in the removal of tumors at the base of the skull via access through the nostrils. To avoid injuring the patients, a collision-avoidance algorithm that depends on having an accurate model for the poses of the instruments' shafts is used. Given that the model's parameters can change over time owing to interactions between instruments and other disturbances, the online estimation of the poses of the instrument's shaft is essential. In this work, we propose a new method to estimate the pose of the surgical instruments' shafts using a monocular endoscope. Our method is based on the use of an automatically annotated training dataset and an improved pose-estimation deep-learning architecture. In preliminary experiments, we show that our method can surpass state of the art vision-based marker-less pose estimation techniques (providing an error decrease of 55% in position estimation, 64% in pitch, and 69% in yaw) by using artificial images.

In this paper, we introduce a novel single shot approach for 6D object pose estimation of rigid objects based on depth images. For this purpose, a fully convolutional neural network is employed, where the 3D input data is spatially discretized and pose estimation is considered as a regression task that is solved locally on the resulting volume elements. With 65 fps on a GPU, our Object Pose Network (OP-Net) is extremely fast, is optimized end-to-end, and estimates the 6D pose of multiple objects in the image simultaneously. Our approach does not require manually 6D pose-annotated real-world datasets and transfers to the real world, although being entirely trained on synthetic data. The proposed method is evaluated on public benchmark datasets, where we can demonstrate that state-of-the-art methods are significantly outperformed.

In this work, we present the concept, design, and initial testing of a single actuator peristaltic motion robot for subsurface geological exploration and device emplacement. We are researching unconventional methods, including robotics, for the production of energy from oil reservoirs that do not liberate carbon to the atmosphere. For such application, we are developing autonomous robots for data acquisition and tool transportation inside petroleum reservoirs. The mechanism described in this work is a cam-follower configuration worm robot that utilizes peristaltic displacement. We confirmed that the mechanism works on a plane surface and in non-consolidated media.

The coordination of multiple robots operating simultaneously in the same workspace requires the integration of task allocation and motion scheduling. We focus on tasks in which the robot's actions are not confined to small volumes, but can also occupy a large time-varying portion of the workspace, such as in welding along a line. The optimization of such tasks presents a considerable challenge mainly due to the fact that different variants of task execution exist, for instance, there can be multiple starting points of lines or closed curves, differentfilling patterns of areas, etc. We propose a generic and computationally efficient optimization method which is based on constraint programming. It takes into account the kinematics of the robots and guarantees that the motions of the robots are collision-free while minimizing the overall makespan. We evaluate our approach on several use-cases of varying complexity: cutting, additive manufacturing, spot welding, inserting and tightening bolts, performed by a dual-arm robot. In terms of the makespan, the result is superior to task execution by one robot arm as well as by two arms not working simultaneously.

In this paper, we propose a novel method to simultaneously track the deformation of soft objects and estimate their elasticity parameters. The tracking of the deformable object is performed by combining the visual information captured by a RGB-D sensor with interactive Finite Element Method simulations of the object. The visual information is more particularly used to distort the simulated object. In parallel, the elasticity parameter estimation minimizes the error between the tracked object and a simulated object deformed by the forces that are measured using a force sensor. Once the elasticity parameters are estimated, our tracking algorithm can be used to estimate the deformation forces applied to an object without the use of a force sensor. We validated our method on several soft objects with different shape complexities. Our evaluations show the ability of our method to estimate the elasticity parameters as well as its use to estimate the forces applied to a deformable object without any force sensor. These results open novel perspectives to better track and control deformable objects during robotic manipulations.

The development of algorithms for motion discrimination in home rehabilitation sessions poses numerous challenges. Recent studies have used the concept of synergies to discriminate a set of movements. However, the discrimination depends on the correlation of the reconstructed movement with the online data, and the training data requires well-defined movements. In this paper, we introduced the concept of a synergy probe, which makes a direct comparison between synergies and online data. The system represents synergies and movements in the same space and monitors their behavior. The results indicated that conventional methods are influenced by the segmentation of training data, and even though the reconstructed movement is similar to the ground-truth, it does not provide sufficient information to evaluate the data in real time. The synergy probes were used to discriminate and evaluate the performance of natural whole-body exercises without segmentation or previous determination of movements. An analysis of the results also demonstrated the possibility to identify the strategies used by the subjects for movement. Such information aids in gaining a better insight and can prove beneficial in home rehabilitation.