ICRA 2021 Papers

High resolution and large range force/torque (F/T) measurements are usually required in many engineering tasks. However, most existing F/T sensors only have a fixed resolution over their whole ranges. The key lies in that it is difficult to well balance high resolution and large range in the sensor design. Taking the torque sensor for example, this paper presents a better compromise for this problem i.e., a novel variable resolution torque sensor based on variable stiffness principle. From the structural points of view, the sensor is constructed with multiple radial flexures to achieve a pure rotational motion with negligible parasitic center motions. Two resistive strain gauges (RSGs) are selected as the measuring units of the sensor to detect the applied external torque and meanwhile provide variable resolutions in the two different measuring ranges (each RSG for one range). Static and dynamic models of the sensor are established in details and validated through finite element analysis (FEA) to evaluate its characteristics. A principle prototype is finally fabricated and tested to verify the effectiveness of the presented design. RSGs are calibrated through a commercial six-axis F/T sensor from ATI Industrial Automation, Inc. Experimental results show that the torque sensor can provide high and low resolutions in the small and large ranges respectively and possesses the first natural frequency of 67.3 Hz. In addition, the proposed variable resolution method can also be applied to the development of multi-axis F/T sensors.

Torsional actuators are a class of artificial muscle technology that generates torque and produces rotary motion in response to various stimuli. This paper presents a novel torsional actuator combining an origami-inspired twisting skeleton and an artificial muscle. The process of torsional actuator design starts from identifying a foldable twisting skeleton which is capable of achieving helical motion thereby translating linear motion to rotational motion. This is followed by the integration of an artificial muscle to drive the twisting skeleton. Kinematics of both the twisting skeleton and artificial muscle are analyzed. Following the design and kinematic analysis, a prototype is developed by bonding 3D printed polylactic acid (PLA) parts and thermoplastic polyurethane (TPU) films to form the twisting skeleton and laminating TPU membranes by using heat sealing tools to form the artificial muscle. A pneumatic control system is built to evaluate the performances of torsional actuator by testing the relationship between twisting angle, air pressure, driving force and output torque. Experimental results show that the relationship between air pressure, driving force and output torque is proportional at a given twisting angle. The novel torsional actuator augmenting an origami-inspired skeleton and soft artificial muscle leads to simplified analytical model and has potential of driving robotic systems in environment where pneumatically actuated systems are preferred over electrical machines and drives.

Tactile perception on our fingers is a key sensory feedback that enables us to perceive and explore our world using our hands as probes, and is essential for efficient gripping and manipulation of objects. A tactile feedback system can therefore greatly improve the quality of life of individuals with partial or complete sensory loss like during stroke, or with artificial limbs after an amputation. However, most existing tactile texture feedback technologies suffer from two constraints. First, texture decoding and texture feedback have been traditionally examined separately and not as parts of the same problem, and second, texture information has been popularly fed back using sensory modality other than tactile itself. In this study, we propose a prototype on-line direct-texture decoding and feedback system in which the texture touched by a user is decoded using an accelerometer attached to the finger. The feedback is realized by rubbing the user’s skin with the actual material and the speed of the user swipes. The efficacy of the proposed system was tested in two user experiments with five test materials. The results and the corresponding hints for future improvements are discussed.

In this paper, we present an autonomous RGB-D and 2D ultrasound-guided robotic system for collecting 3D localized volumes of peripheral vessels. This compact design, with available commercial components, lends itself to platform utility throughout the human body. The fully integrated system works with force limits for future safety in human use. We propose a PID force controller for smooth and safe robot scanning following a priori 3D trajectory generated from a surface point cloud. System calibration is implemented to determine transformations among sensors, end-effector and robot base. A vascular localization pipeline that consists of detection and tracking is proposed to find the 3D vessel positions in real-time. Precision tests are performed with both predesignated and autonomously selected areas in an arm phantom. The average variance of the autonomously collected ultrasound images (to construct 3D volumes) between repeated tests is shown to be around 0.3 mm, similar to the theoretical spatial resolution a clinical ultrasound system. This fully integrated system demonstrates the capability of autonomous collection of peripheral vessels with built-in safety measures for future human testing.

Dynamic locomotion for legged robots is difficult because the system dynamics are highly nonlinear and complex, nominally underactuated and unstable, multi-input and multi-output, as well as time-variant and hybrid. One usually faces the choice between the intricate full-body dynamics which remains computationally expensive and sometimes even intractable, and the empirically simplified model which inevitably limits the locomotion capability. In this paper, we explore the legged robot dynamics from a different perspective. By decomposing the robot into the body and the legs, with interaction forces and moments connecting them, we enjoy a novel method called Dynamic Model Decomposition that involves lower-dimensional dynamics for each subsystem while their composition maintaining the equivalence to the original full-order robot model. Based on that, we further propose a corresponding model predictive control framework via quadratic programming, which con-siders linearly approximated body dynamics with constrained leg reaction forces as inputs. The overall methodology was successfully applied to a planar five-link biped robot. The simulation results show that the robot is capable of body reference tracking, push recovery, velocity tracking, and even blind locomotion on fairly rough terrain. This suggests a promising dynamic motion control scheme in the future.

The control of complex autonomous systems has significantly improved in recent years and unmanned aerial vehicles (UAVs) have become popular in the research community. Although the use of UAVs is increasing, much work remains to guarantee fault- tolerant control (FTC) properties of these vehicles. Model-based controllers are the standard way to control UAVs, however obtaining models of the system and environment for every possible operating condition a UAV can experience in a real-world scenario is not feasible. Reinforcement Learning has shown promise in controlling complex systems but requires training in a simulator (requiring a model) of the system. Further, stability guarantees do not exist for learning-based controllers, which limits their large scale application in the real-world. We propose a novel hybrid FTC approach that uses a learned supervisory controller (together with low-level PID controllers) with key stability guarantees. We use a robust reinforcement learning approach to learn the supervisory control parameters and prove stability. We empirically validate our framework using trajectory-following experiments (in simulation) for a quadcopter subject to rotor faults, wind disturbances, and severe position and attitude noise.

As a life-threatening disease, stroke can lead to long-term problems affecting the patients’ daily living ability. A common problem facing post-stroke patients is foot drop. An emerging modality of interest for correcting the foot drop is to combine both actuated ankle-foot orthosis (AAFO) and functional electrical stimulation (FES). Such hybrid assistive system not only ensure effective assistance but also can avoid fast muscular fatigue due to excessive muscular stimulation. Due to the significant changes in the ankle joint’s kinematics and kinetics with gait cycles, optimization control strategies for hybrid AAFO and FES systems are highly demanded. However, it is challenging to develop accurate gait phase detection algorithms to guide the control of AAFO and FES while ensuring robustness with respect to the diversity and variability of patients’ gaits. In this paper, we present a novel swing sub-phase detection algorithm based on a moving average convergence divergence (MACD) indicator. The proposed detection algorithm uses only information collected from the affected leg by means of two inertia measurement units (IMU) and the AAFO. Moreover, a gait-phase based control strategy is developed to optimize the assistive effect of a hybrid AAFO and FES system. Experimental results with five healthy show the potential of the proposed approaches in ensuring both satisfactory ankle joint trajectory tracking and effective reduction in stimulation intensity, compared to the use of conventional FES assistance.

Existing radar sensors can be classified into automotive and scanning radars. While most radar odometry (RO) methods are only designed for a specific type of radar, our RO method adapts to both scanning and automotive radars. Our RO is simple yet effective, where the pipeline consists of thresholding, probabilistic submap building, and an Normal Distribution Transform-based (NDT-based) radar scan matching. The proposed RO has been tested on two public radar datasets: the Oxford Radar RobotCar dataset and the nuScenes dataset, which provide scanning and automotive radar data respectively. The results show that our approach surpasses state-of-the-art RO using either automotive or scanning radar by reducing translational error by 51% and 30%, respectively, and rotational error by 17% and 29%, respectively. Besides, we show that our RO achieves centimeter-level accuracy as lidar odometry, and automotive and scanning RO have similar accuracy.

This paper reports on a novel nonparametric rigid point cloud registration framework, Semantic Continuous Visual Odometry (CVO), that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizing the inner product between two functions, essentially a sum of weighted kernels, each of which exploits the local geometric and semantic features. As a result of the continuous models, analytical gradients can be computed, and a local solution can be obtained by optimization over the rigid body transformation group. Besides, we present a new point cloud alignment metric that is intrinsic to the proposed framework and takes into account geometric and semantic information. The evaluations using publicly available stereo and RGB-D datasets show that the proposed method outperforms state-of-the-art outdoor and indoor frame-to-frame registration methods. An open-source GPU implementation is also provided.

Visible images have been widely used for motion estimation. Thermal images, in contrast, are more challenging to be used in motion estimation since they typically have lower resolution, less texture, and more noise. In this paper, a novel dataset for evaluating the performance of multi-spectral motion estimation systems is presented. All the sequences are recorded from a handheld multi-spectral device. It consists of a standard visible-light camera, a long-wave infrared camera, an RGB-D camera, and an inertial measurement unit (IMU). The multi-spectral images, including both color and thermal images in full sensor resolution (640 × 480), are obtained from a standard and a long-wave infrared camera at 32Hz with hardware-synchronization. The depth images are captured by a Microsoft Kinect2 and can have benefits for learning cross-modalities stereo matching. For trajectory evaluation, accurate ground-truth camera poses obtained from a motion capture system are provided. In addition to the sequences with bright illumination, the dataset also contains dim, varying, and complex illumination scenes. The full dataset, including raw data and calibration data with detailed data format specifications, is publicly available.

The problem of safe interception of multiple intruder UAVs by a team of cooperating autonomous aerial vehicles is addressed in this paper. The presented work is motivated by the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where this task was simplified to an interaction with a set of static and dynamic objects (balloons and a UAV), and by a real autonomous aerial interception system of Eagle.One that our team has been working on. We propose a general control, perception, and coordination system for the fast and reliable interception of targets in a 3D environment relying only on onboard sensors and processing. The proposed methods and the entire complex multi-robot system were successfully verified in demanding desert conditions, with the main focus on reliability and fast deployment. In the MBZIRC competition, the proposed approach exhibited the greatest reliability and fastest solution. It was crucial to our team in winning the entire competition and achieving the second place in the intruder UAV interception scenario.

Although multi-person human pose estimation has made great progress in recent years, the challenges such as various scales of persons, occluded keypoints, and crowded backgrounds in complex scenes are still remained to be solved. In this paper, we propose a novel multi-level pose estimation network (MLPE) to learn multi-level features that can preserve both the strong semantic clues and spatial resolution for keypoint prediction and location. More specifically, a multi-level prediction network with a feature enhancement strategy is first proposed to learn multi-level features to achieve a good trade-off between the global context information and spatial resolution. We then build a high-resolution fine network to restore high spatial resolution information based on transposed convolutions to accurately locate the keypoints. We have conducted extensive experiments on the challenging MS COCO dataset, which has proved the effectiveness of our proposed method. Code † and the experimental results are publicly online available for further research.

For a multi-fingered robot hand, the individual control over single joints cannot guarantee their fine collaboration. For achieving a high-precision synchronization, a theory of synchronous control is introduced to multi-fingered robot hands. This paper introduced a new model-free and cross-coupling control strategy. It had been tested on the humanoid robot fingers and showed high positioning performance. For realizing the mutual influence between the control of all joints, we establish the synchronization error by the differential disposal of adjacent actuator errors, then position errors and synchronization errors are incorporated into a unified control frame. Meanwhile, considering the complex dynamic formulations of the dexterous hand and the characteristics of the control system, a model-free, cross-coupled trajectory tracking method is introduced and the explicit dynamic modeling parameters is not necessary. Finally, we tested our method on a multi-fingered hand platform HIT/DLR-II. The results prove that the new method has superior performance over traditional non-synchronous approaches.

During the COVID-19 pandemic, due to the unprecedented workload and cross-infection hazard, the health-care workers’ lives are under a significant threat. However, minimizing the duration and frequency of close clinician-to-patient contacts using simple technologies that enable physical distancing could reduce the risk of spreading the disease. In this context, this paper presents the conceptual design and preliminary assessment of a low-cost and intrinsically safe remote service delivery platform that can assist clinicians in doing various tasks at a safe distance from patients. This mechanism is capable of manipulating objects in three-dimensional Cartesian space and can be adapted to handling a wide variety of medical devices. Moreover, its passive weight-compensating design provides the mechanism with high maneuverability, enhanced dynamic manipulability, and better force feedback quality. The advantages and effectiveness of the proposed mechanism are demonstrated through experiments. In the experiment, an ultrasound probe is mounted at the end effector of the device to perform an imaging task from a safe distance. Due to the existence of the force feedback, the user could remotely manipulate the ultrasound probe for having a successful vertical and pivot scanning to get high-quality images with a low physical and mental demand.

Flexible flat cables (FFC) are very popular for connecting different components in modern electronics (e.g., mobile phones, laptops, tablets, etc.). The manipulation of FFCs typically relies on highly trained workers that spend hours performing the same repetitive processes, or on autonomous robotic systems that are equipped with simple clamping mechanisms or pneumatically driven suction cups. Such robotic systems are difficult to program and reprogram and often rely on sophisticated sensing elements and complicated control laws. Moreover, the performance and robustness of such systems is far from sufficient, hindering their mass adoption. The manipulation of FFCs is also quite challenging. A good gripper should be able to pinch the cable steadily and execute insertion tasks of the cable connector with ease. The suction cup based solution is a good approach for holding the cable, but it makes the cable connector insertion very challenging as it can only apply limited shear forces. In this paper, we propose a locally-adaptive, pneumatic, parallel-jaw robot gripper equipped with fingertips that are able to both pinch the cable with a soft clamping mechanism and roll the cable surface on the soft fingertip structure until it reaches the desired connector. The gripper base accommodates a camera that allows for the recognition and pose estimation of the flat, flexible cables and other electronic components. The gripper is of low-cost and low-complexity and it can facilitate the efficient and robust execution of FFC grasping and assembly tasks.

Accurate localization is of crucial importance for autonomous driving tasks. Nowadays, we have seen a lot of sensor-rich vehicles (e.g. Robo-taxi) driving on the street autonomously, which rely on high-accurate sensors (e.g. Lidar and RTK GPS) and high-resolution map. However, low-cost production cars cannot afford such high expenses on sensors and maps. How to reduce costs? How do sensor-rich vehicles benefit low-cost cars? In this paper, we proposed a light-weight localization solution, which relies on low-cost cameras and compact visual semantic maps. The map is easily produced and updated by sensor-rich vehicles in a crowd-sourced way. Specifically, the map consists of several semantic elements, such as lane line, crosswalk, ground sign, and stop line on the road surface. We introduce the whole framework of on-vehicle mapping, on-cloud maintenance, and user-end localization. The map data is collected and preprocessed on vehicles. Then, the crowd-sourced data is uploaded to a cloud server. The mass data from multiple vehicles are merged on the cloud so that the semantic map is updated in time. Finally, the semantic map is compressed and distributed to production cars, which use this map for localization. We validate the performance of the proposed map in real-world experiments and compare it against other algorithms. The average size of the semantic map is 36 kb/km. We highlight that this framework is a reliable and practical localization solution for autonomous driving.

This paper presents an open-source untethered quadrupedal soft robot platform for dynamic locomotion (e.g., high-speed running and backflipping). The robot is mostly soft (80 vol.%) while driven by four geared servo motors. The robot’s soft body and soft legs were 3D printed with gyroid infill using a flexible material, enabling it to conform to the environment and passively stabilize during locomotion in multi-terrain environments. In addition, we simulated the robot in a real-time soft body simulation. With tuned gaits in simulation, the real robot can locomote at a speed of 0.9 m/s (2.5 body length/second), substantially faster than most untethered legged soft robots published to date. We hope this platform, along with its verified simulator, can catalyze agile soft robots' development.

Collaborative robots offer increased interaction capabilities at relatively low cost but in contrast to their industrial counterparts they inevitably lack precision. Moreover, in addition to the robots' own imperfect models, day-to-day operations entail various sources of errors that despite being small rapidly accumulate. This happens as tasks change and robots are re-programmed, often requiring time-consuming calibrations. These aspects strongly limit the application of collaborative robots in tasks demanding high precision (e.g. watch-making). We address this problem by relying on a dual-arm system with laser-based sensing to measure relative poses between objects of interest and compensate for pose errors coming from robot proprioception. Our approach leverages previous knowledge of object 3D models in combination with point cloud registration to efficiently extract relevant poses and compute corrective trajectories. This results in high-precision assembly behaviors. The approach is validated in a needle threading experiment, with a 150μm thread and a 300μm hole, and a USB insertion task using two 7-axis Panda robots.

Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and fully autonomous inter-floor navigation is still not available. As the very first step of inter-floor navigation, elevator button segmentation and recognition hold an important position. Therefore, we release the first large-scale publicly available elevator panel dataset in this work, containing 3,718 panel images with 35,100 button labels, to facilitate more powerful algorithms on autonomous elevator operation. Together with the dataset, a number of deep learning based implementations for button segmentation and recognition are also released to benchmark future methods in the community. The dataset is available at https://github.com/zhudelong/elevator_button_recognition