IROS 2023 Papers

Soft compliant microrobots have the potential to deliver significant societal impact when deployed in applications such as search and rescue. In this research we present mCLARI, a body compliant quadrupedal microrobot of 20mm neutral body length and 0.97g, improving on its larger predecessor, CLARI. This robot has four independently actuated leg modules with 2 degrees of freedom, each driven by piezoelectric actuators. The legs are interconnected in a closed kinematic chain via passive body joints, enabling passive body compliance for shape adaptation to external constraints. Despite scaling its larger predecessor down to 60 % in length and 38% in mass, mCLARI maintains 80% of the actuation power to achieve high agility. Additionally, we demonstrate the new capability of passively shape-morphing mCLARI - omnidirectional laterally confined locomotion - and experimentally quantify its running performance achieving a new unconstrained top speed of ~3 bodylengths/s (60 mms-1). Leveraging passive body compliance, mCLARI can navigate through narrow spaces with a body compression ratio of up to 1.5 × the neutral body shape.

This paper presents the gatekeeper algorithm, a real-time and computationally-lightweight method to ensure that nonlinear systems can operate safely in dynamic environments despite limited perception. gatekeeper integrates with existing path planners and feedback controllers by introducing an additional verification step that ensures that proposed trajectories can be executed safely, despite nonlinear dynamics subject to bounded disturbances, input constraints and partial knowledge of the environment. Our key contribution is that (A) we propose an algorithm to recursively construct committed trajectories, and (B) we prove that tracking the committed trajectory ensures the system is safe for all time into the future. The method is demonstrated on a complicated firefighting mission in a dynamic environment, and compares against the state-of-the-art techniques for similar problems.

This paper introduces evoBOT, a novel robot platform for research on highly dynamic locomotion and human-machine interaction. evoBOT is capable of performing complex tasks such as handovers or manipulation while moving at high speeds. We provide an overview of the robot's core features and the underlying design decisions on both the mechanical and the electronic level. Moreover, we propose a reinforcement learning (RL) based control approach for training highly dynamic motions that is evaluated on a first set of robotic tasks, including robust balancing and dynamic locomotion. Lastly, we conduct extensive benchmarking on the adopted sim-to-real methods and present an initial sim-to-real pipeline for first transfer of the trained policies to the real robot. To accelerate robotics research in this direction, the full simulation model of the robot is released as open-source.

Soft robots present unique capabilities, but have been limited by the lack of scalable technologies for construction and the complexity of algorithms for efficient control and motion. These depend on soft-body dynamics, high-dimensional actuation patterns, and external/onboard forces. This paper presents scalable methods and platforms to study the impact of weight distribution and actuation patterns on fully untethered modular soft robots. An extendable Vibrating Intelligent Piezo-Electric Robot (eViper), together with an open-source Simulation Framework for Electroactive Robotic Sheet (SFERS) implemented in PyBullet, was developed as a platform to analyze the complex weight-locomotion interaction. By integrating power electronics, sensors, actuators, and batteries onboard, the eViper platform enables rapid design iteration and evaluation of different weight distribution and control strategies for the actuator arrays. The design supports both physics-based modeling and data-driven modeling via onboard automatic data-acquisition capabilities. We show that SFERS can provide useful guidelines for optimizing the weight distribution and actuation patterns of the eViper, thereby achieving maximum speed or minimum cost of transport (COT).

We consider the detection of faults in robotic manipulators, with particular emphasis on faults that have not been observed or identified in advance, which naturally includes those that occur very infrequently. Recent studies indicate that the reward function obtained through Inverse Reinforcement Learning (IRL) can help detect anomalies caused by faults in a control system (i.e. fault detection). Current IRL methods for fault detection, however, either use a linear reward representation or require extensive sampling from the environment to estimate the policy, rendering them inappropriate for safety-critical situations where sampling of failure observations via fault injection can be expensive and dangerous. To address this issue, this paper proposes a zero-shot and exogenous fault detector based on an approximate variational reward imitation learning (AVRIL) structure. The fault detector recovers a reward signal as a function of externally observable information to describe the normal operation, which can then be used to detect anomalies caused by faults. Our method incorporates expert knowledge through a customizable reward prior distribution, allowing the fault detector to learn the reward solely from normal operation samples, without the need for a simulator or costly interactions with the environment. We evaluate our approach for exogenous partial fault detection in multi-stage robotic manipulator tasks, comparing it with several baseline methods. The results demonstrate that our method more effectively identifies unseen faults even when they occur within just three controller time steps.

In order for a humanoid robot to balance on the movable ground, balance feedback control in response to its unpredictable movement is required. However, feedback control in response to ground movement has the following two issues, (A) Interaction between the ground dynamics and the balance control may cause vibration. (B) The balance control may rather deteriorate the stability due to the response delay. To solve these problems, this study proposes the support foot acceleration term in the walking stabilizer and gives its gain by considering the following two conditions, (A) Avoiding steady-state vibration in a two-mass linear inverted pendulum model on an arbitrary ground, and (B) reducing the influence of inertial forces resulting from the delay of ZMP feedback. Experiments with a life-size humanoid JAXON verified the steady-state vibration phenomenon and improved the stability of acceleration and deceleration when boarding the Two-Wheeled Scooter.

This paper proposes an external wrench estimation method for modular manipulators, where each link module is driven with external actuation (e.g., rotors, thrusters) and inter-module joints can be locked to increase end-effector stiffness or workforce of the manipulator. For such systems, the commonly-used momentum-based observer (MBO [1]) is not suitable due to the presence of unknown joint locking (JL) torque and also the degeneracy of Jacobian transpose relation with the system degree-of-freedom (DOF) becoming less than six with the joint locking. To overcome this, we propose two novel external wrench estimation algorithms: a distributed algorithm based on recursive Newton-Euler dynamics and a centralized algorithm based on D'Alembert's principle, both using an F/T (force/torque) sensor at the base. Experiments are conducted to demonstrate the effectiveness of the proposed algorithms.

The use of impedance control has become widespread in applications requiring simultaneous position tracking and compliance in contact. However, disturbances such as friction and model uncertainties can adversely affect the performance of impedance-based motion control. The disturbance observer (DOB) has been proposed to address this issue, which is a widely-utilized robust controller that eliminates observed disturbances with the nominal model. However, current DOB applications fail to consider the aspect of interactive force control properly. This study proposes a novel Workspace Force/Acceleration Disturbance Observer (WFADOB) controller, which utilizes both interaction force and acceleration to design a disturbance observer loop, enabling precise motion tracking even with low-impedance gain settings. Additionally, the proposed controller offers fine impedance rendering performance, offering safe contact while maintaining low impedance. This paper discusses the problem of motion tracking performance due to friction and the interaction force that arises during contact. The proposed controller is theoretically analyzed and experimentally verified, demonstrating its performance compared to conventional methods.

Data collected at Hurricane Ian (2022) quantifies the demands that small uncrewed aerial systems (UAS), or drones, place on the network communication infrastructure and identifies gaps in the field. Drones have been increasingly used since Hurricane Katrina (2005) for disaster response, however getting the data from the drone to the appropriate decision makers throughout incident command in a timely fashion has been problematic. These delays have persisted even as countries such as the USA have made significant investments in wireless infrastructure, rapidly deployable nodes, and an increase in commercial satellite solutions. Hurricane Ian serves as a case study of the mismatch between communications needs and capabilities. In the first four days of the response, nine drone teams flew 34 missions under the direction of the State of Florida FL-UAS1, generating 636GB of data. The teams had access to six different wireless communications networks but had to resort to physically transferring data to the nearest intact emergency operations center in order to make the data available to the relevant agencies. The analysis of the mismatch contributes a model of the drone data-to-decision workflow in a disaster and quantifies wireless network communication requirements throughout the workflow in five factors. Four of the factors-availability, bandwidth, burstiness, and spatial distribution-were previously identified from analyses of Hurricanes Harvey (2017) and Michael (2018). This work adds upload rate as a fifth attribute. The analysis is expected to improve drone design and edge computing schemes as well as inform wireless communication research and development.

Uncertainties in grasp prediction for unknown, arbitrarily shaped objects in cluttered environments and un- certainty of the kinematics (e.g., series elastic robots or soft robots) can lead to poor grasps. For delicate objects, such poor grasps may damage the objects when they are dropped or when high local pressure is introduced in the grasping process. We propose a tactile sensor concept that allows predicting the quality of a grasp such that the object can be safely moved without being dropped. This prediction is done using an initial low force grasp and the force is only increased when the contact area is sufficiently large. The proposed customizable wireless Capacitive Tactile Sensor Array (CTSA) uses the deformation of a polymer to assess the contact area and the force distribution. A common homogeneous deformable electrode is embedded in the polymer. This electrode does not require any patterning nor any electrical connection but to ground. We present the manufacturing process which allows for robust yet cost effective realizations with a variety of electrode materials including conductive inks, conductive textiles, metal meshes and metal sheets. With the different approaches, parameters such as sensitivity and recovery time can be adjusted. Furthermore, the robustness of the sensor towards strong forces and objects with sharp edges and corners is shown. Finally, we demonstrate the benefits of the proposed sensor for grasping in a series of scenarios with rigid and soft 3D printed objects of various shapes. Allowing a reasonable false positive rate, 100 % of unsuccessful grasps in our evaluation experiments could be detected from the initial low force grasp.

Humanoids operate in repeated contact and non-contact with their environment and so the motion of humanoids such as walking on uneven terrain or in a narrow space requires the accurate force and position control. Joint torque control systems are suitable for position and force control, but are prone to friction and other modeling errors. To solve this problem, methods have been proposed to realize torque control in combination with joint position control systems or by improving joint structures such as sensors and actuators, but these methods have problems such as response delay and increased weight and volume. Thus, it is difficult to achieve motion of life-sized humanoids by whole-body torque control. In this paper, we solve challenges not with one specific layer, but rather with multiple layers that complement each other. We propose a hierarchical whole-body torque control method using four layers: friction compensation based on a vibration-suppressed model, whole-body resolved acceleration control using priority, center-of-gravity acceleration control based on foot-guided control, and landing position time modification based on capture point. We verify through walking experiments that the proposed methods can control the life-sized humanoid robot driven by high-reduction ratio joints by whole-body torque control without a torque sensor or joint position control, and that it enables the robot to move and even transport an object on outdoor uneven terrain.

This study presents a method for estimating the three-dimensional (3D) shapes of transparent objects from an RGB-D image using a statistical shape model. Statistical shape models compress the dimensions of multiple shapes to represent shape variations using fewer parameters. It is difficult to measure the depth of a transparent object using sensors. Therefore, the statistical shape model is deformed to fit the contour extracted from an RGB image and estimate the shape of the object. The depth image is used only to detect the plane on which the transparent objects are placed. Unlike other estimation methods, the proposed method estimates the whole shape of a transparent object. To validate the proposed method, the obtained estimation accuracy is compared with that of a machine-learning-based method. In addition, the estimated whole shape is compared with the measured data from a 3D scanner.

This study extends the whole-body control (WBC) formulation for bipedal humanoid robots that include closed (parallel) kinematic chains in their structure. Along with general formulation, we also stress the implementation of this formulation on Kangaroo, which is a highly dynamic humanoid robot developed by PAL Robotics. This 76-DOF robot includes 24 independent closed-kinematic chains in its structure and constitutes a good case study for our approach. We discuss the WBC formulation for various control structures, including inverse dynamics control (IDC) and Modular Passive Tracking Control (MPTC). As a test scenario, we employ a 3D spring-loaded inverted pendulum (SLIP) jumping trajectory with disturbance rejection as the desired CoM trajectory.

Long-term visual localization is an essential problem in robotics and computer vision, but remains challenging due to the environmental appearance changes caused by lighting and seasons. While many existing works have attempted to solve it by directly learning invariant sparse keypoints and descriptors to match scenes, these approaches still struggle with adverse appearance changes. Recent developments in image transformations such as neural style transfer have emerged as an alternative to address such appearance gaps. In this work, we propose to combine an image transformation network and a feature-learning network to improve long-term localization performance. Given night-to-day image pairs, the image transformation network transforms the night images into day-like conditions prior to feature matching; the feature network learns to detect keypoint locations with their associated descriptor values, which can be passed to a classical pose estimator to compute the relative poses. We conducted various experiments to examine the effectiveness of combining style transfer and feature learning and its training strategy, showing that such a combination greatly improves long-term localization performance.

Weakly supervised referring expression grounding (WREG) is an attractive and challenging task for grounding target regions in images by understanding given referring expressions. WREG learns to ground target objects without the manual annotations between image regions and referring expressions during the model training phase. Different from the predominant grounding pattern of existing models, which locates target objects by reconstructing the region-expression correspondence, we investigate WREG from a novel perspective and enrich the prevailing pattern with self-knowledge distillation. Specifically, we propose a target-guided self-knowledge distillation approach that adopts the target prediction knowledge learned from the previous training iterations as the teacher to guide the subsequent training procedure. In order to avoid the misleading caused by the teacher knowledge with low prediction confidence, we present an uncertaintyaware knowledge refinement strategy to adaptively rectify the teacher knowledge by learning dynamic threshold values based on the model prediction uncertainty. To validate the proposed approach, we implement extensive experiments on three benchmark datasets, i.e., Ref Coco, RefCOCO+, and RefCOCOg. Our approach achieves new state-of-the-art results on several splits of the benchmark datasets, showcasing the advantage of the proposed framework for WREG. The implementation codes and trained models are available at: https://github.com/dami23IWREG.sar_KD.

Underwater caves are challenging environments that are crucial for water resource management, and for our understanding of hydro-geology and history. Mapping underwater caves is a time-consuming, labor-intensive, and hazardous operation. For autonomous cave mapping by underwater robots, the major challenge lies in vision-based estimation in the complete absence of ambient light, which results in constantly moving shadows due to the motion of the camera-light setup. Thus, detecting and following the caveline as navigation guidance is paramount for robots in autonomous cave mapping missions. In this paper, we present a computationally light caveline detection model based on a novel Vision Transformer (ViT)-based learning pipeline. We address the problem of scarce annotated training data by a weakly supervised formulation where the learning is reinforced through a series of noisy predictions from intermediate sub-optimal models. We validate the utility and effectiveness of such weak supervision for caveline detection and tracking in three different cave locations: USA, Mexico, and Spain. Experimental results demonstrate that our proposed model, CL-ViT, balances the robustness-efficiency trade-off, ensuring good generalization performance while offering 10+ FPS on single-board (Jetson TX2) devices.

This paper presents a water walking robot with 6 feet which consists of a rimless wheel and a flywheel, and has a foot attached to the tip of each leg. First, in order to make the robot walk on water, we propose a foot control method by imitating the legs of some animals that can walk on water. Second, in order to increase the robot's forward speed, we improved the control method. In addition we have analysed the effect of some different physical parameters on the motion of the robot. This research is aimed at building a versatile water walking model for such applications as marine exploration.

The pedestrian crossing intention prediction problem is to estimate whether or not the target pedestrian will cross the street. State-of-the-art techniques heavily depend on visual data acquired through the front camera of the ego-vehicle to make a prediction of the pedestrian's crossing intention. Hence, the efficiency of current methodologies tends to decrease notably in situations where visual input is imprecise, for instance, when the distance between the pedestrian and ego-vehicle is considerable or the illumination levels are inadequate. To address the limitation, in this paper, we present the design, implementation, and evaluation of the first-of-its-kind pedestrian crossing intention prediction model based on integration of motion sensor data gathered through the smartwatch (or smartphone) of the pedestrian. We propose an innovative machine learning framework that effectively integrates motion sensor data with visual input to enhance the predictive accuracy significantly, particularly in scenarios where visual data may be unreliable. Moreover, we perform an extensive data collection process and introduce the first pedestrian intention prediction dataset that features synchronized motion sensor data. The dataset comprises 255 video clips that encompass diverse distances and lighting conditions. We trained our model using the widely-used JAAD and our own datasets and compare the performance with a state-of-the-art model. The results demonstrate that our model outperforms the current state-of-the-art method, particularly in cases where the distance between the pedestrian and the observer is considerable (more than 70 meters) and the lighting conditions are inadequate.

This paper presents a safety-critical locomotion control framework for quadrupedal robots. Our goal is to enable quadrupedal robots to safely navigate in cluttered environments. To tackle this, we introduce exponential Discrete Control Barrier Functions (exponential DCBFs) with duality-based obstacle avoidance constraints into a Non-linear Model Predictive Control (NMPC) with Whole-Body Control (WBC) framework for quadrupedal locomotion control. This enables us to use polytopes to describe the shapes of the robot and obstacles for collision avoidance while doing locomotion control of quadrupedal robots. Compared to most prior work, especially using CBFs, that utilize spherical and conservative approximation for obstacle avoidance, this work demonstrates a quadrupedal robot autonomously and safely navigating through very tight spaces in the real world. (Our open-source code is available at https://github.com/HybridRobotics/quadruped_nmpc_dcbf_duality, and the video is available at https://youtu.be/plgSQjwXm1Q.)