IROS 2023 论文列表

Human action recognition and motion forecasting is becoming increasingly successful, in particular with utilizing graphs. We aim to transfer this success into the context of industrial Human-Robot Collaboration (HRC), where humans work closely with robots and interact with workpieces in defined workspaces. For this purpose, it is necessary to use all the available information extractable in such a workspace and represent it with a natural structure, such as graphs, that can be used for learning. Since humans are the center of HRC, it is mandatory to construct the graph in a human-centered way and use real-world 3D information as well as object labels to represent their environment. Therefore, we present a novel Graph Neural Network (GNN) architecture which combines, human action recognition and motion forecasting for industrial HRC environments. We evaluate our method with two different and publicly available human action datasets, including one that is a particularly realistic representation of the industrial HRC, and compare the results with baseline methods for classifying the current human action and predicting the human motion. Our experiments show that our combined GNN approach improves the accuracy of action recognition compared to previous work, and significantly on the CoAx dataset by up to 20%. Further, our motion forecasting approach performs better than existing baselines, predicting human trajectories with a Final Displacement Error (FDE) of less than 10cm for a prediction horizon of 1s.

Robot-assisted rehabilitation is expected to reduce locomotor limitations of children and young adults with Cerebral Palsy (CP). However, to achieve this result, it is essential that the robot is transparent, allowing the user to move freely, and generate joint torques only when the exoskeleton joints significantly deviate from physiological gait patterns. Nevertheless, the development of transparent operation in lower-limb exoskeletons is still an open problem with several implementation challenges. In this study, we implemented a transparent operation strategy on the ExoRoboWalker, an overground exoskeleton designed for children and young adults with CP. The approach employs a feedback zero-torque controller with feedforward compensations for the exoskeleton's dynamics and actuators' impedance. We experimented the proposed system in five healthy subjects walking overground with the exoskeleton in transparent mode (ExoTransp) and non-transparent mode (ExoOff). The proposed transparent controller reduced the user-robot interaction torque and improved user gait kinematics relative to ExoOff. This is a significant step toward an overground gait training exoskeleton for CP population.

This paper presents a point cloud downsampling algorithm for fast and accurate trajectory optimization based on global registration error minimization. The proposed algorithm selects a weighted subset of residuals of the input point cloud such that the subset yields exactly the same quadratic point cloud registration error function as that of the original point cloud at the evaluation point. This method accurately approximates the original registration error function with only a small subset of input points (29 residuals at a minimum). Experimental results using the KITTI dataset demonstrate that the proposed algorithm significantly reduces processing time (by 87%) and memory consumption (by 99%) for global registration error minimization while retaining accuracy.

The capability of a robot to perform tasks depends not only on precise motion control, but also on a well-suited body morphology. Adapting both morphology and control of robots to improve their task performance has been a widely studied and long-standing issue. While the bio-inspired bi-level optimization framework has gained popularity in recent years, it suffers from high computation complexity due to the time-consuming and inefficient learning process for each morphology. In fact, in nature, besides the adaptive morphology and the intelligent brain, animals also possess an important gift, which is physical instinct. These instincts allow animals to respond quickly to their surroundings in the neonatal period, facilitating skills acquisition. Inspired by this, we propose an evolvable instinct controller to enhance the morphology-control co-adaption. The instinct controller suggests rough motion inclinations, which require minimal domain knowledge and entail less sophisticated design. Its purpose is to assist the main controller in learning fine-grained and robust control efficiently. We implemented this idea in the context of legged locomotion and designed the instinct controller using phase-based FSMs. We propose the instinct-based co-adaption algorithm and construct GPU parallel simulation experiments on different morphology prototypes. The results indicate that combining the co-adaption process with instinct evolution leads to the development of superior morphologies and robust controllers compared with the conventional co-adaption approach, with minimal additional time cost.

Offline evolutionary-based methodologies have supplied a successful motion planning framework for the quadrupedal jump. However, the time-consuming computation caused by massive population evolution in offline evolutionary-based jumping framework significantly limits the popularity in the quadrupedal field. This paper presents a time-friendly online motion planning framework based on meta-heuristic Differential evolution (DE), Latin hypercube sampling, and Configuration space (DLC). The DLC framework establishes a multidimensional optimization problem leveraging centroidal dynamics to determine the ideal trajectory of the center of mass (CoM) and ground reaction forces (GRFs). The configuration space is introduced to the evolutionary optimization in order to condense the searching region. Latin hypercube sampling offers more uniform initial populations of DE under limited sampling points, accelerating away from a local minimum. This research also constructs a collection of pre-motion trajectories as a warm start when the objective state is in the neighborhood of the pre-motion state to drastically reduce the solving time. The proposed methodology is successfully validated via real robot experiments for online jumping trajectory optimization with different jumping motions (e.g., ordinary jumping, flipping, and spinning).

Soft grippers have a number of advantages over their conventional stiff-bodied counterparts; not only do they surpass them in ease of fabrication and safety but also, in many cases, require less complex control strategies - due to natural compliance and form-fitting plasticity. Pneumatically actuated soft grippers made from non-extensible fabrics or polyethylene sheets have been shown to outperform soft silicone grippers capable of applying greater forces to the environment. Despite progress in the field, grasping a given object within a confined space proves challenging for both soft and conventional robotic grippers. A key issue is that most grippers use rotary or lateral translational motion when grasping an object, hence other objects in the scene may impede the closing motion as the gripper attempts to reach the target. In this study, we present a novel design for a soft robotic gripper equipped with a brace of fabric-based fingers capable, by way of eversion, of longitudinal extension, bending, and retraction, i.e. returning to a stowed state. Our experiments show that from the retracted to fully-extended state, the gripper fingers extend by up to 200% in length. To test the performance of the design, force characterisation experiments and grasping operations were carried out, demonstrating that each finger is capable of as much as 30 N of maximum tip force and can bend to an angle of 127°• At a bending pressure of 82.7 kPa (the maximum tested pressure in the bending chamber), a maximum (pullout) force of 16 N is needed to release an object that has been grasped by the finger. The experimental scenario detailed features all three mechanisms (eversion, bending and retraction) discussed.

Recognizing and comprehending human actions and gestures is a crucial perception requirement for robots to interact with humans and carry out tasks in diverse domains, including service robotics, healthcare, and manufacturing. Event cameras, with their ability to capture fast-moving objects at a high temporal resolution, offer new opportunities compared to standard action recognition in RGB videos. However, previous research on event camera action recognition has primarily focused on sensor-specific network architectures and image encoding, which may not be suitable for new sensors and limit the use of recent advancement in transformer-based architectures. In this study, we employ using a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame and then utilizes a temporal self-attention mechanism. This approach separates the spatial and temporal operations, resulting in VTN being more computationally efficient than other video transformers that process spatio-temporal volumes directly. In order to better adopt the VTN for the sparse and finegrained nature of event data, we design Event-Contrastive Loss \left(\mathscr{L}_{E C}\right)\left(\mathscr{L}_{E C}\right) and event specific augmentations. Proposed \left(\mathscr{L}_{E C}\right)\left(\mathscr{L}_{E C}\right) promotes learning fine-grained spatial cues in the spatial backbone of VTN by contrasting temporally misaligned frames. We evaluate our method on real-world action recognition of N-EPIC Kitchens dataset, and achieve state-of-the-art results on both protocols - testing in seen kitchen (74.9% accuracy) and testing in unseen kitchens (42.43% and 46.66% Accuracy). Our approach also takes less computation time compared to competitive prior approaches. We also evaluate our method on the standard DVS Gesture recognition dataset, achieving a competitive accuracy of 97.9% compared to prior work that uses dedicated architectures and image-encoding for the DVS dataset. These results demonstra... Show More

Our paper proposes a direct sparse visual odometry method that combines event and RGBD data to estimate the pose of agile-legged robots during dynamic locomotion and acrobatic behaviors. Event cameras offer high temporal resolution and dynamic range, which can eliminate the issue of blurred RGB images during fast movements. This unique strength holds a potential for accurate pose estimation of agile- legged robots, which has been a challenging problem to tackle. Our framework leverages the benefits of both RGBD and event cameras to achieve robust and accurate pose estimation, even during dynamic maneuvers such as jumping and landing a quadruped robot, the Mini-Cheetah. Our major contributions are threefold: Firstly, we introduce an adaptive time surface (ATS) method that addresses the whiteout and blackout issue in conventional time surfaces by formulating pixel-wise decay rates based on scene complexity and motion speed. Secondly, we develop an effective pixel selection method that directly samples from event data and applies sample filtering through ATS, enabling us to pick pixels on distinct features. Lastly, we propose a nonlinear pose optimization formula that simultaneously performs 3D-2D alignment on both RGB-based and event-based maps and images, allowing the algorithm to fully exploit the benefits of both data streams. We extensively evaluate the performance of our framework on both the public dataset and our own quadruped robot dataset, demonstrating its effectiveness in accurately estimating the pose of agile robots during dynamic movements. Supplemental video: https://youtu.be/-5ieQShOg3M

In this work, we evaluated the suitability of using a 7 degrees of freedom robotic manipulator as a planar haptic interface for studies in motor neuroscience. In particular, we assessed to what extent it can measure human movement and forces without applying undesired perturbations. To this aim, we evaluated the amount of perturbation exerted by the robot during planar reaching movements when controlled to be as transparent as possible in the 2D task space, through an impedance control. Two planar specular configurations of the robot were tested, namely G1 and G2, which differ in the position of the “elbow joint” in the workspace. For both configurations, we estimated the inertial ellipsoids and simulated the forces for human-like forward movements. Performance was then experimentally assessed on 8 healthy participants, in 15 different positions in the workspace. The average handpath perturbation decreased and settled to 6 mm after 2 minutes of interaction. Interaction forces resulted specular for G1 and G2, with mean values below 5 N. Overall, the robotic manipulator resulted suitable for studies on planar reaching movements in both configurations, with a preference for the G1 configuration due to its symmetrical distribution of trajectory deviations, which anyway remain well below 1 cm for movements of 15 cm.

Accurate localization systems are still a bottleneck for Unmanned Underwater Vehicles (UUVs). In recent years, fiducial markers have become a readily available, low-cost option. However, an in-depth analysis of marker detection accuracy in the underwater domain has yet to be performed. We propose a methodology to evaluate fiducial marker systems, namely the popular AprilTag system, in experiments. Our study especially focuses on aspects crucial for highly agile micro underwater robots, such as dynamic motions and the calibration medium. This class of robots is typically extensively studied in research tanks which motivates a first focus on clear-water settings. However, the proposed method and the findings can be transferred to similar scenarios. We demonstrate the importance of calibrating underwater and that the detection accuracy decreases linearly with camera distance and could therefore easily be compensated for. Moreover, we identify a suitable camera that maximizes the detection rate during highly dynamic motions. In sum, this work is an initial step towards application-relevant design strategies for designing low-cost, accessible localization systems for agile, mobile robots.

This paper proposes two metrics for evaluating learned object detection models: the proposition-labeled and distance-parametrized confusion matrices. These metrics are leveraged to quantitatively analyze the system with respect to its system-level formal specifications via probabilistic model checking. In particular, we derive transition probabilities from these confusion matrices to compute the probability that the closed-loop system satisfies its system-level specifications expressed in temporal logic. Instead of using object class labels, the proposition-labeled confusion matrix uses atomic propositions relevant to the high-level control strategy. Furthermore, unlike the traditional confusion matrix, the proposed distance-parametrized confusion matrix accounts for variations in detection performance with respect to the distance between the ego and the object. Empirically, these evaluation metrics, chosen by considering system-level specifications and control module design, result in less conservative system-level evaluations than those from traditional confusion matrices. We demonstrate this framework on a car-pedestrian example by computing the satisfaction probabilities for safety requirements formalized in Linear Temporal Logic.

Uncertainty estimation is crucial in safety-critical settings such as automated driving as it provides valuable information for several downstream tasks including high-level decision making and path planning. In this work, we propose EvCenterNet, a novel uncertainty-aware 2D object detection framework using evidential learning to directly estimate both classification and regression uncertainties. To employ evidential learning for object detection, we devise a combination of evidential and focal loss functions for the sparse heatmap inputs. We introduce class-balanced weighting for regression and heatmap prediction to tackle the class imbalance encountered by evidential learning. Moreover, we propose a learning scheme to actively utilize the predicted heatmap uncertainties to improve the detection performance by focusing on the most uncertain points. We train our model on the KITTI dataset and evaluate it on challenging out-of-distribution datasets including BDD100K and nuImages. Our experiments demonstrate that our approach improves the precision and minimizes the execution time loss in relation to the base model.

Solid particles, such as rice and coffee beans, are commonly stored in containers and are ubiquitous in our daily lives. Understanding those particles' properties could help us make later decisions or perform later manipulation tasks such as pouring. Humans typically interact with the containers to get an understanding of the particles inside them, but it is still a challenge for robots to achieve that. This work utilizes tactile sensing to estimate multiple properties of solid particles enclosed in the container, specifically, content mass, content volume, particle size, and particle shape. We design a sequence of robot actions to interact with the container. Based on physical understanding, we extract static force/torque value from the F/T sensor, vibration-related features and topple-related features from the newly designed high-speed GelSight tactile sensor to estimate those four particle properties. We test our method on 37 very different daily particles, including powder, rice, beans, tablets, etc. Experiments show that our approach is able to estimate content mass with an error of 1.8 g, content volume with an error of 6.1 ml, particle size with an error of 1.1 mm, and achieves an accuracy of 75.6% for particle shape estimation. In addition, our method can generalize to unseen particles with unknown volumes. By estimating these particle properties, our method can help robots to better perceive the granular media and help with different manipulation tasks in daily life and industry.

In this study, we proposed a dynamic model that could quantify human comfort in autonomous vehicles (AVs) based on vehicular behaviors and a Kalman filter (KF) based approach to further refine comfort level estimation by leveraging physiological signals. The dynamic model could capture the dynamics in human comfort when the passenger was exposed to a continuous sequence of vehicular behaviors during an AV journey. The KF-based comfort estimation approach could fuse comfort level estimations based on physiological signals and the dynamic model. A simulator-based user study was conducted to evaluate the comfort estimation approaches in which the participants experienced a set of virtual AV journeys on a high-fidelity driving simulator with 6-degree-of-freedom motions. Experimental results show that the proposed approaches could quantify human comfort levels and the KF-based approach outperforms the others.

Our world is non-static, and robots should be able to track its changing geometry. For tracking changes, data asso-ciations between 3D points over time are key. In this paper, we investigate the problem of associating 3D points on plant organs from different mapping runs over time while the plants grow. We achieve a high spatial-temporal matching performance by combining 3D RGB-D SLAM, visual place recognition, and 2D/3D matching exploiting background knowledge. We showcase our approach in a real agricultural glasshouse used to grow sweet peppers, using RGB-D observations from a mobile robot traversing the environment. Our experiments suggest that with our approach, we can robustly make data associations in highly repetitive scenes and under changing geometries caused by plant growth. We see our approach as an important step towards spatial-temporal data association for robotic agriculture.

In many applications such as aerial transportation, delivery, and manipulation, it is essential to know the external wrench exerted on multirotor aerial vehicles precisely. This paper presents an algorithm to estimate external wrench using a rotor speed measurement unit, an inertial measurement unit and a motion capture system. Under a cascaded architecture containing two sub-systems, one error-state Kalman Filter is designed to estimate velocity and attitude and eliminate the bias of the measurement from the inertial measurement unit, the other error-state Kalman Filter is designed to estimate the external wrench. Observability of the two estimation subsystems is verified by the Lie derivative method. The proposed algorithm has been tested in simulations and real-world experiments, which demonstrates its superiority in providing real-time and accurate external wrench estimation.

In applications such as search and rescue or disaster relief, heterogeneous multi-robot systems (MRS) can provide significant advantages for complex objectives that require a suite of capabilities. However, within these application spaces, communication is often unreliable, causing inefficiencies or outright failures to arise in most MRS algorithms. Many researchers tackle this problem by requiring all robots to either maintain communication using proximity constraints or assuming that all robots will execute a predetermined plan over long periods of disconnection. The latter method allows for higher levels of efficiency in a MRS, but failures and environmental uncertainties can have cascading effects across the system, especially when a mission objective is complex or time-sensitive. To solve this, we propose an epistemic planning framework that allows robots to reason about the system state, leverage heterogeneous system makeups, and optimize information dissemination to disconnected neighbors. Dynamic epistemic logic formalizes the propagation of belief states, and epistemic task allocation and gossip is accomplished via a mixed integer program using the belief states for utility predictions and planning. The proposed framework is validated using simulations and experiments with heterogeneous vehicles.

Motion planning tasks are often innately invariant to certain geometric transformations, or in other words, symmetric. This property, however, is not always reflected in learned policies that are trained on these tasks. Although this asymmetry can be addressed through data augmentation or additional training samples, doing so comes at a cost of increased training time. Instead of trying to remedy this issue during the learning process, we leverage this disparity during execution. We propose the symmetry exploitation policy, an augmentation in the post-hoc execution stage of RL policies. During the planning stage, we present the learned policy with an invariant, geometrically transformed version of the observation as an alternate perspective of the state. This allows the policy to produce multiple possible actions for a single state, and choose the action with the highest estimated value. Unlike other symmetry exploitation methods for learning solutions in motion planning, this method completely bypasses the need for additional training. We show the effect of the symmetry exploitation policy on DQN, A2C, and PPO policies, in three motion problems with different dimensions, observation types, and symmetries. The results show that by exploiting the symmetry of the task, a trained model achieves improved performance and better generalization, and can achieve comparable results to retraining, augmentation, or extended training, without incurring any additional training time. The efficacy is most prominent in more complex tasks, as 89 of the 100 models involved in the case study improve when using the method.

In this paper, we propose a system that supports operators who provide services to customers using teleoperated robots. We observed that unprofessional or lazy operators of teleoperated robots are a risk for businesses as they are likely to speak in ways that are inappropriate for customer services. The proposed system lets competent operators talk freely to customers and thus provide high quality service. For subpar operators, the proposed system filters inappropriate utterances to improve the service they provide. We conducted a user study with 21 participants to compare the proposed support system to a baseline system where operators talk freely to customers. For subpar operators, the quality of the service is significantly higher in terms of perceived politeness and reported customer satisfaction when using the proposed support system compared to when using the baseline system. For competent operators, we found no significant differences in the quality of the service between the two systems.