IROS 2022 Papers

The magnetic microrobot has become a promising approach in many biomedical applications due to its small volume, flexible motion, and untethered micromachines. The micro-chain robot is one of the most popular magnetic microrobots. However, the uncontrollable magnetic moment direction and quantity of the magnetic beads consisted in the existing self-assembled micro-chain robot limit their locomotion and applications. This paper proposed an on-chip micro-chain robot fabrication method to assemble the magnetic beads with controllable magnetic moment direction and quantity. The bead quantity can be controlled by the structure limits of the microchannel, and the direction of the magnetic moment can be adj usted by the integrated external magnetic field. The assembled magnetic beads are then glued by the hydrogel under UV exposure. The micro-chain robots with different quantities and magnetic moment directions of the magnetic beads were successfully fabricated and tested in experiments. Due to the array structure of the microfluidic device, batch manufacturing of low-cost magnetic robots was achieved in our method. The movement of dual-bead microrobots with two orthogonal magnetic moment directions was analyzed and compared. One of the dual-bead microrobots was applied in the transportation of the hydrogel module using pushing and pulling modes. It indicated that the proposed controllable on-chip fabrication of the magnetic micro-chain robots has the potential to enhance the microrobot ability in biomedical applications.

The ability to evolve is fundamental for any valuable autonomous agent whose knowledge cannot remain limited to that injected by the manufacturer. Consider for example a home assistant robot: it should be able to incrementally learn new object categories when requested, but also to recognize the same objects in different environments (rooms) and poses (hand-held/on the floor/above furniture), while rejecting unknown ones. Despite its importance, this scenario has started to raise interest in the robotic community only recently and the related research is still in its infancy, with existing experimental testbeds but no tailored methods. With this work, we propose the first learning approach that deals with all the previously mentioned challenges at once by exploiting a single contrastive objective. We show how it learns a feature space perfectly suitable to incrementally include new classes and is able to capture knowledge which generalizes across a variety of visual domains. Our method is endowed with a tailored effective stopping criterion for each learning episode and exploits a self-paced thresholding strategy that provides the classifier with a reliable rejection option. Both these novel contributions are based on the observation of the data statistics and do not need manual tuning. An extensive experimental analysis confirms the effectiveness of the proposed approach in establishing the new state-of-the-art. The code is available at https://github.com/FrancescoCappio/Contrastive_Open_World.

To improve the control of wearable robotics for gait assistance, we present an approach for continuous locomotion mode recognition as well as gait phase and stair slope estimation based on artificial neural networks that include time history information. The input features consist exclusively of processed variables that can be measured with a single shank-mounted inertial measurement unit. We introduce a wearable device to acquire real-world environment test data to demonstrate the performance and the robustness of the approach. Mean absolute error (gait phase, stair slope) and accuracy (locomotion mode) were determined for steady level walking and steady stair ambulation. Robustness was assessed using test data from different sensor hardware, sensor fixations, ambulation environments and subjects. The mean absolute error from the steady gait test data for the gait phase was 2.0-3.5 % for gait phase estimation and 3.3-3.8° for stair slope estimation. The accuracy of classifying the correct locomotion mode on the test data with the utilization of time history information was in between 98.51% and 99.67 %. Results show high performance and robustness for continuously predicting gait phase, stair slope and locomotion mode during steady gait. As hypothesized, time history information improves the locomotion mode recognition. However, while the gait phase estimation performed well for untrained transitions between locomotion modes, our qualitative analysis revealed that it may be beneficial to include transition data into the training of the neural network to improve the prediction of the slope and the locomotion mode. Our results suggest that artificial neural networks could be used for high level control of wearable lower limb robotics.

This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. Geo- tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used for this purpose. The proposed geo-tracking method aligns data from on-board cameras and lidar sensors with geo-registered orthophotos to continuously localize a vehicle. We train a model in a metric learning setting to extract visual features from ground and aerial images. The ground features are projected into a top-down perspective via the lidar points and are matched with the aerial features to determine the relative pose between vehicle and orthophoto. Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos. It exhibits strong generalization, is robust to changes in the environment and requires only geo-poses as ground truth. We evaluate our approach on the KITTI-360 dataset and achieve a mean absolute position error (APE) of 0.94m. We further compare with previous approaches on the KITTI odometry dataset and achieve state-of-the-art results on the geo-tracking task. 3

Wearable robots have been developed to aid or substitute the gait locomotion of humans. To assist gait locomotion based on the intention of a wearer, a gait pattern analysis is required with a wearable sensor by measuring body information, i.e., a joint angular velocity. However, measuring a precise joint angular velocity is difficult because the attachment position of a sensor has a curvature and an anatomical joint axis which is invisible. Therefore, a sensor calibration algorithm, which aligns a sensor axis into an anatomical joint axis, is required to provide an optimal assist for a wearer. Hence, in this paper, a new and simple sensor calibration algorithm is proposed with a unit sensor. Since a wearer shakes the body or collides with the ground when walking, the attachment position of a sensor may be changed. Thus, a continuous sensor compensation algorithm is also proposed. Additionally, the effectiveness of this new algorithm is demonstrated by gait locomotion experiments on various paths.

Learning-based model predictive control has been widely applied in autonomous racing to improve the closed-loop behaviour of vehicles in a data-driven manner. When environmental conditions change, e.g., due to rain, often only the predictive model is adapted, but the controller parameters are kept constant. However, this can lead to suboptimal behaviour. In this paper, we address the problem of data-efficient controller tuning, adapting both the model and objective simultaneously. The key novelty of the proposed approach is that we leverage a learned dynamics model to encode the environmental condition as a so-called context. This insight allows us to employ contextual Bayesian optimization to efficiently transfer knowledge across different environmental conditions. Consequently, we require fewer data to find the optimal controller configuration for each context. The proposed framework is extensively evaluated with more than 3'000 laps driven on an experimental platform with 1:28 scale RC race cars. The results show that our approach successfully optimizes the lap time across different contexts requiring fewer data compared to other approaches based on standard Bayesian optimization.

In recent years, driving context perception has emerged as one of the key aspects to design driving assistance algorithms and user interfaces that are effective in adapting to different traffic situations or environments. To this aim, we introduce the Anonymous Driving Scene Perception (ADSP) Model, a novel deep neural network designed to classify anony-mous Controller Area Network (CAN)-bus data into multiple driving context domains. ADSP extends the idea of driving scene classification to time series signals, as previous works relied heavily on visual features. Our model achieved a multi -domain classification accuracy of 84.9% on our custom-built naturalistic data set, as a combination of 92.7% on road type classification and 90.1 % on binary traffic detection, performing 2.0% and 1.6% better than the state-of-the-art model for multivariate time series classification. Our work demonstrates the feasibility of driving scene classification from anonymous CAN-bus data, without collecting sensitive data from users (images or GPS).

This work explores how contextual information and human intention affect the motion prediction of humans during a handover operation with a social robot. By classifying human intention in four different classes, we developed a model able to generate a different motion for each intention class. Furthermore, the model uses a multi-headed attention architecture to add contextual information to the pipeline, such as the position of the robot end effector (REE) or the position of obstacles in the interaction scene. We generate predictions up to two and half seconds in the future given an input sequence of one second containing the previous motion of the human. The results show an improvement of the prediction accuracy, both for the full skeleton prediction and the human hand used for the delivery. The model also allows to generate different sequences with the desired human intention.

Containerization promises to strengthen platform-independent development, better resource utilization, and secure deployment of software. As these benefits come with negligible overhead in CPU and memory utilization, containerization is increasingly being adopted in mobile robotic applications. An open challenge is supporting software tasks that have mixed-criticality requirements. Even more challenging is the combination of real-time containers with orchestration, which is an emerging paradigm to automate the deployment, networking, scaling, and availability of containerized workloads and services. This paper addresses this challenge by presenting a framework that extends the de-facto reference standard for container orchestration, Kubernetes, to schedule tasks with mixed-criticality requirements. Quantitative experimental results on the software implementing the mission of a Robotnik RB-Kairos mobile robot demonstrate the effectiveness of the proposed approach. The source code is publicly available on GitHub.

Performing highly agile acrobatic motions with a long flight phase requires perfect timing, high accuracy, and coordination of the full-body motion. To address these challenges, we present a novel approach on timings and trajectory optimization framework for legged robots performing aggressive 3D jumping. In our method, we firstly utilize an effective optimization framework using simplified rigid body dynamics to solve for contact timings and a reference trajectory of the robot body. The solution of this module is then used to formulate a full-body trajectory optimization based on the full nonlinear dynamics of the robot. This combination allows us to effectively optimize for contact timings while ensuring that the jumping trajectory can be effectively realized in the robot hardware. We first validate the efficiency of the proposed framework on the A1 robot model for various 3D jumping tasks such as double-backflips off the high altitude of 2m. Experimental validation was then successfully conducted for various aggressive 3D jumping motions such as diagonal jumps, barrel roll, and double barrel roll from a box of heights 0.4m and 0.9m, respectively.

As robots begin to move from structured industrial environments to the real world, they must be equipped to not only safely interact with the environment, but also reason about how to leverage contact to perform tasks. In this work, we develop a modeling and motion planning framework for continuum robots that accounts for contact anywhere along the robot. We first present an analytical model for continuum manipulators under contact and discuss the ideal choice of generalized coordinates given properties of the manipulator and task specifications. We then demonstrate the utility of our model by developing a motion planning framework that can solve a diverse set of tasks. We apply our framework to end effector path planning for a soft arm in an obstacle-rich environment, and grasp planning for soft robotic grippers, where contact can happen anywhere on the arm or gripper. Finally, we verify the utility of our model and planning framework by planning a grasp with a desired contact force for a soft antipodal gripper and testing this grasp in a hardware demonstration. Overall, our model and planning approach further enhance soft and continuum robots where they already excel: utilizing contact with the world to achieve their goals with a gentle touch.

Contact-implicit trajectory optimization offers an appealing method of automatically generating complex and contact-rich behaviors for robot manipulation and locomotion. The scalability of such techniques has been limited, however, by the challenge of ensuring both numerical reliability and physical realism. In this paper, we present preliminary results suggesting that the Iterative Linear Quadratic Regulator (iLQR) algorithm together with the recently proposed pressure-field-based hydroelastic contact model enables reliable and physically realistic trajectory optimization through contact. We use this approach to synthesize contact-rich behaviors like quadruped locomotion and whole-arm manipulation. Furthermore, open-loop playback on a Kinova Gen3 robot arm demonstrates the physical accuracy of the whole-arm manipulation trajectories. Code is available at https://bit.ly/ilqr_hc and videos can be found at https://youtu.be/IqxJKbM8_ms.

In this work, we propose a novel differential dynamic programming (DDP) framework for systems involving contact with the ground. The approach converts a general constrained differential dynamic programming into contact-implicit one by incorporating contact dynamics in a linear complementarity problem (LCP) formulation. Analytical gradients of the contact dynamics are obtained through a relaxed complementarity condition in the LCP formulation that helps the search directions of optimization avoid stalling in bad local minima or saddle points. Incorporation of contact dynamics and its analytical gradients into DDP enables an online discovery of not only dynamically-feasible trajectories of states, control inputs, and contact forces but also contact mode sequences. We demonstrate that our Contact-Implicit Differential Dynamic Programming framework successfully finds totally new dynamic motions with contact mode sequences in a variety of robotic systems including an one-legged hopping robot and planar quadrupedal robot in simulation environment.

This study aims to construct a running simulator based on a motion generation and control system that enables the description of motion strategies using the spring-loaded inverted pendulum (SLIP) model. The problems of stability and robustness encountered in the running simulation with the SLIP model are elucidated, and stable running is achieved by controlling the stiffness and the attitude angle dynamically at touchdown, as well as human energy adjustment that is introduced to consider the active motion strategy. As a result, passive and active control by humans can be expressed, and a framework that can express the changes in running due to motion strategies is constructed. Finally, we discuss the possibility of describing and elucidating the motion strategies.

To economically deploy robotic manipulators the programming and execution of robot motions must be swift. To this end, we propose a novel, constraint-based method to intuitively specify sequential manipulation tasks and to compute time-optimal robot motions for such a task specification. Our approach follows the ideas of constraint-based task specification by aiming for a minimal and object-centric task description that is largely independent of the underlying robot kinematics. We transform this task description into a non-linear optimization problem. By solving this problem we obtain a (locally) time-optimal robot motion, not just for a single motion, but for an entire manipulation sequence. We demonstrate the capabilities of our approach in a series of experiments involving five distinct robot models, including a highly redundant mobile manipulator.

Trajectory optimization is an efficient approach for solving optimal control problems for complex robotic systems. It relies on two key components: first the transcription into a sparse nonlinear program, and second the corresponding solver to iteratively compute its solution. On one hand, differential dynamic programming (DDP) provides an efficient approach to transcribe the optimal control problem into a finite-dimensional problem while optimally exploiting the sparsity induced by time. On the other hand, augmented Lagrangian methods make it possible to formulate efficient algorithms with advanced constraint-satisfaction strategies. In this paper, we propose to combine these two approaches into an efficient optimal control algorithm accepting both equality and inequality constraints. Based on the augmented Lagrangian literature, we first derive a generic primal-dual augmented Lagrangian strategy for nonlinear problems with equality and inequality constraints. We then apply it to the dynamic programming principle to solve the value-greedy optimization problems inherent to the backward pass of DDP, which we combine with a dedicated globalization strategy, resulting in a Newton-like algorithm for solving constrained trajectory optimization problems. Contrary to previous attempts of formu-lating an augmented Lagrangian version of DDP, our approach exhibits adequate convergence properties without any switch in strategies. We empirically demonstrate its interest with several case-studies from the robotics literature.

This paper presents a method for Bayesian multi-robot peer-to-peer data fusion where any pair of autonomous robots hold non-identical, but overlapping parts of a global joint probability distribution, representing real world inference tasks (e.g., mapping, tracking). It is shown that in dynamic stochastic systems, filtering, which corresponds to marginalization of past variables, results in direct and hidden dependencies between variables not mutually monitored by the robots, which might lead to an overconfident fused estimate. The paper makes both theoretical and practical contributions by providing (i) a rigorous analysis of the origin of the dependencies and (ii) a conservative filtering algorithm for heterogeneous data fusion in dynamic systems that can be integrated with existing fusion algorithms. This work uses factor graphs as both the analysis tool and the inference engine. Each robot in the network maintains a local factor graph and communicates only relevant parts of it (a sub-graph) to its neighboring robot. We discuss the applicability to various multi-robot robotic applications and demonstrate the performance using a multi-robot multi-target tracking simulation, showing that the proposed algorithm produces conservative estimates at each robot.

We develop two consensus-based learning algorithms for multi-robot systems applied on complex tasks involving collision constraints and force interactions, such as the cooperative peg-in-hole placement. The proposed algorithms integrate multi-robot distributed consensus and normalizing-flow-based reinforcement learning. The algorithms guarantee the stability and the consensus of the multi-robot system's generalized variables in a transformed space. This transformed space is obtained via a diffeomorphic transformation parameterized by normalizing-flow models that the algorithms use to train the underlying task, learning hence skillful, dexterous trajectories required for the task accomplishment. We validate the proposed algorithms by parameterizing reinforcement learning policies, demonstrating efficient cooperative learning, and strong generalization of dual-arm assembly skills in a dynamics-engine simulator.

This paper investigates using a sampling-based approach, the RRT*, to reconfigure a 2D set of connected tiles in complex environments, where multiple obstacles might be present. Since the target application is automated building of discrete, cellular structures using mobile robots, there are constraints that determine what tiles can be picked up and where they can be dropped off during reconfiguration. We compare our approach to two algorithms as global and local planners, and show that we are able to find more efficient build sequences using a reasonable amount of samples, in environments with varying degrees of obstacle space.