ICRA 2023 Papers

PARSEC (Payload Anchoring Robotic System for the Exploration of Cliffs) is an autonomy-equipped aerial manipulator that can deploy self-anchoring payloads on rocky vertical surfaces. It consists of a hexacopter and a two Degrees of Freedom (2 DoF) mass balancing manipulator, which can autonomously deploy a self-anchoring payload from its custom end-effector. The payload anchors itself via an actuated microspine gripper. Payload sensor data is wirelessly transmitted to the primary vehicle during and after deployment. A novel state machine controls the four-stage PARSEC deployment process. First, the rotorcraft brings the payload into contact with the surface and applies a constant 6 N normal force through a feedback control loop to preload the payload microspine gripper. Second, while the rotorcraft maintains the constant normal force, the gripper is commanded to close until engagement with the surface is confirmed through the current feedback sensing. Then, the aerial manipulator pulls with 5 N force on the anchored payload to ensure a secure grip before releasing the package and flying away. We present experimental validation of a successful deployment of a 430 g payload on a vertical vesicular basalt surface.

Oysters play a pivotal role in the bay living ecosystem and are considered the living filters for the ocean. In recent years, oyster reefs have undergone major devastation caused by commercial over-harvesting, requiring preservation to maintain ecological balance. The foundation of this preservation is to estimate the oyster density which requires accurate oyster detection. However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments. To this end, we present a novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data. Utilizing our synthetic data along with real data for oyster detection, we obtain up to 35.1 % boost in performance as compared to using only real data with our OysterNet network. We also improve the state-of-the-art by 12.7%. This shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully and we hope more researchers adopt such a strategy for hard-to-obtain datasets.

The configuration spaces of parallel manipulators exhibit more nonlinearity than serial manipulators. Qualitatively, they can be seen to possess extra folds. Projection onto smaller spaces of engineering relevance, such as an output workspace or an input actuator space, these folds cast edges that exhibit boundary behavior. For example, inside the global workspace bounds of a five-bar linkage appear several local workspace bounds that only constrain certain output modes of the mechanism. The presence of such boundaries, which manifest in both input and output projections, serve as a source of confusion when these projections are studied exclusively instead of the configuration space itself. Particularly, the design of nonsymmetric parallel manipulators has been confounded by the presence of exotic projections in their input and output spaces. In this paper, we represent the configuration space with a radius graph, then weight each edge by solving an optimization problem using homotopy continuation to quantify transmission quality. We then employ a graph path planner to approximate geodesics between configuration points that avoid regions of low transmission quality. Our methodology automatically generates paths capable of transitioning between non-neighboring output modes, a motion which involves osculating multiple workspace boundaries (local, global, or both). We apply our technique to two nonsymmetric five-bar examples that demonstrate how transmission properties and other characteristics of the workspace can be selected by switching output modes.

Soft robots have long been attractive to robotic engineers due to their remarkable dexterity; however, reports that standardize soft actuators into modularized off-shelf devices akin to rigid robots are still rare, and the mechanical efficiency of existing designs is still limited. This work identifies origami folding to enable the design of LEGO-like modularized soft actuators with high mechanical efficiency in terms of payload capability and workspace. Herein, three modularized origami actuators that can generate translational, bending, and twisting motion are designed, prototyped, and tested. The translational actuator can contract to 40% of its original length, and the twisting and bending actuators can exert 31° and 52° angular motions, respectively. The translational actuator can exert a blocked force of about 821 times self-weight. The motion of origami soft actuators is accurately modeled using rigid body kinematics, and complex systems built by them are captured by homogeneous transformation. Finally, the modularized design and efficient kinematic model are verified on a manipulator and a reconfigurable letter. Benefiting from the unprecedented modularity and mechanical efficiency, these LEGO-like origami actuators are promising for practical applications like food handling and healthcare.

A spatial AI that can perform complex tasks through visual signals and cooperate with humans is highly anticipated. To achieve this, we need a visual SLAM that easily adapts to new scenes without pre-training and generates dense maps for downstream tasks in real-time. None of the previous learning-based and non-learning-based visual SLAMs satisfy all needs due to the intrinsic limitations of their components. In this work, we develop a visual SLAM named Orbeez-SLAM, which successfully collaborates with implicit neural representation and visual odometry to achieve our goals. Moreover, Orbeez-SLAM can work with the monocular camera since it only needs RGB inputs, making it widely applicable to the real world. Results show that our SLAM is up to 800x faster than the strong baseline with superior rendering outcomes. Code link: https://github.com/MarvinChung/Orbeez-SLAM.

Hierarchical Imitation Learning (HIL) has been proposed to recover highly-complex behaviors in long-horizon tasks from expert demonstrations by modeling the task hierarchy with the option framework. Existing methods either overlook the causal relationship between the subtask and its corresponding policy or cannot learn the policy in an end-to-end fashion, which leads to suboptimality. In this work, we develop a novel HIL algorithm based on Adversarial Inverse Reinforcement Learning and adapt it with the Expectation-Maximization algorithm in order to directly recover a hierarchical policy from the unannotated demonstrations. Further, we introduce a directed information term to the objective function to enhance the causality and propose a Variational Autoencoder framework for learning with our objectives in an end-to-end fashion. Theoretical justifications and evaluations on challenging robotic control tasks are provided to show the superiority of our algorithm. The codes are available at https://github.com/LucasCJYSDL/HierAIRL.

In this paper, we explore the space of running gaits for the bipedal robot Cassie. Our first contribution is to present an approach for optimizing gait efficiency across a spectrum of speeds with the aim of enabling extremely high-speed running on hardware. This raises the question of how the resulting gaits compare to human running mechanics, which are known to be highly efficient in comparison to quadrupeds. Our second contribution is to conduct this comparison based on established human biomechanical studies. We find that despite morphological differences between Cassie and humans, key properties of the gaits are highly similar across a wide range of speeds. Finally, our third contribution is to integrate the optimized running gaits into a full controller that satisfies the rules of the real-world task of the 100m dash, including starting and stopping from a standing position. We demonstrate this controller on hardware to establish the Guinness World Record for Fastest 100m by a Bipedal Robot.

Capsule endoscopic robot holds great promise for the early diagnosis of gastrointestinal diseases without causing discomfort to patients. However, currently available active capsule endoscopic robots suffer from issues such as complex structure, poor mobility, large size, and high cost, which have hindered their widespread adoption and resulted in a lower screening rate for gastrointestinal diseases. To address these challenges, this paper proposes a highly integrated propeller-driven capsule endoscopic robot (PCER) system that integrates STM32 processor, magnetic sensor, IMU, RF communication unit, and motor drive. The micro propeller of the PCER has been analyzed through finite element simulation to ensure its efficiency. FLUENT software has been utilized to simulate the fluid force acting on the PCER as it moves through a liquid medium. The results of the simulation are then used to determine the optimal pitch angle for the robot's movement. The thrust generated by the capsule robot propellers has been measured using a lever mechanism to investigate the relationship between the thrust and voltage applied to the motors. The experiments confirmed that the PCER is capable of performing flexible motions within fluid environments, such as changing pitch angle during movement, passing circular obstacles, horizontal motion, and spiral ascent. These findings demonstrate the feasibility of the proposed PCER as an effective tool for non-invasive early screening of gastrointestinal diseases.

Workpiece placement with respect to an industrial robot plays an important role in robotic manufacturing due to its influence on the configuration-dependent properties of industrial robots. Suboptimal placements of the workpiece may increase the required joint torques and decrease the dexterity of the robot. The focus of this work is to identify an optimal workpiece pose that enables a robot to carry out surface finishing with configurations that require the lowest possible joint torques while having maximum possible manipulability. We present a non-linear optimization-based algorithm to solve this problem and demonstrate the algorithm's capability on dif-ferent workpieces which we share to facilitate further research in this area.

Model predictive control (MPC) is a powerful tool to control systems with non-linear dynamics and constraints, but its computational demands impose limitations on the dynamics model used for planning. Instead of using a single complex model along the MPC horizon, model hierarchy predictive control (MHPC) reduces solve times by planning over a sequence of models of varying complexity within a single horizon. Choosing this model sequence can become intractable when considering all possible combinations of reduced order models and prediction horizons. We propose a framework to systematically optimize a model schedule for MHPC. We leverage trajectory optimization (TO) to approximate the accumulated cost of the closed-loop controller. We trade off performance and solve times by minimizing the number of decision variables of the MHPC problem along the horizon while keeping the approximate closed-loop cost near optimal. The framework is validated in simulation with a planar humanoid robot as a proof of concept. We find that the approximated closed-loop cost matches the simulated one for most of the model schedules, and show that the proposed approach finds optimal model schedules that transfer directly to simulation, and with total horizons that vary between 1.1 and 1.6 walking steps.

The redundant manipulators' analytical solutions can be obtained by the parameterization method. Multiple parameterized joints and their corresponding parametric representations exist for a redundant manipulator. However, how to select the optimal parameterized joints has yet to be well-addressed. This paper delves into the mechanism of the parameterization method and proposes a method to select the optimal parametric representations to improve the motion planning performance of manipulators. We tested the proposed method on an 8-degree-of-freedom (DOF) manipulator. First, all feasible parametric representations are derived, followed by an approach to obtain solution manifolds. We then introduce a metric called the “feasible rate,” which characterizes the percentage of the solution manifold in the joint space. This metric is used to rapidly assess the influence of different parameterized joints on the manipulator's motion planning performance. To verify the proposed method's correctness, we evaluated the performance of different representations with the MOEA/D algorithm in solving the same path optimization problems based on the algorithm running time and overall motion magnitude of the manipulator. Our simulation results demonstrate that different selections of parameterized joints affect the motion planning performance, and the performance planned by the optimal parametric representation is up to four times greater than that of the worst one.

Coverage path planning (CPP) is the task of computing an optimal path within a region to completely scan or survey the area of interest by using robotic sensor footprints. In this work, we propose a novel approach to find the multi-robot optimal coverage path of an agricultural field using motion dynamics while minimizing the mission time. Our approach consists of three steps: (i) divide the agricultural field into convex polygonal areas to optimally distribute them among the robots, (ii) generate an optimal coverage path to ensure minimum coverage time for each of the polygonal areas, and (iii) generate the trajectory for each coverage path using Dubins motion dynamics. Several experiments and simulations were performed to check the validity and feasibility of our approach, and the results and limitations are discussed.

Many methods that solve robot planning problems, such as task and motion planners, employ discrete symbolic search to find sequences of valid symbolic actions that are grounded with motion planning. Much of the efficacy of these planners lies in this grounding-bad placement and grasp choices can lead to inefficient planning when a problem has many geometric constraints. Moreover, grounding methods such as naïve sampling often fail to find appropriate values for these choices in the presence of clutter. Towards efficient task and motion planning, we present a novel optimization-based approach for grounding to solve cluttered problems that have many constraints that arise from geometry. Our approach finds an optimal grounding and can provide feedback to discrete search for more effective planning. We demonstrate our method against baseline methods in complex simulated environments.

We study the problem of allocating many mobile robots for the execution of a pre-defined sweep schedule in a known two-dimensional environment, with applications toward search and rescue, coverage, surveillance, monitoring, pursuit-evasion, and so on. The mobile robots (or agents) are assumed to have one-dimensional sensing capability with probabilistic guarantees that deteriorate as the sensing distance increases. In solving such tasks, a time-parameterized distribution of robots along the sweep frontier must be computed, to minimize the number of robots used to achieve some desired coverage quality guarantee or to maximize the probabilistic guarantee for a given the number of robots. We propose a max-flow-based algorithm for solving the allocation task, which builds on a decomposition technique of the workspace as a generalization of the well-known boustrophedon decomposition. Our proposed algorithm has a very low polynomial running time and completes in under two seconds for polygonal environments with over 105 vertices. Simulation experiments are carried out on three realistic use cases with randomly generated obstacles of varying shapes, sizes, and spatial distributions, demonstrating our proposed method's applicability and scalability. Introduction video: https://youtu.be/8taX92rzC5k.

This paper presents the novel use of air gaps in flexible optical light pipes to create coded patterns for use in bend localization. The OptiGap sensor system allows for the creation of extrinsic intensity modulated bend sensors that function as flexible absolute linear encoders. Coded air gap patterns are identified by a Gaussian naive Bayes (GNB) classifier running on an STM32 microcontroller. The fitting of the classifier is aided by a custom software suite that simplifies data collection and processing from the sensor. The sensor model is analyzed and verified through simulation and experiments, highlighting key properties and parameters that aid in the design of OptiGap sensors using different light pipe materials and for various applications. The OptiGap system allows for real-time and accurate bend localization in many robotics and automation applications, in both wet and dry conditions.

Human-assistive systems, such as robots, need to correctly understand the surrounding situation based on obser-vations and output the required support actions for humans. Language is one of the important channels to communicate with humans, and robots are required to have the ability to express their understanding and action-planning results. In this study, we propose a new task of operative action captioning that estimates and verbalizes the actions to be taken by the system in a human-assisting domain. We constructed a system that outputs a verbal description of a possible operative action that changes the current state to the given target state. We collected a dataset consisting of two images as observations, which express the current state and the state changed by actions and a caption that describes the actions that change the current state to the target state, by crowdsourcing in daily life situations. Then we constructed a system that estimates an operative action by a caption. Since the operative action's caption is expected to contain some state-changing actions, we use scene graph prediction as an auxiliary task because the events written in the scene graphs correspond to the state changes. Experimental results showed that our system successfully described the operative actions that should be conducted between the current and target states. The auxiliary tasks that predict the scene graphs improved the quality of the estimation results.

Typical robotic systems rely on models for planning. Therefore, the quality of the robot's behavior is heavily dependent on how accurately the model can predict the outcome of the robot's actions in the environment. A challenge, however, is that no model is perfect; moreover, we often do not know where discrepancies between the model's prediction and the actual outcome occur prior to observing executions in the real-world. One way to address this is to bias the planner away from these discrepancies by inflating the cost of states and actions where we previously observed the model to be inaccurate. Making such decisions about where and how to bias purely at the planning-level, however, neglects valuable information from the control-level, which gives a more fine-grained understanding of where and how the model went wrong during execution. Based on this observation, our key idea is to first infer a statistical model over discrepancies in the control-level's model. Then, we translate this model to the planning-level, where we use it to more informatively bias the planner away from states and actions where the model's predicted outcome is likely to be inaccurate. We demonstrate that our framework enables a robot to complete tasks, despite an inaccurate planning model, with greater efficiency than existing approaches. We do so through an experimental evaluation in simulation and real-robot experiments on NASA's Astrobee free-flyer.

Large language models (LLMs) have unlocked new capabilities of task planning from human instructions. However, prior attempts to apply LLMs to real-world robotic tasks are limited by the lack of grounding in the surrounding scene. In this paper, we develop NLMap, an open-vocabulary and queryable scene representation to address this problem. NLMap serves as a framework to gather and integrate contextual information into LLM planners, allowing them to see and query available objects in the scene before generating a context-conditioned plan. NLMap first establishes a natural language queryable scene representation with Visual Language models (VLMs). An LLM based object proposal module parses instructions and proposes involved objects to query the scene representation for object availability and location. An LLM planner then plans with such information about the scene. NLMap allows robots to operate without a fixed list of objects nor executable options, enabling real robot operation unachievable by previous methods. Project website: https://nlmap-saycan.github.io.

This paper presents OpTaS, a task specification Python library for Trajectory Optimization (TO) and Model Predictive Control (MPC) in robotics. Both TO and MPC are increasingly receiving interest in optimal control and in particular handling dynamic environments. While a flurry of software libraries exists to handle such problems, they either provide interfaces that are limited to a specific problem formulation (e.g. TracIK, CHOMP), or are large and statically specify the problem in configuration files (e.g. EXOTica, eTaSL). OpTaS, on the other hand, allows a user to specify custom nonlinear constrained problem formulations in a single Python script allowing the controller parameters to be modified during execution. The library provides interface to several open source and commercial solvers (e.g. IPOPT, SNOPT, KNITRO, SciPy) to facilitate integration with established workflows in robotics. Further benefits of OpTaS are highlighted through a thorough comparison with common libraries. An additional key advantage of OpTaS is the ability to define optimal control tasks in the joint-space, task-space, or indeed simultaneously. The code for OpTaS is easily installed via pip, and the source code with examples can be found at github.com/cmower/optas.