IROS 2022 Papers

This paper presents a novel concept to develop robots capable of crawling in tubular environments, inspired by the movement of earthworms and the biological musculoskeletal systems in nature. A tensegrity structures-based robotic module with shape changeability actuated by only one linear actuator is proposed. The mechanical structure of the robotic module is determined on the basis of force density method. By serially cascading three uniform modules, the in-pipe crawling robot is designed and manufactured. The robot has the abilities to crawl in both horizontal and vertical pipes with different inner diameters, and to pass through elbow pipes adaptively under the control of a simple actuation sequence. The effectiveness of the robot is demonstrated by experimental results on the prototype. Compared with existing robots, this proposed approach enables compact yet robust structures, along with enhanced compliance, mobility, and adaptability.

The fit of a wearable device, such as a prosthesis, can be quantitatively characterized by the mechanical coupling at the user-device interface. It is thought that the mechanical impedance, specifically the stiffness and damping, of wearable device interfaces can significantly impact human performance while using them. To test this theory, we develop a forearm-mounted testbed with a motorized, two degree of freedom (2-DOF) gimbal to simulate variations in the mechanical fit of an upper-extremity wearable device during pointing and target tracking tasks. The two gimbal motors are impedance-controlled to vary the mechanical stiffness and damping between the user and the device's laser pointer end-effector. In this paper, experiments are conducted to determine the torque constants of the motors before implementation in the testbed, and to validate the accuracy of the joint impedance controller. The completed impedance-controlled wearable interface testbed is validated further by comparing the gimbal joint displacements and torques, recorded during 2-DOF base excitation experiments, to MATLAB Simulink simulation data.

In this paper, an Event-triggered Image-based Visual Servoing Nonlinear Model Predictive Controller (ET-IBVS-NMPC) for multirotor aerial vehicles is presented. The proposed scheme is developed for the autonomous surveillance of contour-based areas with different characteristics (e.g. forest paths, coastlines, road pavements). For this purpose, an appropriately trained Deep Neural Network (DNN) is employed for the accurate detection of the contours. In an effort to reduce the remarkably large computational cost required by an IBVS-NMPC algorithm, a triggering condition is designed to define when the Optimal Control Problem (OCP) should be resolved and new control inputs will be calculated. Between two successive triggering instants, the control input trajectory is applied to the robot in an open-loop fashion, which means that no control input computations are required. As a result, the system's computing effort and energy consumption are lowered, while its autonomy and flight duration are increased. The visibility and input constraints, as well as the external disturbances, are all taken into account throughout the control design. The efficacy of the proposed strategy is demonstrated through a series of real-time experiments using a quadrotor and an octorotor both equipped with a monocular downward looking camera.

This paper reports on a new error-state Model Predictive Control (MPC) approach to connected matrix Lie groups for robot control. The linearized tracking error dynamics and the linearized equations of motion are derived in the Lie algebra. Moreover, given an initial condition, the linearized tracking error dynamics and equations of motion are globally valid and evolve independently of the system trajectory. By exploiting the symmetry of the problem, the proposed approach shows faster convergence of rotation and position simultaneously than the state-of-the-art geometric variational MPC based on variational-based linearization. Numerical simulation on tracking control of a fully-actuated 3D rigid body dynamics confirms the benefits of the proposed approach compared to the baselines. Furthermore, the proposed MPC is also verified in pose control and locomotion experiments on a quadrupedal robot MIT Mini Cheetah.

A skid-steer rover's power consumption is highly dependent on the turning radius of its path. For example, a point turn consumes a lot of power compared to a straight-line motion. Thus, in path planning for this kind of rover, turning radius is a factor that should be considered explicitly. There is a lack of any analytical approach in literature for finding energy-optimal paths for skid-steer rovers. The key contribution of this work is an energy-time equivalency theorem, for skid-steer rovers on obstacle-free hard ground. The theorem converts the energy-optimal problem into an equivalent time-optimal problem. This non-intuitive result stems from the fact that with this model of the system the total energy is fully parameterized by the geometry of the path alone. Hence, instead of directly solving the energy-optimal path planning problem, which is highly nonlinear, the equivalent time-optimal problem can be solved. Furthermore, experimental results are provided to experimentally prove the equivalency theorem while using Husky UGV skid-steer rover.

In this paper, we propose an accurate and simultaneously efficient solution to perspective-n-point-and-line (PnPL) problem by null space analysis. Although many PnPL-like methods have been proposed, it is hard to obtain the optimal solution considering both calculation efficiency and accuracy at the same time. Based on the remarkable EOPnP method, the proposed algorithm integrates linear-expressed line constraints with original point constraints, leading to a new solution named EOPnPL. For line error measurement, instead of using the algebraic error built with reprojected endpoints and image lines, we adopt line error function containing the distance from the reprojected midpoint of the model line segment to the image line and giving it a weight 4 times as that of the endpoints. A system of linear homogeneous equations only involving the rotation are derived, containing point and line constraints. Minima are obtained by a null space analysis according to rotation constraints and then the solution is selected by reprojection errors before iterative refinement. Experimental results show that the proposed method showcases great performances in both simulations and real-data experiment on VGG dataset. Great advantages are presented by EOPnPL compared to PnL methods when using only lines. Among the state-of-the-arts methods, the proposed method stands out with high accuracy under ordinary condition and comparable results in planar condition with the speed close to the fastest.

Cleaning robots are one of the market dominators in the commercialized robot space. So far, numerous robots have been introduced that can perform cleaning tasks in various settings, including floor, pavement, pool, lawn, windows, etc. However, none of the existing commercial cleaning robots targets the staircase, commonly found in multi-story buildings. Even though few works in the literature introduced robotic solutions for staircase cleaning, they primarily focused on cleaning the ascending staircase often, with a loose connection to access the descending staircase. In this paper, we propose a novel autonomous reconfigurable robotic platform called sTetro-D that can autonomously detect the descending staircase, approach the step, and perform area coverage in an unknown environment. The developed autonomy framework consists of two modes which are search mode and clean mode. In search mode, we implemented an RGB-D camera-based fusion technique wherein we combined the image bounding box from DCNN (Deep Convolution Neural Network) with the depth information to find the 3D first step pose that assists the robot in approaching it precisely. After the successful stair approach, the cleaning mode enables the staircase area coverage process. We described all these aspects and concluded with an experimental analysis of the proposed robotic system in a real-world scenario. The results demonstrate that the robot has a significant performance in detecting the descending staircase, staircase approach, and area coverage.

A fully tested autonomous system works predictably under ideal or assumed environment. However, its behavior is not fully defined when some components malfunction or fail. In this paper, we consider automated guided vehicle (AGV), equipped with multiple sensors, executing a traversal task in a static unknown environment. We have analytically studied the system, computed a set of performance and safety metrics, and validated it with simulation results in Webots. We have also analyzed the effect on system performance under independent and correlated sensing errors. We have also performed sensitivity analysis to identify the most critical components in any given system; and this can be utilized to increase the reliability of the system and its conformance to safety objectives.

For traditional cable-driven parallel robots (CD-PRs), changing the workspace is relatively difficult, which needs to reconfigure the anchor points and the external frame. The main reason is that the winch is separated from the moving platform, and a series of pulleys are applied to guide the driving cables. This paper proposes a novel all-in-one suspended CDPR that integrates all components in the moving platform to realize a flexible workspace. For the rapid construction of the CDPR, the ends of the cables only need to be connected to the existing anchor point. However, due to the flexible workspace, the position values of the anchor points should be recalibrated by appropriate calibration methods, especially by using a rapid auto-calibration method in application sites. Thus, based on the kinetostatic model considering sagging cable, an auto-calibration method with an inclinometer is proposed. Simulation and experiment are conducted respectively, and the experiment results indicate that 78.56% totally reduces the errors of the fixed anchor points. Moreover, experiments were carried out to validate the design and kinematics.

To communicate, the ~ 1.5 million Americans living with deafblindess use tactile American Sign Language (t-ASL). To provide Deafßilind (DB) individuals with a means of using their primary communication language without the use of an interpreter, we developed an assistive technology that promotes their autonomy. The TATUM (Tactile ASL Translational User Mechanism) anthropomorphic arm hand system leverages previous developments of a fingerspelling hand to sign more complex ASL words and phrases. The TATUM hand-wrist system is attached onto a 4 DOF robot arm and a human motion recognition and human to robot gesture transfer framework is used for signing recognition and replication. In particular, signing trajectories based on vision-based motion capture data from a sign demonstrator were used to control the robot's actuators. The performance of the system was evaluated through tactile based sign recognition performed by a blinded user and for its accuracy with novice, sighted users.

The wheel-bipedal robot has the advantages of both wheeled robots and legged robots, but as a cost, it is more challenging to perform flexible movements in various surroundings while keeping it balanced. The inaccurate dynamics of the robot makes the balance problem even more intractable. To solve this problem, the robot Ollie is used as a testbed. The whole-body control (WBC) framework is adopted to enhance the dexterity of the robot with multiple degrees of freedom in the task space. Moreover, a learning-based adaptive technique is applied to assist the WBC such that the balance controller can be designed in the absence of the accurate dynamics. Physical experiments demonstrate that the robot can manage various actions, with the help of the combination of the WBC and the learning-based adaptive technique.

In this work we present the design of a swimming robot that is inspired by the body shape modulation of small microorganisms. Amoebas are small single celled organisms that locomote through deformation and shape change of their body. To achieve similar shape modulation for swimming propulsion in a robot we developed a novel flexible appendage using tape springs. A tape spring is an elongated strip of metal with a curved cross-section that can act as a stiff structure when loaded against the curvature, while it can easily buckle when loaded with the curvature. We develop a tape spring appendage that is capable of freely deforming its perimeter through two actuation inputs. In the first portion of this paper we develop the kinematics of the appendage mechanisms and compare with experiment. Next we present the design of a surface locomoting robot that uses two appendages for propulsion. From the appendage kinematics we derive the local connection vector field for locomotion kinematics and study the optimal gait for forward swimming. Lastly, we demonstrate robot swimming performance in open water conditions. The novel appendage design in this robot is advantageous because it enables omnidirectional movement, the appendages will not tangle in debris, and they are robust to collisions and contact with structures.

Learning-based perception systems in robotics often requires large-scale image segmentation annotation. Current approaches rely on human labelers, which can be expensive, or simulation data, which can visually differ from real data. This paper proposes Labels from UltraViolet (LUV), a novel framework that enables rapid, automated, inexpensive, high quality data collection in real. LUV uses transparent, UV-fluorescent paint with programmable UV LEDs to collect paired images of a scene in standard and UV lighting. This makes it possible to autonomously extract segmentation masks and keypoints via color thresholding. We apply LUV to a suite of diverse robot perception tasks: locating fabric keypoints, cable segmentation, and surgical needle detection to evaluate its labeling quality, flexibility, and data collection rate. Results suggest that LUV is 180–2500 times faster than a human labeler across the tasks while retaining accuracy and strong task performance. Code, datasets, visualizations, and supplementary material can be found at https://sites.google.com/berkeley.edu/luv.

Due to labor shortage and rising labor cost for the apple industry, there is an urgent need for the development of robotic systems to efficiently and autonomously harvest apples. In this paper, we present a system overview and algorithm design of our recently developed robotic apple harvester prototype. Our robotic system is enabled by the close integration of several core modules, including visual perception, planning, and control. This paper covers the main methods and advancements in deep learning-based multi-view fruit detection and localization, unified picking and dropping planning, and dexterous manipulation control. Indoor and field experiments were conducted to evaluate the performance of the developed system, which achieved an average picking rate of 3.6 seconds per apple. This is a significant improvement over other reported apple harvesting robots with a picking rate in the range of 7–10 seconds per apple. The current prototype shows promising performance towards further development of efficient and automated apple harvesting technology. Finally, limitations of the current system and future work are discussed.

We propose a human-in-the-loop reinforcement learning mechanism to help robots learn emotional behavior. Unlike the previous methods of providing explicit feedback via pressing keyboard buttons or mouse clicks, we provide a more natural way for ordinary people to train social robots how to perform social tasks according to their preferences - facial expressions. The whole experiment is carried out on the desktop robot Haru, which is mainly used for the research of emotion and empathy participation. Our experimental results show that through learning from implicit feedback of facial features, Haru can quickly understand and dynamically adapt to individual preferences, and obtain a similar performance to learning from explicit feedback. In addition, we observe that the recognition error of human feedback will cause a “temporary regress” of the robot's learning performance, which is more obvious at the beginning of the training process. This phenomenon is shown to be correlated with the accuracy of recognizing negative implicit feedback.

Training a high-dimensional simulated agent with an under-specified reward function often leads the agent to learn physically infeasible strategies that are ineffective when deployed in the real world. To mitigate these unnatural behaviors, reinforcement learning practitioners often utilize complex reward functions that encourage physically plausible behaviors. However, a tedious labor-intensive tuning process is often required to create hand-designed rewards which might not easily generalize across platforms and tasks. We propose substituting complex reward functions with “style rewards” learned from a dataset of motion capture demonstrations. A learned style reward can be combined with an arbitrary task reward to train policies that perform tasks using naturalistic strategies. These natural strategies can also facilitate transfer to the real world. We build upon Adversarial Motion Priors - an approach from the computer graphics domain that encodes a style reward from a dataset of reference motions - to demonstrate that an adversarial approach to training policies can produce behaviors that transfer to a real quadrupedal robot without requiring complex reward functions. We also demonstrate that an effective style reward can be learned from a few seconds of motion capture data gathered from a German Shepherd and leads to energy-efficient locomotion strategies with natural gait transitions.

Advances in deep learning have resulted in steady progress in computer vision with improved accuracy on tasks such as object detection and semantic segmentation. Nevertheless, deep neural networks are vulnerable to adversarial attacks, thus presenting a challenge in reliable deployment. Two of the prominent tasks in 3D scene-understanding for robotics and advanced driver assistance systems are monocular depth and pose estimation, often learned together in an unsupervised manner. While studies evaluating the impact of adversarial attacks on monocular depth estimation exist, a systematic demonstration and analysis of adversarial perturbations against pose estimation are lacking. We show how additive imperceptible perturbations can not only change predictions to increase the trajectory drift but also catastrophically alter its geometry. We also study the relation between adversarial perturbations targeting monocular depth and pose estimation networks, as well as the transferability of perturbations to other networks with different architectures and losses. Our experiments show how the generated perturbations lead to notable errors in relative rotation and translation predictions and elucidate vulnerabilities of the networks.11Code can be found at https://github.com/NeurAI-Lab/mono-pose-attack.

The common approach for local navigation on challenging environments with legged robots requires path planning, path following and locomotion, which usually requires a locomotion control policy that accurately tracks a commanded velocity. However, by breaking down the navigation problem into these sub-tasks, we limit the robot's capabilities since the individual tasks do not consider the full solution space. In this work, we propose to solve the complete problem by training an end-to-end policy with deep reinforcement learning. Instead of continuously tracking a precomputed path, the robot needs to reach a target position within a provided time. The task's success is only evaluated at the end of an episode, meaning that the policy does not need to reach the target as fast as possible. It is free to select its path and the locomotion gait. Training a policy in this way opens up a larger set of possible solutions, which allows the robot to learn more complex behaviors. We compare our approach to velocity tracking and additionally show that the time dependence of the task reward is critical to successfully learn these new behaviors. Finally, we demonstrate the successful deployment of policies on a real quadrupedal robot. The robot is able to cross challenging terrains, which were not possible previously, while using a more energy-efficient gait and achieving a higher success rate. Supplementary videos can be found on the project website: https://sites.google.com/leggedrobotics.com/end-to-end-loco-navigation

For universal robotic limbs, having a large workspace with high stiffness and adjustable output properties is important to adapt to various situations. A combination of parallel mechanisms that can change output characteristics is promising to meet these demands. As such, we propose a lever mechanism with double parallel link platforms. This mechanism is composed of a lever mechanism with the effort point and the pivot point; each is supported by a parallel link mechanism. First, we calculated the differential kinematics of this mechanism. Next, we investigated the workspace of the mechanism. The proposed mechanism can reach nearer positions than the posture with the most shrinking actuators thanks to the three-dimensional movable effort point. Then, we confirmed that this mechanism could change the output force profile at the end-effector by changing the lever ratio. The main change is the directional change of the maximum output force. The change range is larger when the squatting depth is larger. The changing tendency of the shape of the maximum output force profile by the position of the pivot plate depends on the force balance of the actuators. These analytical results show the potential of the proposed mechanism and would aid in the design of this mechanism for robotic limbs.