IROS 2021 Papers

Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. However, they may suffer from sample inefficiency, which leads to computational burden for finding a global optimal path. To address this issue, we suggest a perception-aware path planner which utilizes topological information of environments. Since the topological class of a path and visible landmarks during traveling the path are closely related, the proposed algorithm checks distinctive topological classes to choose the class with abundant visual information. Topological graph is extracted from the generalized Voronoi diagram of the environment and initial paths with different topological classes are found. To evaluate the perception quality of the classes, we divide the initial path into discrete segments where the points in each segment share similar visual information. The optimal class with high perception quality is selected, and a graph-based planner is utilized to generate path within the class. With simulations and real-world experiments, we confirmed that the proposed method could guarantee accurate visual navigation compared with the perception-agnostic method while showing improved computational efficiency than the sampling-based perception-aware planner.

The Modboat is an autonomous surface robot that turns the oscillation of a single motor into a controlled paddling motion through passive flippers. Inertial control methods developed in prior work can successfully drive the Modboat along trajectories and enable docking to neighboring modules, but have a non-constant cycle time and cannot react to dynamic environments. In this work we present a thrust direction control method for the Modboat that significantly improves the time-response of the system and increases the accuracy with which it can be controlled. We experimentally demonstrate that this method can be used to perform more compact maneuvers than prior methods or comparable robots can. We also present an extension to the controller that solves the reaction wheel problem of unbounded actuator velocity, and show that it further improves performance.

The quality of robot vision greatly affects the performance of automation systems, where occlusions stand as one of the biggest challenges. If the target is occluded from the sensor, detecting and grasping such objects become very challenging. For example, when multiple robot arms cooperate in a single workplace, occlusions will be created under the robot arm itself and hide objects underneath. While occlusions can be greatly reduced by installing multiple sensors, the increase in sensor costs cannot be ignored. Moreover, the sensor placements must be rearranged every time the robot operation routine and layout change.To diminish occlusions, we propose the first robot vision system with tilt-type mirror reflection sensing. By instantly tilting the sensor itself, we obtain two sensing results with different views: conventional direct line-of-sight sensing and non-line-of-sight sensing via mirror reflections. Our proposed system removes occlusions adaptively by detecting the occlusions in the scene and dynamically configuring the sensor tilt angle to sense the detected occluded area. Thus, sensor rearrangements are not required even after changes in robot operation or layout. Since the required hardware is the tilt-unit and a commercially available mirror, the cost increase is marginal. Through experiments, we show that our system can achieve a similar detection accuracy as systems with multiple sensors, regardless of the single-sensor implementation.

Legged robots primarily energize their center of mass through external contact during stance phase. This links their range of possible motions to actuator power limits applied during usually short periods of time. Enabling limb actuators to pump energy into the system during non-contact phases can greatly extend the energetic profile of possible motions. However, funneling this extra energy into useful dynamics is a problem of its own. In this paper, we propose designing limb mechanisms that purposefully integrate kinematic singularities into their configuration space to enable airborne energization and enhance gait periodicity before sensors and feedback control are incorporated. The result produces a mechanical reflex that unloads spring energy stored during flight phase into a useful push-off motion as triggered by the onset of stance phase. The implementation generates an "S" shaped curve into a specific slice of the configuration space, motivating the name S-curve. We compare our proposed approach to more conventional strategies and survey the design parameters that can be used to shape an S-curve. Ranges of useful S-curves are determined through a simulation study and a linkage mechanism capable of producing an S-curve is displayed.

The need for robotic hands capable of gentle in-hand manipulation is growing rapidly as robots enter the real world. In this work, we show that the arrangement of digits in a soft robotic hand has a strong effect on in-hand manipulation capabilities. Introducing task-based performance metrics which quantify the range of motion, repeatability, and accuracy of in-hand manipulation tasks, we investigate hand designs with finger arrangements ranging from axisymmetric-circular to anthropomorphic. Using an open-source soft robot simulator, the effect of object size and aspect ratio on the in-hand manipulation performance is studied for a variety of finger arrangements, and findings are validated using a physical hardware platform. We found that the ideal finger arrangement is task-dependent; anthropomorphic arrangements excel at lateral translations, and axisymmetric arrangements are best suited for rotations. The aspect ratio of the object also has a strong effect on in-hand manipulation, with anthropomorphic designs performing best on objects of high aspect ratio, and axisymmetric arrangements doing well on objects of low aspect ratio. These findings are further confirmed in a real-world task with delicate pastries, where gentle in-hand manipulation is critical. Overall, our results suggest that active control of digit arrangement is necessary for soft robotic hands to maximize in-hand manipulation capabilities with arbitrary objects.

Autonomous vehicles must balance a complex set of objectives. There is no consensus on how they should do so, nor on a model for specifying a desired driving behavior. We created a dataset to help address some of these questions in a limited operating domain. The data consists of 92 traffic scenarios, with multiple ways of traversing each scenario. Multiple annotators expressed their preference between pairs of scenario traversals. We used the data to compare an instance of a rulebook [1], carefully hand-crafted independently of the dataset, with several interpretable machine learning models such as Bayesian networks, decision trees, and logistic regression trained on the dataset. To compare driving behavior, these models use scores indicating by how much different scenario traversals violate each of 14 driving rules. The rules are interpretable and designed by subject-matter experts. First, we found that these rules were enough for these models to achieve a high classification accuracy on the dataset. Second, we found that the rulebook provides high interpretability without excessively sacrificing performance. Third, the data pointed to possible improvements in the rulebook and the rules, and to potential new rules. Fourth, we explored the interpretability vs performance trade-off by also training non-interpretable models such as a random forest. Finally, we make the dataset publicly available to encourage a discussion from the wider community on behavior specification for AVs. Please find it at github.com/bassam-motional/Reasonable-Crowd.

Radar sensors have a long tradition in advanced driver assistance systems (ADAS) and also play a major role in current concepts for autonomous vehicles. Their importance is reasoned by their high robustness against meteorological effects, such as rain, snow, or fog, and the radar’s ability to measure relative radial velocity differences via the Doppler effect. The cause for these advantages, namely the large wavelength, is also one of the drawbacks of radar sensors. Compared to camera or lidar sensor, a lot more surfaces in a typical traffic scenario appear flat relative to the radar’s emitted signal. This results in multi-path reflections or so called ghost detections in the radar signal. Ghost objects pose a major source for potential false positive detections in a vehicle’s perception pipeline. Therefore, it is important to be able to segregate multipath reflections from direct ones. In this article, we present a dataset with detailed manual annotations for different kinds of ghost detections. Moreover, two different approaches for identifying these kinds of objects are evaluated. We hope that our dataset encourages more researchers to engage in the fields of multi-path object suppression or exploitation.

This paper investigates the pursuit-evasion problem of a defensive gun turret and one or more attacking drones. The turret must "visit" each attacking drone once, as quickly as possible, to defeat the threat. This constitutes a Shortest Hamiltonian Path (SHP) through the drones. The investigation considers situations with increasing fidelity, starting with a 2D kinematic model and progressing to a 3D dynamic model. In 2D we determine the region from which one or more drones can always reach a turret, or the region close enough to it where they can evade the turret. This provides optimal starting angles for n drones around a turret and the maximum starting radius for one and two drones.We show that safety regions also exist in 3D and provide a controller so that a drone in this region can evade the pan-tilt turret. Through simulations we explore the maximum range n drones can start and still have at least one reach the turret, and analyze the effect of turret behavior and the drones’ number, starting configuration, and behaviors.

Micro Aerial Vehicles (MAV) with Vertical Takeoff and Landing (VTOL) capabilities, such as quadrotors, have offered significant value to many research fields and markets. However, only recently, MAV began to be explored as systems capable of interacting with the environment, performing manipulation tasks, and participating in more versatility-demanding operations. Pursuing the goal of turning flying machines into more versatile instruments, many researchers have resorted to using tilting rotor mechanisms to create new aerial vehicle concepts. Nevertheless, most such new concepts are bulky and lack the required versatility, and are restricted to particular applications. In this work, we address these issues by proposing a novel coaxial, versatile, modular tilt-rotor UAV concept. The Omnirotor platform can apply its full thrust in any direction, regardless of the frame’s orientation where it is mounted. The platform does not have any limitations regarding rotation’s range. It can change its thrust direction continuously without needing to unwind back to a specific configuration. With the addition of control surfaces between the coaxial rotors, the Omnirotor is turned into a functional VTOL MAV with hovering capabilities that can be used as a ground vehicle, a UAV, and an all-terrain vehicle.

The ongoing global healthcare crisis has amplified the need for automation of manual tasks in several industries and service sectors. Simple household tasks such as tidying and cleaning are in high demand, with only a few robotic platforms capable of performing them due to the mobility, workspace, and dexterity requirements. This work presents ARoA, an autonomous robotic assistant that can execute complex tasks in industrial, service, and home environments. It is equipped with two lightweight, compliant, 7 degree of freedom arms and a pair of adaptive end-effectors that enable efficient execution of a wide range of tasks. Due to the linear rail based torso system that supports the arms, the ARoA offers exceptional flexibility in terms of reachable workspace. A framework for vision-based execution of tidying and cleaning tasks is also proposed and integrated in the platform. The efficiency of the ARoA platform was experimentally validated through two everyday life applications: i) picking up and tidying randomly scattered household objects and ii) cleaning of common surfaces.

Identification of textile properties is an important milestone toward advanced robotic manipulation tasks that consider interaction with clothing items such as assisted dressing, laundry folding, automated sewing, textile recycling and reusing. Despite the abundance of work considering this class of deformable objects, many open problems remain. These relate to the choice and modelling of the sensory feedback as well as the control and planning of the interaction and manipulation strategies. Most importantly, there is no structured approach for studying and assessing different approaches that may bridge the gap between the robotics community and textile production industry. To this end, we outline a textile taxonomy considering fiber types and production methods, commonly used in textile industry. We devise datasets according to the taxonomy, and study how robotic actions, such as pulling and twisting of the textile samples, can be used for the classification. We also provide important insights from the perspective of visualization and interpretability of the gathered data.

Studies on the production of emotions have been conducted to create robotic facial expressions. The reported methodologies for generating emotions for a robot have focused on recognizing a user’s emotions using devices, such as cameras and microphones, and then generating the reactive emotions of a robot according to the user’s emotions. However, these methodologies may have some limitations in delivering emotions in the robot that match the robot’s utterances to users. In this paper, we propose a methodology for producing robotic emotions suitable for a robot’s utterances based on texts so that it can be applied to various fields such as robotic dialogue and reading services. To produce human-like emotions in a robot, our methodology applied patterns of human emotional changes as well as the resilience theory that humans have an ability to recover their emotional states considering their own personality over time. We measured the performance of the model for analyzing texts and observed that there is a linear correlation between the predicted emotions and the annotated ones from humans. Furthermore, when we carried out the experiments based on scenarios, our methodology could produce human-like patterns of emotional changes and the robot could recover its emotional state on its own.

When it comes to the control system of quadruped robots, deep reinforcement learning (DRL) is considered to be a promising solution. Despite years of development in this field, difficulties remain in guaranteeing the action stability of DRL-based quadruped robots’ locomotion, especially in tough terrain. In this paper, a terrain-aware teacher-student controller integrating a risk assessment network (RAN) is proposed to alleviate this problem. During the training phase, the RAN can evaluate the risk level of historical observation or current state and further guide the update of the policy, thereby assisting the policy in selecting better actions and avoid risky ones. Furthermore, the real-time elevation map is transmitted to the controller as visual information, so that it can perceive the terrain to produce higher performance locomotion. With the aforementioned configuration, we enable a robot to traverse various challenging terrain in simulation and bound or trot stably in the real environment.

A commonly-used representation for motion prediction of actors is a sequence of waypoints (comprising positions and orientations) for each actor at discrete future time-points. While regressing waypoints is simple and flexible, it can exhibit unrealistic higher-order derivatives (such as acceleration) and approximation errors at intermediate time steps. To address this issue we propose a general representation for temporally-continuous probabilistic trajectory prediction that regresses polynomial parameterization coefficients. We evaluate the proposed representation on supervised trajectory prediction tasks using two large self-driving data sets. The results show realistic higher-order derivatives and better accuracy at interpolated time-points, as well as the benefits of the inferred noise distributions over the trajectories. Extensive experimental studies based on existing state-of-the-art models demonstrate the effectiveness of the proposed approach relative to other representations in predicting the future motions of vehicle, bicyclist, and pedestrian traffic actors.

6D object pose estimation is an essential task in vision-based robotic grasping and manipulation. Prior works extract spatial features by fusing the RGB image and depth without considering the temporal motion information, limiting their performance in heavy occlusion robotic grasping scenarios. In this paper, we present an end-to-end model named TemporalFusion, which integrates the temporal motion information from RGB-D images for 6D object pose estimation. The core of proposed TemporalFusion model is to embed and fuse the temporal motion information from multi-frame RGB-D sequences, which could handle heavy occlusion in robotic grasping tasks. Furthermore, the proposed deep model can also obtain stable pose sequences, which is essential for real-time robotic grasping tasks. We evaluated the proposed method in the YCB-Video dataset, and experimental results show our model outperforms state-of-the-art approaches. Our code is available at https://github.com/mufengjun260/TemporalFusion21.

Humans can intuitively grasp objects of different shape and weight. Throughout the grasp execution they control and coordinate the grasp forces at all contact points between the hand and the object to achieve a stable grasp. Dexterous grasping with humanoid hands relies on the perfect coordination between grasp posture and force balance at the contact points in a high dimensional space and remains a challenge. In this paper, we present temporal force synergies describing the change in human grasp forces during the grasp execution in a low-dimensional space based on two new grasp synergy models: 1) static force synergies that are derived by a Principal Component Analysis and represent temporal grasp forces as a sequence of time-independent synergy configurations and 2) dynamic force synergies that are learned by a recurrent neural network and encode the temporal change of grasp forces throughout grasp execution in a latent synergy space clustered by grasp types. We show that both synergy spaces encode human grasp forces with an error of less than 2% and allow the generation of human-like grasp force patterns. Grasp forces for stable grasps described by the dynamic force synergies achieve a grasp quality comparable to demonstrated human grasps in simulation.

Extreme environments, such as search and rescue missions, defusing bombs, or exploring extraterrestrial planets, are unsafe environments for humans to be in. Robots enable humans to explore and interact in these environments through remote presence and teleoperation and virtual reality provides a medium to create immersive and easy-to-use teleoperation interfaces. However, current virtual reality interfaces are still very limited in their capabilities. In this work, we aim to advance robot teleoperation virtual reality interfaces by developing an environment reconstruction methodology capable of recognizing objects in a robot’s environment and rendering high fidelity models inside a virtual reality headset. We compare our proposed environment reconstruction method against traditional point cloud streaming by having operators plan waypoint trajectories to accomplish a pick-and-place task. Overall, our results show that users find our environment reconstruction method more usable and less cognitive work compared to raw point cloud streaming.

Many municipalities and large organizations have fleets of vehicles that need to be coordinated for tasks such as garbage collection or infrastructure inspection. Motivated by this need, this paper focuses on the common subproblem in which a team of vehicles needs to plan coordinated routes to patrol an area over iterations while minimizing temporally and spatially dependent costs. In particular, at a specific location (e.g., a vertex on a graph), we assume the cost accumulates over time and its growth rate is a random variable with a fixed but unknown mean, and the cost is reset to zero whenever any vehicle visits the vertex (representing the robot "servicing" the vertex). We formulate this problem in graph terminology and call it Team Orienteering Coverage Planning with Uncertain Reward (TOCPUR). We propose to solve TOCPUR by simultaneously estimating the accumulated cost at every vertex on the graph and solving a novel variant of the Team Orienteering Problem (TOP) iteratively, which we call the Team Orienteering Coverage Problem (TOCP). We provide the first mixed integer programming formulation for the TOCP, as a significant adaptation of the original TOP. We introduce a new benchmark consisting of hundreds of randomly generated graphs for comparing different methods. We show the proposed solution outperforms both the exact TOP solution and a greedy algorithm. In addition, we provide a demo of our method on a team of three physical robots in a real-world environment. The code is publicly available at https://github.com/Cranial-XIX/TOCPUR.git.

In this paper, we explore the problem of task-consistent path planning for printing-in-motion via Mobile Manipulators (MM). MM offer a potentially unlimited planar workspace and flexibility for print operations. However, most existing methods have only mobility to relocate an arm which then prints while stationary. In this paper we present a new fully autonomous path planning approach for mobile material deposition. We use a modified version of Rapidly-exploring Random Tree Star (RRT*) algorithm, which is informed by a constrained Inverse Reachability Map (IRM) to ensure task consistency. Collision avoidance and end-effector reachability are respected in our approach. Our method also detects when a print path cannot be completed in a single execution. In this case it will decompose the path into several segments and reposition the base accordingly.