IROS 2020 Papers

The lane change maneuver is one of the typical maneuvers in various driving situations. Therefore the automatic lane change function is one of the key functions for autonomous vehicles. Many researches have been conducted in this field. Most existing work focused on the solutions for the static environment and assume that the surrounding vehicles are running at constant speeds. However, in reality, if not all the vehicles on the road are fully autonomous, the situation could be much more complicated and the ego vehicle has to deal with the dynamic environment. This paper proposes a Model Predictive Control (MPC)-based method to achieve automatic lane change in a dynamic environment. A two-wheel dynamic bicycle model, which combines the longitudinal and lateral motion of the ego vehicle, together with a utility function, which helps to automatically determine the target lane have been used in the algorithm. The simulation results have demonstrated the capability of the proposed algorithm in a dynamic environment.

An important issue when synthesizing legged locomotion plans is the combinatorial complexity that arises from gait pattern selection. Though it can be defined manually, the gait pattern plays an important role in the feasibility and optimality of a motion with respect to a task. Replacing human intuition with an automatic and efficient approach for gait pattern selection would allow for more autonomous robots, responsive to task and environment changes. To this end, we propose the idea of building a map from task to gait pattern selection for given environment and performance objective. Indeed, we show that for a 2D half-cheetah model and a quadruped robot, a direct mapping between a given task and an optimal gait pattern can be established. We use supervised learning to capture the structure of this map in a form of gait regions. Furthermore, we propose to construct a warm-starting trajectory for each gait region. We empirically show that these warm-starting trajectories improve the convergence speed of our trajectory optimization problem up to 60 times when compared with random initial guesses. Finally, we conduct experimental trials on the ANYmal robot to validate our method.

Current unmanned aerial vehicle (UAV) visual tracking algorithms are primarily limited with respect to: (i) the kind of size variation they can deal with, (ii) the implementation speed which hardly meets the real-time requirement. In this work, a real-time UAV tracking algorithm with powerful size estimation ability is proposed. Specifically, the overall tracking task is allocated to two 2D filters: (i) translation filter for location prediction in the space domain, (ii) size filter for scale and aspect ratio optimization in the size domain. Besides, an efficient two-stage re-detection strategy is introduced for long-term UAV tracking tasks. Large-scale experiments on four UAV benchmarks demonstrate the superiority of the presented method which has computation feasibility on a low-cost CPU.

In this paper, we propose an abstraction that captures high-level formation and location-based swarm behaviors, and an automated control synthesis framework to generate correct-by-construction behaviors. Our abstraction includes symbols representing both possible formations and physical locations in the workspace. We allow users to write linear temporal logic (LTL) specifications over the symbols to specify high-level tasks for the swarm. To satisfy a specification, we automatically synthesize a centralized symbolic plan, and environment and swarm-size-dependent motion controllers that are guaranteed to implement the symbolic transitions. In addition, using integer programming (IP), we assign robots to different sub-swarms to execute the synthesized symbolic plan. Our framework gives insights into controlling a large fleet of autonomous robots to achieve complex tasks which require composition of behaviors at different locations and coordination among different groups of robots in a correct-by-construction way. We demonstrate the proposed framework in simulation with 16 UAVs and 8 ground vehicles, and on a physical platform with 20 ground robots, showcasing the generality of the approach and discussing the implications of controlling constrained physical hardware.

This paper presents a model-free approach to automate folding of a deformable object with robot manipulators, where its surface was labelled with markers to facilitate vision-based control and alignment. While performing the task involves solving nonconvex or nonlinear terms, in this paper, linearization was first performed to approximate the problem. By using the Levenberg-Marquardt algorithm, the task of folding a deformable thin object can be reformulated as a convex optimization problem. The mapping relationship between the motions of markers on the image and the joint inputs of the robot manipulator was evaluated through a Jacobian matrix. To account for the uncertainty in the matrix due to the deformable object, a two-stage evaluation scheme, which consists of approximate-rigidity rule and Broyden-update rule, was performed. Proper constraints were also added to avoid causing damage to the object. The performance and the robustness of the proposed approach were examined through simulation using Bullet simulator. The video of the simulation can be retrieved from the attachment. The results confirm that the thin object can be precisely folded together based on different markers labelled on the surface.

Currently, laparoscopic surgery systems are adapted for a large number of indications and patients and are therefore not optimized for one specific case. The challenge to create systems with an optimized kinematic structure for a specific patient regarding reachability and manipulability in the needed workspace is the automated design and construction process. We have developed an automated design and construction process for a patient-specific Single Incision Laparoscopic System that is optimized for a specific indication, procedure, patient, and surgeon. The kinematic structure is adapted to the required workspace, needed instrumentation, and manufacturing parameters. First results show that the patient-specific Single Incision Laparoscopic System is better suited for the specific application regarding the combination of reachability, manipulability, and system size in the required workspace than the standard Single Incision Laparoscopic System in different standard sizes or one simple standard size.

Given an image or a video captured from a monocular camera, amodal layout estimation is the task of predicting semantics and occupancy in bird's eye view. The term amodal implies we also reason about entities in the scene that are occluded or truncated in image space. While several recent efforts have tackled this problem, there is a lack of standardization in task specification, datasets, and evaluation protocols. We address these gaps with AutoLay, a dataset and benchmark for amodal layout estimation from monocular images. AutoLay encompasses driving imagery from two popular datasets: KITTI [1] and Argoverse [2]. In addition to fine-grained attributes such as lanes, sidewalks, and vehicles, we also provide semantically annotated 3D point clouds. We implement several baselines and bleeding edge approaches, and release our data and code.1.

This work proposes a novel method of incorporating calls to a motion planner inside a potential field control policy for safe multi-robot navigation with uncertain dynamics. The proposed framework can handle more general scenes than the control policy and has low computational costs. Our work is robust to uncertain dynamics and quickly finds high-quality paths in scenarios generated from real-world floor plans. In the proposed approach, we attempt to follow the control policy as much as possible, and use calls to the motion planner to escape local minima. Trajectories returned from the motion planner are followed using a path-following controller guaranteeing robustness. We demonstrate the utility of our approach with experiments based on floor plans gathered from real buildings.

Recent advances in the design of head-mounted augmented reality (AR) interfaces for assistive human-robot interaction (HRI) have allowed untrained users to rapidly and fluently control single-robot platforms. In this paper, we investigate how such interfaces transfer onto multirobot architectures, as several assistive robotics applications need to be distributed among robots that are different both physically and in terms of software. As part of this investigation, we introduce a novel head-mounted AR interface for heterogeneous multirobot control. This interface generates and displays dynamic joint-affordance signifiers, i.e. signifiers that combine and show multiple actions from different robots that can be applied simultaneously to an object. We present a user study with 15 participants analysing the effects of our approach on their perceived fluency. Participants were given the task of filling-out a cup with water making use of a multirobot platform. Our results show a clear improvement in standard HRI fluency metrics when users applied dynamic joint-affordance signifiers, as opposed to a sequence of independent actions.

The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements. Previous appearances are also gradually forgotten due to the exponential decay of historical views in traditional appearance updating scheme of DCF framework, reducing the model's robustness. In this work, a novel tracker based on DCF framework is proposed to augment memory of previously appeared views while running at real-time speed. Several historical views and the current view are simultaneously introduced in training to allow the tracker to adapt to new appearances as well as memorize previous ones. A novel rapid compressed context learning is proposed to increase the discriminative ability of the filter efficiently. Substantial experiments on UAVDT and UAV123 datasets have validated that the proposed tracker performs competitively against other 26 top DCF and deep-based trackers with over 40fps on CPU.

Robot-assisted retinal surgery has become increasingly prevalent in recent years in part due to the potential for robots to help surgeons improve the safety of an immensely delicate and difficult set of tasks. The integration of robots into retinal surgery has resulted in diminished surgeon perception of tool-to-tissue interaction forces due to robot's stiffness. The tactile perception of these interaction forces (sclera force) has long been a crucial source of feedback for surgeons who rely on them to guide surgical maneuvers and to prevent damaging forces from being applied to the eye. This problem is exacerbated when there are unfavorable sclera forces originating from patient movements (dynamic eyeball manipulation) during surgery which may cause the sclera forces to increase even drastically. In this study we aim at evaluating the efficacy of providing warning auditory feedback based on the level of sclera force measured by force sensing instruments. The intent is to enhance safety during dynamic eye manipulations in robot-assisted retinal surgery. The disturbances caused by lateral movement of patient's head are simulated using a piezo-actuated linear stage. The Johns Hopkins Steady-Hand Eye Robot (SHER), is then used in a multi-user experiment. Twelve participants are asked to perform a mock retinal surgery by following painted vessels inside an eye phantom using a force sensing instrument while auditory feedback is provided. The results indicate that the users are able to handle the eye motion disturbances while maintaining the sclera forces within safe boundaries when audio feedback is provided.

This paper presents an bio-inspired event-based perception scheme for agile aerial robot maneuvering. It tries to mimic birds, which perform purposeful maneuvers by closing the separation in the retinal image (w.r.t. the goal) to follow time-to-contact trajectories. The proposed approach is based on event cameras, also called artificial retinas, which provide fast response and robustness against motion blur and lighting conditions. Our scheme guides the robot by only adjusting the position of features extracted in the event image plane to their goal positions at a predefined time using smooth time-to-contact trajectories. The proposed scheme is robust, efficient and can be added on top of commonly-used aerial robot velocity controllers. It has been validated on-board a UAV with real-time computation in low-cost hardware during sets of experiments with different descent maneuvers and lighting conditions.

This work presents an asynchronous multi-robot adaptive sampling strategy through the synthesis of an intermittently connected mobile robot communication network. The objective is to enable a team of robots to adaptively sample and model a nonlinear dynamic spatiotemporal process. By employing an intermittently connected communication network, the team is not required to maintain an all-time connected network enabling them to cover larger areas, especially when the team size is small. The approach first determines the next meeting locations for data exchange and as the robots move towards these predetermined locations, they take measurements along the way. The data is then shared with other team members at the designated meeting locations and a reducedorder-model (ROM) of the process is obtained in a distributed fashion. The ROM is used to estimate field values in areas without sensor measurements, which informs the path planning algorithm when determining a new meeting location for the team. The main contribution of this work is an intermittent communication framework for asynchronous adaptive sampling of dynamic spatiotemporal processes. We demonstrate the framework in simulation and compare different reduced-order models under full, all-time and intermittent connectivity.

We present a method to assist the operator in teleoperation of mobile robots by generating trajectories such that the vehicle completes the desired task with ease in unstructured environments. Traditional assisted teleoperation methods have focused on reactive methods to avoid collisions, but neglect the operator's intention in doing so. Instead, we generate long horizon, smooth trajectories that follow the operator's intended direction while circumventing obstacles for a seamless teleoperation experience. For mobile robot teleoperation, an explicit goal in the state space is often unclear in cases such as exploration or navigation. Therefore, we model the intent as a direction and encode it as a cost function. As trajectories of various lengths can satisfy the same directional objective, we iteratively construct a tree of sequential actions that form multiple trajectories along the intended direction. We show our algorithm on a real-time teleoperation task of a simulated hexarotor vehicle in a dense random forest environment. By doing so, our approach allows operator to achieve the navigation task while requiring less effort than reactive methods.

This paper presents a method for performing free-fall penetrometer tests for soft soils using an instrumented dart deployed by a quadcopter. Tests were performed with three soil types and used to examine the effect of drop height on the penetration depth and the deceleration profile. Further tests analyzed the force required to remove a dart from the soil and the effect of pulling at different speeds and angles. The pull force of a consumer drone was measured, and tests were performed where a drone delivered and removed darts in soil representative of a wetland environment.

To realize effective heterogeneous multi-agent teams, we must be able to leverage individual agents' relative strengths. Recent work has addressed this challenge by introducing trait-based task assignment approaches that exploit the agents' relative advantages. These approaches, however, assume that the agents' traits remain static. Indeed, in real-world scenarios, traits are likely to vary as agents execute tasks. In this paper, we present a transformation-based modeling framework to bridge the gap between state-of-the-art task assignment algorithms and the reality of dynamic traits. We define a transformation as a function that approximates dynamic traits with static traits based on a specific statistical measure. We define different candidate transformations, investigate their effects on different dynamic trait models, and the resulting task performance. Further, we propose a variance-based transformation as a general solution that approximates a variety of dynamic models, eliminating the need for hand specification. Finally, we demonstrate the benefits of reasoning about dynamic traits both in simulation and in a physical experiment involving the game of capture-the-flag.

The joint detection of drivable areas and road anomalies is a crucial task for ground mobile robots. In recent years, many impressive semantic segmentation networks, which can be used for pixel-level drivable area and road anomaly detection, have been developed. However, the detection accuracy still needs improvement. Therefore, we develop a novel module named the Normal Inference Module (NIM), which can generate surface normal information from dense depth images with high accuracy and efficiency. Our NIM can be deployed in existing convolutional neural networks (CNNs) to refine the segmentation performance. To evaluate the effectiveness and robustness of our NIM, we embed it in twelve state-of-the-art CNNs. The experimental results illustrate that our NIM can greatly improve the performance of the CNNs for drivable area and road anomaly detection. Furthermore, our proposed NIM-RTFNet ranks 8th on the KITTI road benchmark and exhibits a real-time inference speed.

Force perturbation is used in this paper to study cervical neuromuscular responses which can be used in the future to assess impairments in patients with neurological diseases. Current literature on this topic is limited to applying forces on the head in the anterior-posterior direction, perhaps due to technological limitations. In this paper, we propose to use a robotic neck brace to address these shortcomings due to its lightweight portable design and the ability to control forces. A controller is implemented to apply direction-specific perturbations on the head. To demonstrate the effectiveness of this capability, a human study was carried out with able-bodied subjects. We used this robotic brace to apply forces on the head of the subjects and observed their movement and muscle responses both when their eyes were open and closed. Our results suggest that the robotic brace is capable of perturbing the head and tracking the kinematic response. It revealed that ablebodied subjects reacted to the perturbations differently when their eyes were closed. They showed longer head trajectories and more muscle activation when the eyes were closed. We also show that the direction-specific perturbation feature enables us to analyze kinematic and muscle variables with respect to the direction of perturbation. This helps better understand the neuromuscular response in the head-neck.

The musculoskeletal humanoid has various biomimetic benefits, and it is important that we can embed and evaluate human reflexes in the actual robot. Although stretch reflex has been implemented in lower limbs of musculoskeletal humanoids, we apply it to the upper limb to discover its useful applications. We consider the implementation of stretch reflex in the actual robot, its active/passive applications, and the change in behavior according to the difference of parameters.