ICRA 2022 Papers

Unresolved data association in ambiguous and perceptually aliased environments leads to multi-modal hypotheses on both the robot's and the environment state. To avoid catastrophic results, when operating in such ambiguous environments, it is crucial to reason about data association within Belief Space Planning (BSP). However, explicitly considering all possible data associations, the number of hypotheses grows exponentially with the planning horizon and determining the optimal action sequence quickly becomes intractable. Moreover, with hard budget constraints where some non-negligible hypotheses must be pruned, achieving performance guarantees is crucial. In this work we present a computationally efficient novel approach that utilizes only a distilled subset of hypotheses to solve BSP problems while reasoning about data association. Furthermore, to provide performance guarantees, we derive error bounds with respect to the optimal solution. We then demonstrate our approach in an extremely aliased environment, where we manage to significantly reduce computation time without compromising on the quality of the solution.

Cyclops, introduced in this paper, is an open research platform for everyone who wants to validate novel ideas and approaches in self-driving heavy-duty vehicle platooning. The platform consists of multiple 1/14 scale semi-trailer trucks equipped with associated computing, communication and control modules that enable self-driving on our scale proving ground. The perception system for each vehicle is composed of a lidar-based object tracking system and a lane detection/control system. The former maintains the gap to the leading vehicle, and the latter maintains the vehicle within the lane by steering control. The lane detection system is optimized for truck platooning, where the field of view of the front-facing camera is severely limited due to a small gap to the leading vehicle. This platform is particularly amenable to validating mitigation strategies for safety-critical situations. Indeed, the simplex architecture is adopted in the computing modules, enabling various fail-safe operations. In particular, we illustrate a scenario where the camera sensor fails in the perception system, but the vehicle is able to operate at a reduced capacity to a graceful stop. Details of Cyclops, including 3D CAD designs and algorithm source codes, are released for those who want to build similar testbeds.

3D environment mapping has been actively stud-ied recently with the development of autonomous driving and augmented reality. Although many image-based methods are proposed due to their convenience and flexibility compared to other complex sensors, few works focus on fixing the inherent scale ambiguity of image-based methods and registering the reconstructed structure to the real-world 3D map, which is very important for autonomous driving. This paper presents a low-cost mapping solution that is able to refine and align the monocular reconstructed point cloud given a public street map. Specifically, we first find the association between the street map and the reconstructed point cloud structure by a novel graph-based geolocalization method. Then, optimized with the corresponding relationship, the map accuracy is significantly improved. The rich environment information can also be associated with the point cloud by the geographical location. Experiments show that our geolocalization algorithm can locate the scene on a gigantic city-scale map (173.46 km2) in two minutes and support 3D map reconstruction with absolute scale and rich environmental information from Internet videos.

Providing guidance about collision avoidance, pedestrian trajectory prediction is an important task for autonomous driving. In this paper, to produce plausible trajectory predictions in the first-person view circumstance, we propose a crossmodal transformer based generative framework which could leverage sequences of cues from multiple modalities as well as pedestrian attributes. For the encoder, crossmodal transformers are exploited during the past stage to explore the cross-relation features of four modality-modality pairs, which are then fused with the help of a branch assigning operation and a modality attention module. For the decoder, we employ a bézier curve interpolation based method to project encoder features into trajectory results. Our training process not only considers the pedestrian's intention of crossing road but also optimizes our model to achieve more accurate predictions at the terminal time steps. Experimental results demonstrate that our framework outperforms state-of-the-art methods on both JAAD and PIE datasets. Especially, compared with the best baseline, our method could achieve 15.1%/14.3% and 14.3%/22.2% improvement for deterministic/multimodal prediction in the metric of box center final displacement error on JAAD and PIE, respectively.

Animals are able to imitate each others' behavior, despite their difference in biomechanics. In contrast, imitating other similar robots is a much more challenging task in robotics. This problem is called cross domain imitation learning (CDIL). In this paper, we consider CDIL on a class of similar robots. We tackle this problem by introducing an imitation learning algorithm based on invariant representation. We propose to learn invariant state and action representations, which align the behavior of multiple robots so that CDIL becomes possible. Compared with previous invariant representation learning methods for similar purposes, our method does not require human-labeled pairwise data for training. Instead, we use cycle-consistency and domain confusion to align the representation and increase its robustness. We test the algorithm on multiple robots in the simulator and show that unseen new robot instances can be trained with existing expert demonstrations successfully. Qualitative results also demonstrate that the proposed method is able to learn similar representations for different robots with similar behaviors, which is essential for successful CDIL.

Traversing soft soils represents a major concern of planetary rover missions. In this paper, we present a new chassis mechanism capable of a crawling gait that enhances trafficability on soft soil while relying on as few actuators as possible. Articulated by two actuated joints, MARCEL is a four-wheeled rover chassis which name stands for Mobile Active Rover Chassis for Enhanced Locomotion. MARCEL's crawling leverages a continuous adjustment of the load distribution on the four wheels using an internal torque applied between two halves of the chassis by series elastic actuation. This allows the pressure on two wheels to be minimized while they are moving forward with the assistance of the chassis's articulated motion. As a result, the wheels can be propelled forward one pair after another while avoiding the bulldozing resistance of the sand. This crawling motion is tested experimentally and is shown to generate more drawbar pull than both rolling or using a mere “push-pull” locomotion. Its ability to extricate the rover from deep sand entrapment is also tested successfully. This will allow future missions to deal with unforeseen terrain properties or to venture in more challenging areas while minimizing design complexity.

Coverage Path Planning is the work horse of contemporary service task automation, powering autonomous floor cleaning robots and lawn mowers in households and office sites. While steady progress has been made on indoor cleaning and outdoor mowing, these environments are small and with simple geometry compared to general urban environments such as city parking garages, highway bridges or city crossings. To pave the way for autonomous road sweeping robots to operate in such difficult and complex environments, a benchmark suite with three large-scale 3D environments representative of this task is presented. On this benchmark we evaluate a new Coverage Path Planning method in comparison with previous well performing algorithms, and demonstrate state-of-the-art performance of the proposed method. Part of the success, for all evaluated algorithms, is the usage of automated domain adaptation by in-the-loop parameter optimization using Bayesian Optimization. Apart from improving the performance, tedious and bias-prone manual tuning is made obsolete, which makes the evaluation more robust and the results even stronger.

This paper develops a decentralized approach to mobile sensor coverage by a multi-robot system. We consider a scenario where a team of robots with limited sensing range must position itself to effectively detect events of interest in a region characterized by areas of varying importance. Towards this end, we develop a decentralized control policy for the robots-realized via a Graph Neural Network-which uses inter-robot communication to leverage non-local information for control decisions. By explicitly sharing information between multi-hop neighbors, the decentralized controller achieves a higher quality of coverage when compared to classical approaches that do not communicate and leverage only local information available to each robot. Simulated experiments demonstrate the efficacy of multi-hop communication for multi-robot coverage and evaluate the scalability and transferability of the learning-based controllers.

Goal inference is of great importance for a variety of applications that involve interaction, coordination, and/or competition with goal-oriented agents. Typical goal inference approaches use as many pointwise measurements of the agent's trajectory as possible to pursue a most accurate a-posteriori estimate of the goal. However, taking frequent measurements may not be preferred in situations where sensing is associated with high cost (e.g., sensing + perception may involve high computational/bandwidth cost and sensing may raise security concerns in privacy-critical/data-sensitive applications). In such situations, a sensible tradeoff between the information gained from measurements and the cost associated with sensing actions is highly desirable. This paper introduces a cost-effective sensing strategy for goal inference tasks based on hybrid Kalman filtering and model predictive control. Our key insights include: 1) a model predictive approach can be used to predict the amount of information gained from new measurements over a horizon and thus to optimize the tradeoff between information gain and sensing action cost, and 2) the high computational efficiency of hybrid Kalman filtering can ensure real-time feasibility of such a model predictive approach. We evaluate the proposed cost-effective sensing approach in a goal-oriented task, where we show that compared to standard goal inference approaches, our approach takes a considerably reduced number of measurements while not impairing the speed, accuracy, and reliability of goal inference by taking measurements smartly.

We consider the problem of retrieving a target object from a confined space by two robotic manipulators where overhand grasps are not allowed. If other movable obstacles occlude the target, more than one object should be relocated to clear the path to reach the target object. With two robots, the relocation could be done efficiently by simultaneously performing relocation tasks. However, the precedence constraint between the tasks (e.g, some objects at the front should be removed to manipulate the objects in the back) makes the simultaneous task execution difficult. We propose a coordination method that determines which robot relocates which object so as to perform tasks simultaneously. Given a set of objects to be relocated, the objective is to maximize the number of switches between the robots in performing relocation tasks. Thus, one robot can pick an object in the clutter while the other robot places an object in hand to the outside of the clutter. However, the object to be relocated may not be accessible to all robots, so switching could not always be achieved. Our method is based on the uniform-cost search so the number of switches can be maximized. We also propose a greedy variant whose computation time is shorter. From experiments, we show that our method reduces the completion time of the mission by at least 22.9% (at most 27.3%) compared to the methods with no consideration of switching.

As collaborative robots move closer to human environments, motion generation and reactive planning strategies that allow for elaborate task execution with minimal easy-to-implement guidance whilst coping with changes in the environment is of paramount importance. In this paper, we present a novel approach for generating real-time motion plans for point-to-point tasks using a single successful human demonstration. Our approach is based on screw linear interpolation, which allows us to respect the underlying geometric constraints that characterize the task and are implicitly present in the demonstration. We also integrate an original reactive collision avoidance approach with our planner. We present extensive experimental results to demonstrate that with our approach, by using a single demonstration of moving one block, we can generate motion plans for complex tasks like stacking multiple blocks (in a dynamic environment). Analogous generalization abilities are also shown for tasks like pouring and loading shelves. For the pouring task, we also show that a demonstration given for one-armed pouring can be used for planning pouring with a dual-armed manipulator of different kinematic structure.

Cooperative transportation via multiple aerial robots has the potential to support various payloads and reduce the chances of them being dropped. Furthermore, autonomously controlled robots render the system scalable with respect to the payload. In this study, a cooperative transportation system was developed using rigidly attached single-rotor robots, and a decentralized controller was proposed to guarantee asymptotic stability of the error dynamics for unknown strictly positive real systems. A feedback controller was used to transform unstable systems into strictly positive real ones considering the shared attachment positions. First, the cooperative transportation of unknown payloads with different shapes larger than the carrier robots was investigated via numerical simulations. Second, cooperative transportation of an unknown payload (with a weight of approximately 2.7 kg and maximum length of 1.6 m) was demonstrated using eight robots, even under robot failure. Finally, the proposed system was shown to be capable of carrying an unknown payload, even if the attachment positions were not shared, that is, even if asymptotic stability was not strictly guaranteed.

In this paper, we introduce a Modular Single Actuator Monocopter (M-SAM), which is capable of flying in both singular configuration and cooperative configuration. From singular mode, M-SAMs can be manually assembled into cooperative mode, using magnetic connectors built into the body of each M-SAM unit. The design of the connectors allow for passive separation of the units without the need for a dedicated separating actuator, by harnessing the variable centrifugal force from controlled adjustment of the rotating speed of the craft. To achieve control in both configurations, we firstly studied and analyzed their full dynamic models by introducing equilibrium state and relaxed hovering condition. Next, we derived a reduced model to approximate the dynamical behavior of both singular and cooperative configuration in flight to design a generalized cyclic-based cascaded flight controller. Finally, we validated the proposed controller and separation mechanism by conducting several flight experiments for two M-SAMs in singular mode, cooperative mode as well as mid-air separating under motion capture system.

Non-linear trajectory optimisation methods require good initial guesses to converge to a locally optimal solution. A feasible guess can often be obtained by allocating a large amount of time for the trajectory to be complete. However for unstable dynamical systems such as humanoid robots, this quasi-static assumption does not always hold. We propose a conservative formulation of the trajectory problem that simultaneously computes a feasible path and its time allocation. The problem is solved as a convex optimisation problem guaranteed to converge to a feasible local optimum. The approach is evaluated with the computation of feasible trajectories that traverse sequentially a sequence of polytopes. We demonstrate that on instances of the problem where quasi static solutions are not admissible, our approach is able to find a feasible solution with a success rate above 80% in all the scenarios considered, in less than 10ms for problems involving traversing less than 5 polytopes and less than 1s for problems involving 20 polytopes, thus demonstrating its ability to reliably provide initial guesses to advanced non linear solvers.

This paper presents a convex model predictive control framework for versatile dynamic legged motions with negligible leg dynamics. The framework utilizes the single rigid body model linearly approximated around the operating point. With ground reaction forces as direct control inputs to the system, no reference control trajectory needs to be specified in advance. By using the rotation matrix for the evolution of rotational dynamics, issues arising from other representations can be avoided. Moreover, the rotation matrix is parametrized using the history of angular velocity without introducing additional variables. The effect is that we can still take the orientation into consideration efficaciously without directly working on it. The framework tackles the robot reference tracking problem via trajectory optimization, which is formulated into a standard quadratic program and can be solved efficiently in real time with guaranteed optimality. It was verified on various legged robots with different numbers of legs for performing different types of dynamic motions in the simulation environment. We thus envision a promising future of the proposed convex model predictive control framework in legged robots and potentially in other applications as well.

Autonomous vehicle software is typically structured as a modular pipeline of individual components (e.g., perception, prediction, and planning) to help separate concerns into interpretable sub-tasks. Even when end-to-end training is possible, each module has its own set of objectives used for safety assurance, sample efficiency, regularization, or interpretability. However, intermediate objectives do not always align with overall system performance. For example, optimizing the likelihood of a trajectory prediction module might focus more on easy-to-predict agents than safety-critical or rare behaviors (e.g., jaywalking). In this paper, we present control-aware prediction objectives (CAPOs), to evaluate the down-stream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories. Experimentally, we show our objectives improve overall system performance in suburban driving scenarios using the CARLA simulator.

For immersive interaction in a virtual reality (VR) environment, an omnidirectional treadmill (ODT) can support performance of various locomotive motions (curved walk, side walk, moving with shooting stance) in any direction. When a user performs lateral locomotive motions on an ODT, a control scheme to achieve immersive and safe interaction with the ODT should satisfy robustness in terms of position error of a user to keep a reference position of the ODT by accurately estimating intentional walking speed (IWS) of the user, and it should guarantee postural stability of the user during the control actions. Existing locomotion interface (LI) control focuses on the reference position tracking performance regarding the position of the user's center of mass (COM) in order to respond to forward locomotion that can move at high speed. However, in sideways walking, the movement of the lower extremities is different from that of forward walking, and when the conventional LI control was directly applied to sideways walking, it was observed that excessive acceleration commands caused postural instability. For appropriate interface of sideways walking, we propose an estimation scheme based on an accurate walking model including the movement of the ankle joint. The proposed observer estimates the acting torque generated by the force of both lower extremities through the position information of COM and ankle joint to more accurately predict the user's intentional walking speed (IWS). In the sideways walking experiment conducted using a 1-dimensional user-driven treadmill (UDT), the proposed method allowed more natural interface of the lateral-side locomotion with better postural stability compared to the conventional estimation method that uses only the COM position information.

We propose a continuous-time spline-based formulation for visual-inertial odometry (VIO). Specifically, we model the poses as a cubic spline, whose temporal derivatives are used to synthesize linear acceleration and angular velocity, which are compared to the measurements from the inertial measurement unit (IMU) for optimal state estimation. The spline boundary conditions create constraints between the camera and the IMU, with which we formulate VIO as a constrained nonlinear optimization problem. Continuous-time pose representation makes it possible to address many VIO challenges, e.g., rolling shutter distortion and sensors that may lack synchronization. We conduct experiments on two publicly available datasets that demonstrate the state-of-the-art accuracy and real-time computational efficiency of our method.

Conventional methods to autonomous grasping rely on a pre-computed database with known objects to synthesize grasps, which is not possible for novel objects. On the other hand, recently proposed deep learning-based approaches have demonstrated the ability to generalize grasp for unknown objects. However, grasp generation still remains a challenging problem, especially in cluttered environments under partial occlusion. In this work, we propose an end-to-end deep learning approach for generating 6-DOF collision-free grasps given a 3D scene point cloud. To build robustness to occlusion, the proposed model generates candidates by casting votes and accumulating evidence for feasible grasp configurations. We exploit contextual information by encoding the dependency of objects in the scene into features to boost the performance of grasp generation. The contextual information enables our model to increase the likelihood that the generated grasps are collision-free. Our experimental results confirm that the proposed system performs favorably in terms of predicting object grasps in cluttered environments in comparison to the current state of the art methods.