ICRA 2022 论文列表

Humans engaged in collaborative activities are naturally able to convey their intentions to teammates through multi-modal communication, which is made up of explicit and implicit cues. Similarly, a more natural form of human-robot collaboration may be achieved by enabling robots to convey their intentions to human teammates via multiple communication channels. In this paper, we postulate that a better communication may take place should collaborative robots be able to anticipate their movements to human teammates in an intuitive way. In order to support such a claim, we propose a robot system's architecture through which robots can communicate planned motions to human teammates leveraging a Mixed Reality interface powered by modern head-mounted displays. Specifically, the robot's hologram, which is superimposed to the real robot in the human teammate's point of view, shows the robot's future movements, allowing the human to understand them in advance, and possibly react to them in an appropriate way. We conduct a preliminary user study to evaluate the effectiveness of the proposed anticipatory visualization during a complex collaborative task. The experimental results suggest that an improved and more natural collaboration can be achieved by employing this anticipatory communication mode.

The hierarchical quadratic programming (HQP) is commonly applied to consider strict hierarchies of multi-tasks and robot's physical inequality constraints during whole-body compliance. However, for the one-step HQP, the solution can oscillate when it is close to the boundary of constraints. It is because the abrupt hit of the bounds gives rise to unrealizable jerks and even infeasible solutions. This paper proposes the mixed control, which blends the single-axis model predictive control (MPC) and proportional derivative (PD) control for the whole-body compliance to overcome these deficiencies. The MPC predicts the distances between the bounds and the control target of the critical tasks, and it provides smooth and feasible solutions by prediction and optimization in advance. However, applying MPC will inevitably increase the computation time. Therefore, to achieve a 500 Hz servo rate, the PD controllers still regulate other tasks to save computation resources. Also, we use a more efficient null space projection (NSP) whole-body controller instead of the HQP and distribute the single-axis MPCs into four CPU cores for parallel computation. Finally, we validate the desired capabilities of the proposed strategy via simulations and the experiment on the humanoid robot Walker X.

Seemingly in defiance of basic physics, cats consistently land on their feet after falling. In this paper, we design a controller that lands the Mini Cheetah quadruped robot on its feet as well. Specifically, we explore how trajectory optimization and machine learning can work together to enable highly dynamic bioinspired behaviors. We find that a reflex approach, in which a neural network learns entire state trajectories, outperforms a policy approach, in which a neural network learns a mapping from states to control inputs. We validate our proposed controller in both simulation and hardware experiments, and are able to land the robot on its feet from falls with initial pitch angles between −90 and 90 degrees.

A substantially advance skill-set is a prerequisite in the domain of retinal surgery, given that the surgical instruments, constrained by small incisions made on the sclera, should be manipulated in a confined intraocular space. Therefore, robotic technologies with a snake-like architecture may be critical in retinal surgery to overcome this problem. These robots are expected to approach a target on the retina from a suitable direction when accessing its anterior portion for procedures such as vein cannulation or membrane peeling. Typical end-effectors or tools for retinal surgery include needles, light pipes, pipettes, and grippers. However, there are no retinal surgery robots or devices equipped with enough bending and grasping functionalities. We developed an Improved Integrated Robotic Intraocular Snake (I2RIS) in previous works. I2RIS has a user interface (a tactile switch or joystick unit) to provide maneuverability to the snake-like distal end. This study presents a new microgripper with its drive mechanism and an interface for retinal surgery; this microgripper is implemented into I2RIS. The proposed microgripper has a simple mechanism; it comprised only three parts, including a nitinol drive wire that functions as a flexible hinge. The microgripper diameter is 0.9 mm, and the length is 2.6 mm. A real-size prototype model was used to demonstrate the effectiveness of the proposed microgripper. In addition, a pick-and-place task using an eye model was performed by I2RIS with the proposed microgripper. It is trusted that the utility of this microgripper can extend beyond retinal surgery into other microsurgical applications.

Although experts carefully specify the high-level decision-making models in autonomous systems, it is infeasible to guarantee safety across every scenario during operation. We therefore propose a safety metareasoning system that optimizes the severity of the system's safety concerns and the interference to the system's task: the system executes in parallel a task process that completes a specified task and safety processes that each address a specified safety concern with a conflict resolver for arbitration. This paper offers a formal definition of a safety metareasoning system, a recommendation algorithm for a safety process, an arbitration algorithm for a conflict resolver, an application of our approach to planetary rover exploration, and a demonstration that our approach is effective in simulation.

Spatio-temporal graphs (ST-graphs) have been used to model time series tasks such as traffic forecasting, human motion modeling, and action recognition. The high-level structure and corresponding features from ST-graphs have led to improved performance over traditional architectures. However, current methods tend to be limited by simple features, despite the rich information provided by the full graph structure, which leads to inefficiencies and suboptimal performance in downstream tasks. We propose the use of features derived from meta-paths, walks across different types of edges, in ST-graphs to improve the performance of Structural Recurrent Neural Network. In this paper, we present the Meta-path Enhanced Structural Recurrent Neural Network (MESRNN), a generic framework that can be applied to any spatio-temporal task in a simple and scalable manner. We employ MESRNN for pedestrian trajectory prediction, utilizing these meta-path based features to capture the relationships between the trajectories of pedestrians at different points in time and space. We compare our MESRNN against state-of-the-art ST-graph methods on standard datasets to show the performance boost provided by meta-path information. The proposed model consistently outperforms the baselines in trajectory prediction over long time horizons by over 32%, and produces more socially compliant trajectories in dense crowds. For more information please refer to the project website at https://sites.google.com/illinois.edu/mesrnn/home.

We propose a novel framework to estimate the confidence of a disparity map taking into account, for the first time, the uncertainty affecting the confidence estimation process itself. Conversely to other tasks such as disparity estimation, the uncertainty of confidence directly hints that the confidence should be increased if initially low, but with high uncertainty, decreased otherwise. By modelling such a cue in the form of a second-level confidence, or meta-confidence, our solution allows for finding incorrect predictions inferred by confidence estimator and for learning a correction for them. Our strategy is suited for any state-of-the-art method known in literature, either implemented using random forest classifiers or deep neural networks. Especially, for deep neural networks-based models, we present a multi-headed confidence estimator followed by an uncertainty network, so as to predict mean confidence and meta-confidence within a single network without the cost of lower accuracy, a known limitation in literature for uncertainty estimation. Experimental results on a variety of stereo algorithms and confidence estimation models prove that the modeled meta-confidence is meaningful of the reliability of the estimated confidence and allows for refining it.

In Human Robot Interaction (HRI) scenarios, robot systems would benefit from an understanding of the user's state, actions and their effects on the environments to enable better interactions. While there are specialised vision algorithms for different perceptual channels, such as objects, scenes, human pose, and human actions, it is worth considering how their interaction can help improve each other's output. In computer vision, individual prediction modules for these perceptual channels frequently produce noisy outputs due to the limited datasets used for training and the compartmentalisation of the perceptual channels, often resulting in noisy or unstable prediction outcomes. To stabilise vision prediction results in HRI, this paper presents a novel message passing framework that uses the memory of individual modules to correct each other's outputs. The proposed framework is designed utilising common-sense rules of physics (such as the law of gravity) to reduce noise while introducing a pipeline that helps to effectively improve the output of each other's modules. The proposed framework aims to analyse primitive human activities such as grasping an object in a video captured from the perspective of a robot. Experimental results show that the proposed framework significantly reduces the output noise of individual modules compared to the case of running independently. This pipeline can be used to measure human reactions when interacting with a robot in various HRI scenarios.

Deep imitation learning is a promising approach that does not require hard-coded control rules in autonomous robot manipulation. The current applications of deep imitation learning to robot manipulation have been limited to reactive control based on the states at the current time step. However, future robots will also be required to solve tasks utilizing their memory obtained by experience in complicated environments (e.g., when the robot is asked to find a previously used object on a shelf). In such a situation, simple deep imitation learning may fail because of distractions caused by complicated environments. We propose that gaze prediction from sequential visual input enables the robot to perform a manipulation task that requires memory. The proposed algorithm uses a Transformer-based self-attention architecture for the gaze estimation based on sequential data to implement memory. The proposed method was evaluated with a real robot multi-object manipulation task that requires memory of the previous states.

Computing consumes a significant portion of energy in many robotics applications, especially the ones involving energy-constrained robots. In addition, memory access accounts for a significant portion of the computing energy. For mapping a 3D environment, prior approaches reduce the map size while incurring a large memory overhead used for storing sensor measurements and temporary variables during computation. In this work, we present a memory-efficient algorithm, named Single-Pass Gaussian Fitting (SPGF), that accurately constructs a compact Gaussian Mixture Model (GMM) which approximates measurements from a depthmap generated from a depth camera. By incrementally constructing the GMM one pixel at a time in a single pass through the depthmap, SPGF achieves higher throughput and orders-of-magnitude lower memory overhead than prior multipass approaches. By processing the depthmap row-by-row, SPGF exploits intrinsic properties of the camera to efficiently and accurately infer surface geometries, which leads to higher precision than prior approaches while maintaining the same compactness of the GMM. Using a low-power ARM Cortex-A57 CPU on the NVIDIA Jetson TX2 platform, SPGF operates at 32fps, requires 43KB of memory overhead, and consumes only 0.11J per frame (depthmap). Thus, SPGF enables real-time mapping of large 3D environments on energy-constrained robots.

Shelves are common in homes, warehouses, and commercial settings due to their storage efficiency. However, this efficiency comes at the cost of reduced visibility and accessibility. When looking from a side (lateral) view of a shelf, most objects will be fully occluded, resulting in a constrained lateral-access mechanical search problem. To address this problem, we introduce: (1) a novel bluction tool, which combines a thin pushing blade and a suction cup gripper, (2) a simulation pipeline and perception model that combine ray-casting with 2D Minkowski sums to efficiently generate target occupancy distributions, and (3) a novel search policy, which optimally reduces target object distribution support area using the bluction tool. Experimental data from 2000 simulated shelf trials and 18 trials with a physical Fetch robot suggest that a bluction tool can improve the average success rate by 26% in simulation and 67% in physical experiments over the highest-performing push-only policy.

In physical human-robot interaction (pHRI), safety is a key requirement. As collisions between humans and robots can generally not be avoided, it must be ensured that the human is not harmed. The robot reflected mass, the contact geometry, and the relative velocity between human and robot are the parameters that have the most significant influence on human injury severity during a collision. The reflected mass depends on the robot configuration and can be optimized especially in kinematically redundant robots. In this paper, we propose the Mean Reflected Mass (MRM) metric. The MRM is independent of the direction of contact/motion and enables assessing and optimizing the robot posture w.r.t. safety. In contrast to existing metrics, it is physically interpretable, meaning that it can be related to biomechanical injury data for realistic and model-independent safety analysis. For the Franka Emika Panda, we demonstrate in simulation that an optimization of the robot's MRM reduces the mean collision force. Finally, the relevance of the MRM for real pHRI applications is confirmed through a collision experiment.

When a robot observes another agent unexpectedly modifying their behavior, inferring the most likely cause is a valuable tool for maintaining safety and reacting appropriately. In this work, we present a novel method for inferring constraints that works on continuous, possibly sub-optimal demonstrations. We first learn a representation of the continuous-state maximum entropy trajectory distribution using deep reinforcement learning. We then use Monte Carlo sampling from this distribution to generate expected constraint violation probabilities and perform constraint inference. When the demonstrator's dynamics and objective function are known in advance, this process can be performed offline, allowing for real-time constraint inference at the moment demonstrations are observed. We evaluate our approach on two continuous dynamical systems: a 2-dimensional inverted pendulum model, and a 4-dimensional unicycle model that was successfully used for fast constraint inference on a 1/10 scale car remote-controlled by a human.

In this paper, we present a novel maximum entropy formulation of the Differential Dynamic Programming algorithm and derive two variants using unimodal and multimodal value functions parameterizations. By combining the maximum entropy Bellman equations with a particular approximation of the cost function, we are able to obtain a new formulation of Differential Dynamic Programming which is able to escape from local minima via exploration with a multimodal policy. To demonstrate the efficacy of the proposed algorithm, we provide experimental results using four systems on tasks that are represented by cost functions with multiple local minima and compare them against vanilla Differential Dynamic Programming. Furthermore, we discuss connections with previous work on the linearly solvable stochastic control framework and its extensions in relation to compositionality. Link to Video.

This paper proposes a teleoperation framework to exploit the maximum manipulation capability during teleoperation. Here, exploiting maximum manipulation capacity means that the robot moves with its maximum control input while not violating the given constraints, and it is a nonlinear optimization problem with nonlinear constraints which is hard to be solved. The proposed framework relaxes the optimization problem into a simple QP problem and unifies the various constraints in the joint configuration space and Cartesian task space by utilizing control barrier function and control Lyapunov function techniques. The joint angle, velocity, acceleration limits are imposed on a teleoperated robot so that the robot does not generate any emergency stop during the teleoperation. Especially, the robot shows stable motion even near the kinematic singularity, so the operator can explore almost every reachable and admissible state of the robot via teleoperation.

Recent innovations in miniature sensors are driving a shift from robotic to bio-hybrid systems for exploration of unstructured environments. The ubiquity of honey bees in modern agriculture and ecology along with their superior agility, olfactory sense, and collective foraging skills make them a promising complement to traditional robots. This paper explores the potential of such systems based on a custom honey bee foraging simulator and models of state-of-the art miniature flight recorders which can measure solar heading at regular time intervals, as well as exploratory data collected from the sensor mounted on an autonomous quadrotor. The size and functionality of the sensor is heavily influenced by its memory footprint, therefore, we investigate the impact of sensor sampling time on map accuracy. Our results indicate that a sampling rate down to 5Hz can be used to sense obstacle locations in a 5-acre field with an accuracy corresponding to 70% of the obstacle radius, and within 4% of its true area. This technique shows promise for using instrumented honey bees to map and monitor unstructured environments which are difficult or costly for robots to robustly navigate, monitor, and map.

We revisit the problem of efficiently leveraging prior map information within a visual-inertial estimation framework. The use of traditional landmark-based maps with 2D-to-3D measurements along with the recently introduced keyframe-based maps with 2D-to-2D measurements are inves-tigated. The full joint estimation of the prior map is compared within a visual-inertial simulator to the Schmidt-Kalman filter (SKF) and measurement inflation methods in terms of their computational complexity, consistency, accuracy, and memory usage. This study shows that the SKF can enable efficient and consistent estimation for small workspace scenarios and the use of 2D-to-3D landmark maps have the highest levels of accuracy. Keyframe-based 2D-to-2D maps can reduce the required state size while still enabling accuracy gains. Finally, we show that measurement inflation methods, after tuning, can be accurate and efficient for large-scale environments if the guarantee of consistency is relaxed.

We present an approach to robotic manipulation of unknown objects through regulation of the object's contact configuration: the location, geometry, and mode of all contacts between the object, robot, and environment. A contact configu-ration constrains the forces and motions that can be applied to the object; however, synthesizing these constraints generally requires knowledge of the object's pose and geometry. We develop an object-agnostic approach for estimation and control that circumvents this need. Our framework directly estimates a set of wrench and motion constraints which it uses to regulate the contact configuration. We use this to reactively manipulate unknown planar objects in the gravity plane. A video describing our work can be found on our project page: http://mcube.mit.edu/research/contactConfig.html.

Soft continuum robots have been widely used as guide wires or catheters for minimally invasive surgery (MIS), and the miniaturization and dexterity are very important characteristics of soft continuum robots. As a representative actuation method for soft continuum robots, the steering using an external magnetic field has been actively studied. The magnetic actuation method is appropriate for the miniaturization of soft continuum robots but shows limitations in implementing high dexterity. To achieve miniaturization and high dexterity, this paper proposes a magnetically steerable asymmetric magnetized soft continuum robot (AMSCR). The proposed AMSCR includes an active steering part (ASP) of a thin PDMS cylinder in which NdFeB powder is uniformly dispersed and magnetized asymmetrically to the longitudinal direction. Therefore, compared to the conventional symmetric magnetized soft continuum robot (SMSCR), the proposed AMSCR has a larger steering range and high dexterity. Through various experiments, simulations, and phantom experiments using the proposed AMSCR, we analyzed its characteristics, verified its enhanced steering range, and demonstrated the applicability to the surgeries requiring high dexterity in the eye or cerebrovascular.