ICRA 2021 论文列表

Deep reinforcement learning (DRL) provides a promising way for learning navigation in complex autonomous driving scenarios. However, identifying the subtle cues that can indicate drastically different outcomes remains an open problem with designing autonomous systems that operate in human environments. In this work, we show that explicitly inferring the latent state and encoding spatial-temporal relationships in a reinforcement learning framework can help address this difficulty. We encode prior knowledge on the latent states of other drivers through a framework that combines the reinforcement learner with a supervised learner. In addition, we model the influence passing between different vehicles through graph neural networks (GNNs). The proposed framework significantly improves performance in the context of navigating T-intersections compared with state-of-the-art baseline approaches.

This paper describes the control of a novel Magnetic Actuated Flexible-joint Robotic Surgical (MAFRS) camera system with four degrees of freedom (4-DOF) for single incision laparoscopic surgery. Based on the idea of motion decoupling, we designed a novel MAFRS system which is consists of an external driving device and a motor-free insertable wireless robotic device with a hollow flexible joint. Due to the problems of abdominal wall obstruction and variability in abdominal wall thickness during the actual application of the MAFRS system, as well as the existence of multiple permanent magnets and magnetically conductive media, high- precision position and attitude control of the insertable device without onboard motors has always been a challenge. We use the external driving device to generate a magnetic field to control the position and attitude of the internal robotic device. Aiming at the automatic precise tilt motion control of the novel MAFRS camera system, we have developed a closed-loop control scheme using the Deep Deterministic Policy Gradient (DDPG) algorithm. By referring to the damping characteristics of human muscles, a virtual-muscle method is proposed to eliminate the chattering problem of the MAFRS camera at specific angles. The experimental investigations indicate that the internal robotic device can be effectively controlled under different abdominal wall thicknesses. The tilt motion control accuracy is within 0.5°, and it has good adaptability and antiinterference performance.

This paper presents a model-free reinforcement learning (RL) algorithm to synthesize a control policy that maximizes the satisfaction probability of complex tasks, which are expressed by linear temporal logic (LTL) specifications. Due to the consideration of environment and motion uncertainties, we model the robot motion as a probabilistic labeled Markov decision process (PL-MDP) with unknown transition probabilities and probabilistic labeling functions. The LTL task specification is converted to a limit deterministic generalized Büchi automaton (LDGBA) with several accepting sets to maintain dense rewards during learning. The novelty of applying LDGBA is to construct an embedded LDGBA (E-LDGBA) by designing a synchronous tracking-frontier function, which enables the record of non-visited accepting sets of LDGBA at each round of the repeated visiting pattern, to overcome the difficulties of directly applying conventional LDGBA. With appropriate dependent reward and discount functions, rigorous analysis shows that any method, which optimizes the expected discount return of the RL-based approach, is guaranteed to find the optimal policy to maximize the satisfaction probability of the LTL specifications. A model-free RL-based motion planning strategy is developed to generate the optimal policy in this paper. The effectiveness of the RL-based control synthesis is demonstrated via simulation and experimental results.

Robot locomotion is a major challenge in robotics. Model-based approaches are vulnerable to model errors, and incur high computation overhead resulted from long control horizon. Model-free approaches are trained with a large number of training samples, which are expensive to obtain. In this paper, we develop a hybrid control and learning framework, called Reinforced iLQR (RiLQR), which combines the advantages of model-based iLQR control with model-free RL policy learning to simultaneously achieve high sample efficiency, low computation overhead, and high robustness against model errors in robot locomotion. Through extensive evaluation on the Mujoco platform, we demonstrate that RiLQR outperforms the state-of-the-art model-based and model-free baselines by big margins in a set of tasks with different complexities.

A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies. This trend often presents itself in the form of control signal oscillation and can result in poor control, high power consumption, and undue system wear. We introduce Conditioning for Action Policy Smoothness (CAPS), an effective yet intuitive regularization on action policies, which offers consistent improvement in the smoothness of the learned state-to-action mappings of neural network controllers, reflected in the elimination of high-frequency components in the control signal. Tested on a real system, improvements in controller smoothness on a quadrotor drone resulted in an almost 80% reduction in power consumption while consistently training flight-worthy controllers. Project website: http://ai.bu.edu/caps

Attempts to achieve robotic Within-Hand-Manipulation (WIHM) generally utilize either high-DOF robotic hands with elaborate sensing apparatus or multi-arm robotic systems. In prior work we presented a simple robot hand with variable friction robot fingers, which allow a low-complexity approach to within-hand object translation and rotation, though this manipulation was limited to planar actions. In this work we extend the capabilities of this system to 3D manipulation with a novel region-based WIHM planning algorithm and utilizing extrinsic contacts. The ability to modulate finger friction enhances extrinsic dexterity for three-dimensional WIHM, and allows us to operate in the quasi-static level. The region-based planner automatically generates 3D manipulation sequences with a modified A* formulation that navigates the contact regions between the fingers and the object surface to reach desired regions. Central to this method is a set of object-motion primitives (i.e. within-hand sliding, rotation and pivoting), which can easily be achieved via changing contact friction. A wide range of goal regions can be achieved via this approach, which is demonstrated via real robot experiments following a standardized in-hand manipulation benchmarking protocol.

This paper aims to tackle the problem of referring image segmentation, which is targeted at reasoning the region of interest referred by a query natural language sentence. One key issue to address the referring image segmentation is how to establish the cross-modal representation for encoding the two modalities, namely, the query sentence and the input image. Most existing methods are designed to concatenate the features from each modality or to gradually encode the cross-modal representation concerning each word’s effect. In contrast, our approach leverages the correlation between the two modalities for constructing the cross-modal representation. To make the resulting cross-modal representation more discriminative for the segmentation task, we propose a novel mechanism of language-driven attention to encode the cross-modal representation for reflecting the attention between every single visual element and the entire query sentence. The proposed mechanism, named as Language-Driven Attention (LDA), first decouples the cross-modal correlation to channel-attention and spatial-attention and then integrates the two attentions for obtaining the cross-modal representation. The channel attention and the spatial attention respectively reveal how sensitive each channel or each pixel of a particular feature map is with respect to the query sentence. With a proper fusion of the two kinds of feature attention, the proposed LDA model can effectively guide the generation of the final cross-modal representation. The resulting representation is further strengthened for capturing the multi-receptive-field and multi-level-semantic for the intended segmentation. We assess our referring image segmentation model on four public benchmark datasets, and the experimental results show that our model achieves state-of-the-art performance

Powered exoskeletons for people with paraplegia are subjected to repetitive and large impacts due to the repeated ground contacts. The repetitive impact forces not only deteriorate the wear comfort but also cause a serious damage to the muscles and bones of the human wearing the powered exoskeleton. To address this issue, a novel shock absorption mechanism for powered exoskeletons that can reduce the peak of ground reaction force up to 28% is designed in this paper. The designed absorption mechanism is integrated into the WalkON Suit, a powered exoskeleton for people with paraplegia and verified by experimental results with a human subject in this paper also.

Deep reinforcement learning (DRL) has been demonstrated to provide promising results in several challenging decision making and control tasks. However, the required inference costs of deep neural networks (DNNs) could prevent DRL from being applied to mobile robots which cannot afford high energy-consuming computations. To enable DRL methods to be affordable in such energy-limited platforms, we propose an asymmetric architecture that reduces the overall inference costs via switching between a computationally expensive policy and an economic one. The experimental results evaluated on a number of representative benchmark suites for robotic control tasks demonstrate that our method is able to reduce the inference costs while retaining the agent’s overall performance.

In this paper, we propose the reduced model for the full dynamics of a bicycle and analyze its nonlinear behavior under a proportional control law for steering. Based on the Gibbs-Appell equations for the Whipple bicycle, we obtain a second-order nonlinear ordinary differential equation (ODE) that governs the bicycle’s controlled motion. Two types of equilibrium points for the governing equation are found, which correspond to the bicycle’s uniform straight forward and circular motions, respectively. By applying the Hurwitz criterion to the linearized equation, we find that the steer coefficient must be negative, consistent with the human’s intuition of turning toward a fall. Under this condition, a critical angular velocity of the rear wheel exists, above which the uniform straight forward motion is stable, and slightly below which a pair of symmetrical stable uniform circular motions will occur. These theoretical findings are verified by both numerical simulations and experiments performed on a powered autonomous bicycle.

Fully insertable laparoscopic cameras represent a promising future of minimally invasive surgery. The most characteristic technology adopted on these devices is transabdominal anchoring and actuation based on magnetic coupling. However, few have paid adequate attention to the safety concerns. As the camera is anchored against the interior abdominal wall without any force feedback, the patient is being exposed to a high risk of getting injured by inappropriate stress on the tissue. We have recovered the camera-tissue interaction force via a non-invasive approach in our previous work. Aiming to access the stress distribution, this paper presents a viscoelastic camera-tissue interaction model, which establishes explicit relations between the contact force and the stress distribution on the tissue. For the first time, a geometric constraint between the contact angle and the tissue indentation is introduced, which helps make the multivariable model solvable. Ex-vivo experiments on porcine abdomen tissue facilitated by non-invasive force measurement validate effectiveness of the model. This work lays foundation for improving control and surgical safety of using a magnetic actuated insertable laparoscopic surgical camera.

In this paper, we rethink the problem of scene reconstruction from an embodied agent’s perspective: While the classic view focuses on the reconstruction accuracy, our new perspective emphasizes the underlying functions and constraints such that the reconstructed scenes provide actionable information for simulating interactions with agents. Here, we address this challenging problem by reconstructing an interactive scene using RGB-D data stream, which captures (i) the semantics and geometry of objects and layouts by a 3D volumetric panoptic mapping module, and (ii) object affordance and contextual relations by reasoning over physical common sense among objects, organized by a graph-based scene representation. Crucially, this reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models for finer-grained robot interactions. In the experiments, we demonstrate that (i) our panoptic mapping module outperforms previous state-of-the-art methods, (ii) a high-performant physical reasoning procedure that matches, aligns, and replaces objects’ meshes with best-fitted CAD models, and (iii) reconstructed scenes are physically plausible and naturally afford actionable interactions; without any manual labeling, they are seamlessly imported to ROS-based simulators and virtual environments for complex robot task executions.1

Manipulations of a constrained object often use a non-rigid grasp that allows the object to rotate relative to the end effector. This orientation slip strategy is often present in natural human demonstrations, yet it is generally overlooked in methods to identify constraints from such demonstrations. In this paper, we present a method to model and recognize prehensile orientation slip in human demonstrations of constrained interactions. Using only observations of an end effector, we can detect the type of constraint, parameters of the constraint, and orientation slip properties. Our method uses a novel hierarchical model selection method that is informed by multiple origins of physics-based evidence. A study with eight participants shows that orientation slip occurs in natural demonstrations and confirms that it can be detected by our method.

Active wireless capsule endoscopy (WCE) under magnetic actuation is a promising technology to reduce the inspection time and relieve the burden of physicians. In this paper, we propose a reciprocally rotating magnetic actuation method for trajectory following of a capsule and develop its dynamic model. For the trajectory following task, we investigate the closed-loop tracking control strategies based on different controllers to actuate the capsule in the complex environments. The effectiveness of our method is validated in extensive experiments in a simulation environment as well as in an ex-vivo pig colon. The results demonstrate that the proposed method can accurately and efficiently actuate the capsule to follow the desired trajectory in the complex environments, achieving tracking errors on the order of millimeter. Moreover, the experiments on the ex-vivo pig colon show that the proposed reciprocally rotating magnetic actuation method has the potential to reduce the clinical risks and improve the safety and clinical acceptability of this technology.

To dynamically traverse challenging terrain, legged robots need to continually perceive and reason about upcoming features, adjust the locations and timings of future footfalls and leverage momentum strategically. We present a pipeline that enables flexibly-parametrized trajectories for perceptive and dynamic quadruped locomotion to be optimized in an online, receding-horizon manner. The initial guess passed to the optimizer affects the computation needed to achieve convergence and the quality of the solution. We consider two methods for generating good guesses. The first is a heuristic initializer which provides a simple guess and requires significant optimization but is nonetheless suitable for adaptation to upcoming terrain. We demonstrate experiments using the ANYmal C quadruped, with fully onboard sensing and computation, to cross obstacles at moderate speeds using this technique. Our second approach uses latent-mode trajectory regression (LMTR) to imitate expert data—while avoiding invalid interpolations between distinct behaviors—such that minimal optimization is needed. This enables high-speed motions that make more expansive use of the robot’s capabilities. We demonstrate it on flat ground with the real robot and provide numerical trials that progress toward deployment on terrain. These results illustrate a paradigm for advancing beyond short-horizon dynamic reactions, toward the type of intuitive and adaptive locomotion planning exhibited by animals and humans.

Justifying operational decisions for robots is a challenging task as the operator or the robot itself has to understand the underlying physical interaction between the robot and the environment to predict the potential outcome. It is desirable to understand how the decision influences the operational performance in the way of causal relationship for the purpose of explainable decision-making. Here we propose a novel causal inference framework for the discovery and inference on the reasoning of the operational decisions for robots. It unifies both domain knowledge integration and model-free causal inference, allowing a data-driven causal knowledge learning on time series data. The framework is evaluated in the experiments of an underwater robot with complex environmental interactions. The results show that the framework can learn the causal structure and inference model to accurately explain and predict the operation performance with integrated physics.

Autonomy in robotic surgery is very challenging in unstructured environments, especially when interacting with deformable soft tissues. The main difficulty is to generate model-based control methods that account for deformation dynamics during tissue manipulation. Previous works in vision-based perception can capture the geometric changes within the scene, however, model-based controllers integrated with dynamic properties, a more accurate and safe approach, has not been studied before. Considering the mechanic coupling between the robot and the environment, it is crucial to develop a registered, simulated dynamical model. In this work, we propose an online, continuous, real-to-sim registration method to bridge 3D visual perception with position-based dynamics (PBD) modeling of tissues. The PBD method is employed to simulate soft tissue dynamics as well as rigid tool interactions for model-based control. Meanwhile, a vision-based strategy is used to generate 3D reconstructed point cloud surfaces based on real-world manipulation, so as to register and update the simulation. To verify this real-to-sim approach, tissue experiments have been conducted on the da Vinci Research Kit. Our real-to-sim approach successfully reduces registration error online, which is especially important for safety during autonomous control. Moreover, it achieves higher accuracy in occluded areas than fusion-based reconstruction.

This article proposes a new method that enables Unmanned Aerial Vehicles (UAVs) to actively find targets and shoot photographs of them in an unknown environment, while successfully avoiding surrounding obstacles and planning optimize routes. Owing to the limited computing ability on the UAVs, we obtained the point cloud data of surrounding objects, and selected the best segmentation method of the point cloud to perform real-time semantic segmentation on the collected point cloud data. The point cloud data with semantic attributes were merged into voxels. We reconstruct the real-time distance and angle between the surface of obstacles and the surrounding obstacles through Euclidean Signed Distance Fields (ESDFs), and adjust the gimbal angle and focal length of UAVs and use the two-dimensional image recognition to shoot the photographs of the target precisely. Considering the increasing scale of UAVs power inspections, we can improve the efficiency of fine inspections of power transmission lines by using the method we proposed.

In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and the zooming ratio of the laparoscope, following the surgeon’s instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjustment of zooming ratio may interrupt the workflow of the surgical operation. To this end, we propose a multi-scale Generative Adversarial Network (GAN)-based video super-resolution method to construct a framework for automatic zooming ratio adjustment. It can provide automatic real-time zooming for high-quality visualization of the Region of Interest (ROI) during the surgical operation. In the pipeline of the framework, the Kernel Correlation Filter (KCF) tracker is used for tracking the tips of the surgical tools, while the Semi-Global Block Matching (SGBM)-based depth estimation and Recurrent Neural Network (RNN)-based context-awareness are employed to determine the upscaling ratio for zooming. The framework is validated with the JIGSAW dataset and Hamlyn Centre Laparoscopic/Endoscopic Video Datasets, with results demonstrating its practicability.