IROS 2020 论文列表

Inferring semantic information towards an understanding of the surrounding environment is crucial for autonomous vehicles to drive safely. Deep learning-based segmentation methods can infer semantic information directly from laser range data, even in the absence of other sensor modalities such as cameras. In this paper, we address improving the generalization capabilities of such deep learning models to range data that was captured using a different sensor and in situations where no labeled data is available for the new sensor setup. Our approach assists the domain transfer of a LiDAR-only semantic segmentation model to a different sensor and environment exploiting existing geometric mapping systems. To this end, we fuse sequential scans in the source dataset into a dense mesh and render semi-synthetic scans that match those of the target sensor setup. Unlike simulation, this approach provides a real-to-real transfer of geometric information and delivers additionally more accurate remission information. We implemented and thoroughly tested our approach by transferring semantic scans between two different real-world datasets with different sensor setups. Our experiments show that we can improve the segmentation performance substantially with zero manual re-labeling. This approach solves the number one feature request since we released our semantic segmentation library LiDAR-bonnetal [18].

Being able to estimate the traversability of the area surrounding a mobile robot is a fundamental task in the design of a navigation algorithm. However, the task is often complex, since it requires evaluating distances from obstacles, type and slope of terrain, and dealing with non-obvious discontinuities in detected distances due to perspective. In this paper, we present an approach based on deep learning to estimate and anticipate the traversing score of different routes in the field of view of an on-board RGB camera. The backbone of the proposed model is based on a state-of-the-art deep segmentation model, which is fine-tuned on the task of predicting route traversability. We then enhance the model's capabilities by a) addressing domain shifts through gradient-reversal unsupervised adaptation, and b) accounting for the specific safety requirements of a mobile robot, by encouraging the model to err on the safe side, i.e., penalizing errors that would cause collisions with obstacles more than those that would cause the robot to stop in advance. Experimental results show that our approach is able to satisfactorily identify traversable areas and to generalize to unseen locations.

Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target object in the form of a single, arbitrary-pose RGB image of that object, enabling the system to generalize to unseen objects without retraining. To achieve such a system, we combine several advances in deep reinforcement learning and present a large-scale distributed training system using synchronous SGD that seamlessly scales to multi-node, multi-GPU infrastructure to make rapid prototyping easier. We train and evaluate our system in a simulated environment, identify key components for improving performance, analyze its behaviors, and transfer to a real-world setup.

This paper explores the heterogeneous task allocation problem in Multi-robot systems. A game-theoretic formulation of the problem is proposed to align the goal of individual robots with the system objective. The concept of Nash equilibrium is applied to define a desired solution for the task allocation problem in which each robot can allocate itself to an appropriate task group. We also introduce a market-based distributed mechanism, called DisNE, to allow the robots to exchange messages with tasks and move between task groups, eventually reaching an equilibrium solution. We carry out comprehensive empirical studies to demonstrate that DisNE achieves near-optimal system utility in significantly shorter computation times when compared with the state-of-the-art mechanisms.

We propose a scalable cooperative control approach which coordinates a group of rigidly connected autonomous surface vessels to track desired trajectories in a planar water environment as a single floating modular structure. Our approach leverages the implicit information of the structure’s motion for force and torque allocation without explicit communication among the robots. In our system, a leader robot steers the entire group by adjusting its force and torque according to the structure’s deviation from the desired trajectory, while follower robots run distributed consensus-based controllers to match their inputs to amplify the leader’s intent using only onboard sensors as feedback. To cope with the nonlinear system dynamics in the water, the leader robot employs a nonlinear model predictive controller (NMPC), where we experimentally estimated the dynamics model of the floating modular structure in order to achieve superior performance for leader-following control. Our method has a wide range of potential applications in transporting humans and goods in many of today’s existing waterways. We conducted trajectory and orientation tracking experiments in hardware with three custom-built autonomous modular robotic boats, called Roboat, which are capable of holonomic motions and onboard state estimation. Simulation results with up to 65 robots also prove the scalability of our proposed approach.

We consider the problem of collaborative transport of a payload using several quadrotor vehicles. The payload is assumed to be a rigid body and is attached to the vehicles with rigid rods. The model of the system is presented and is employed to formulate a Model Predictive Controller. The centralized MPC formulation differs from others in the literature in the way the linearized model of the system is employed about a non-equilibrium state-input pair. We then present a decentralized formulation of MPC by distributing the computations among the vehicles. Simulations of both versions of the controller are carried out for a four-quadrotor system carrying out a transport maneuver of a box payload, for a cost penalizing the deviations of the vehicles from the desired trajectory and the attitude perturbations of the payload. The results confirm that the decentralized controller can yield a comparable performance to the centralized MPC implementation, for the same computation time of the two algorithms.

Successful and meaningful human-robot interaction requires robots to have knowledge about the interaction context - e.g., which objects should be interacted with. Unfortunately, the corpora of interactive objects is - for all practical purposes - infinite. This fact hinders the deployment of robots with pre-trained object-detection neural networks other than in pre-defined scenarios. A more flexible alternative to pre-training is to let a human teach the robot about new objects after deployment. However, doing so manually presents significant usability issues as the user must manipulate the object and communicate the object's boundaries to the robot. In this work, we propose streamlining this process by using automatic object location proposal methods in combination with human gaze to distill pertinent object location proposals. Experiments show that the proposed method 1) increased the precision by a factor of approximately 21 compared to location proposal alone, 2) is able to locate objects sufficiently similar to a state-of-the-art pre-trained deep-learning method (FCOS) without any training, and 3) detected objects that were completely missed by FCOS. Furthermore, the method is able to locate objects for which FCOS was not trained on, which are undetectable for FCOS by definition.

Passive dynamic walking is a model that walks down a shallow slope without any control or input. This model has been widely used to investigate how stable walking is generated from a dynamic viewpoint, which is useful to provide design principles for developing energy-efficient biped robots. However, the basin of attraction is very small and thin, and it has a fractal-like complicated shape. This makes it difficult to produce stable walking. Furthermore, the passive dynamic walking shows chaotic attractor through a period-doubling cascade by increasing the slope angle, and the chaotic attractor suddenly disappears at a critical slope angle. These make it further difficult to produce stable walking. In our previous work, we used the simplest walking model and investigated the fractal-like basin of attraction based on dynamical systems theory by focusing on the hybrid dynamics of the model composed of the continuous dynamics with saddle hyperbolicity and the discontinuous dynamics by the impact at foot contact. We elucidated that the fractal-like basin of attraction is generated through iterative stretch and bending deformations of the domain of the Poincaré map by sequential inverse images of the Poincaré map. In this study, we investigated the mechanism for the disappearance of the chaotic attractor by improving our previous analysis. In particular, we focused on the range of the Poincaré map to specify the regions to be stretched and bent by the inverse image of the Poincaré map. We clarified the condition for the chaotic attractor to disappear and the mechanism why the chaotic attractor disappears based on the stretch-bending deformation in the basin of attraction.

In robot telemanipulation tasks, the robot can sometimes occlude a target object from the user's view. We investigate the potential of diminished reality to address this problem. Our method uses an optical see-through head-mounted display to create a diminished reality illusion that the robot is transparent, allowing users to see occluded areas behind the robot. To investigate benefits and drawbacks of robot transparency, we conducted a user study that examined diminished reality in a simple telemanipulation task involving both occluded and unoccluded targets. We discovered that while these visualizations show promise for reducing user effort, there are drawbacks in terms of task efficiency and user preference. We identified several friction points in user experiences with diminished reality interfaces. Finally, we describe several design trade-offs among different visualization options.

In magnetic resonance imaging (MRI) guided robotic catheter ablation procedures, reliable tracking of the catheter within the MRI scanner is needed to safely navigate the catheter. This requires accurate registration of the catheter to the scanner. This paper presents a differential, multi-slice image-based registration approach utilizing active fiducial coils. The proposed method would be used to pre-operatively register the MRI image space with the physical catheter space. In the proposed scheme, the registration is performed with the help of a registration frame, which has a set of embedded electromagnetic coils designed to actively create MRI image artifacts. These coils are detected in the MRI scanner's coordinate system by background subtraction. The detected coil locations in each slice are weighted by the artifact size and then registered to known ground truth coil locations in the catheter's coordinate system via least-squares fitting. The proposed approach is validated by using a set of target coils placed withing the workspace, employing multi-planar capabilities of the MRI scanner. The average registration and validation errors are respectively computed as 1.97 mm and 2.49 mm. The multi-slice approach is also compared to the single-slice method and shown to improve registration and validation by respectively 0.45 mm and 0.66 mm.

This paper proposed a technique to introduce the membrane protein into the lab-on-chip analysis system having a planar lipid bilayer. The proposed technique utilized a dielectrophoretic(DEP) force generated by the asymmetric configuration of the solid electrodes on the aqueous buffer separator. By applying the alternating current to the separator and the counter electrode, we manipulated liposomes that could host the membrane proteins on the surface. The key point for the dielectrophoretic manipulation on this system was the effective configuration of the droplet separator having the taperedge on the contour of the micropore. This configuration made a strong interpenetrating DEP force at the lipid bilayer, and prompted the fusion of liposome into the lipid bilayer. The separator was fabricated by micromachining techniques. Using the separator, we formed the lipid bilayer without evading the solid electrode on the surface. Finally, we elucidated the introduction of the liposome by monitoring with the optical microscopy.

Wireless capsule endoscopy (WCE) is a novel imaging tool that allows the noninvasive visualization of the entire gastrointestinal (GI) tract without causing discomfort to the patients. Although convolutional neural networks (CNNs) have obtained promising performance for the automatic lesion recognition, the results of the current approaches are still limited due to the small lesions and the background interference in the WCE images. To overcome these limits, we propose a Third-order Attention guided Lesion Amplification Network (TALA-Net) for WCE image classification. The TALA-Net consists of two branches, including a global branch and an attention-aware branch. Specifically, taking the high-level features in the global branch as the input, we propose a Third-order Attention (ToA) module to generate attention maps that can indicate potential lesion regions. Then, an Attention Guided Lesion Amplification (AGLA) module is proposed to deform multiple level features in the global branch, so as to zoom in the potential lesion features. The deformed features are fused into the attention-aware branch to achieve finer-scale lesion recognition. Finally, predictions from the global and attention-aware branches are averaged to obtain the classification results. Extensive experiments show that the proposed TALA-Net outperforms state-of-the-art methods with an overall classification accuracy of 94.72% on the WCE dataset.

Juggling manipulation is one of difficult manipulation to acquire since some of such manipulation is unstable and also its physical model is unknown due to the complex non-prehensile manipulation. To acquire these unstable unknown-dynamics juggling manipulation, we propose a method for designing the predictive model of manipulation with a deep neural network, and a real-time optimal control law with some robustness and adaptability using backpropagation of the network. In this study, we apply this method to diabolo orientation stabilization, which is one of unstable unknown-dynamics juggling manipulation. We verify the effectiveness of the proposed method by comparing with basic controllers such as P Controller or PID Controller, and also check the adaptability of the proposed controller by some experiments with a real life-sized humanoid robot.

Unsupervised learning of depth and ego-motion from unlabelled monocular videos has recently drawn great attention, which avoids the use of expensive ground truth in the supervised one. It achieves this by using the photometric errors between the target view and the synthesized views from its adjacent source views as the loss. Despite significant progress, the learning still suffers from occlusion and scene dynamics. This paper shows that carefully manipulating photometric errors can tackle these difficulties better. The primary improvement is achieved by a statistical technique that can mask out the invisible or nonstationary pixels in the photometric error map and thus prevents misleading the networks. With this outlier masking approach, the depth of objects moving in the opposite direction to the camera can be estimated more accurately. To the best of our knowledge, such scenarios have not been seriously considered in the previous works, even though they pose a higher risk in applications like autonomous driving. We also propose an efficient weighted multi-scale scheme to reduce the artifacts in the predicted depth maps. Extensive experiments on the KITTI dataset show the effectiveness of the proposed approaches. The overall system achieves state-of-the-art performance on both depth and ego-motion estimation.

This study develops a training system for a multimodal comprehensive care methodology for dementia patients called Humanitude. Humanitude has attracted much attention as a gentle and effective care technique. It consists of four main techniques, namely, eye contact, verbal communication, touch, and standing up, and more than 150 care elements. Learning Humanitude thus requires much time. To provide an effective training system for Humanitude, we develop a training system that realizes sensing and interaction simultaneously by combining a real entity and augmented reality technology. To imitate the interaction between a patient and a caregiver, we superimpose a three-dimensional CG model of a patient’s face onto the head of a soft doll using augmented reality technology. Touch information such as position and force is sensed using the whole body wearable tactile sensor developed to quantify touch skills. This training system enables the evaluation of eye contact and touch skills simultaneously. We build a prototype of the proposed training system and evaluate the usefulness of the system in public lectures.

In this study, a new type of sling for assisting bedridden patients is developed using a pneumatic growing mechanism. Growing Sling focuses on minimizing the labor input of the caregivers by automating the sling insertion and retraction process while maintaining safety and comfort. Improvements over the typical growing mechanism were made by reinforcing the sling with shafts and filament tape for restricting the height of the sling to ensure its design purpose. Analysis of forces exerted on the structure was made to interpret the driving power of the automated insertion process and to ensure the structural integrity of components. Experiments on materials and prototype devices were conducted to determine the quantitative load that the sling needs to endure and what type of material is suitable for fabrication. Further, we propose a fabrication process for the Growing Sling, including its dimensions, and validate the performance of the fabricated prototype.

A mobile robot that can achieve a stable attitude and locomotion on steep slopes is needed to overcome the problems of slipping and falling for automation of works on steep slopes. The conventional approaches to achieve a stable attitude and locomotion have been researched by adopting tracked wheels and multi-legged mechanisms instead of wheel mechanisms. However, these robots have limitations in term of application angles. A systematic theory for stable attitude and locomotion on steep slopes has not been established. Therefore, research on control strategies for wheeled mobile robots on steep slopes is essential. In this paper, a method to realize a stable attitude and locomotion on a steep slope for the wheeled mobile robot by using propellers for propulsion adhesion is proposed. The proposed robot can generate a large frictional force by pushing its body against the slope with a thrust force. This force prevents the robot from slipping while maneuvering on the slope. The magnitude and the direction of the thrust force is optimized using an appropriate control mechanism influencing the moment of force acting on it to avoid falling and side slipping during locomotion on steep slopes. A simulation experiment was conducted from the perspective of mechanics and dynamics to arrive at an optimal design of the mobile robot. The developed robot has four propellers to generate thrust forces and a rotation axis to control the direction of the generated thrust forces. Evaluation experiments were performed to validate the stability of the robot at a resting position and during lateral locomotion and its ability to climb over a slope. The experimental results confirmed that the proposed robot with propellers realized a steady attitude and locomotion on a slope of up to 90° by controlling the magnitude and the direction of the thrust force.

A spherical two-degree-of- freedom wrist adapting the structure of the spatial parallelogram is proposed. A U type extended link out of three UU type limbs of the spatial parallelogram is selected as an output link. As a result, the wrist can be interpreted as being formed by combination of a U type limb and a (2-UU)+U type hybrid limb. Screw theory is employed to analyze its first-order kinematic model. Then a compact wrist prototype suitable for wrist module supporting the robot hand is designed and implemented. Finally, experiments with the prototype confirm that the wrist has a very high potential application for wrist modules in terms of dexterity and maximum load handling capacity.

This article introduces an innovative model-based strategy for designing a legged robot to generate animal-like running dynamics with differentiated leg braking and thrusting force patterns. Linear springs were utilized as legs, but instead of having one end of each spring connected directly to the hip joint, one extra bar was added to offset the spring's direction. The robot's front and hind legs were offset with the same magnitudes but in different directions. Therefore, the legs produced different ground braking and thrusting force patterns. The robot's running motion was planned based on its reduced-order model. The model's fixed-point and passive-dynamics motion served as the robot's reference motion. The proposed strategy was experimentally validated, and the results confirmed that the robot could successfully perform stable running in a differentiated leg force pattern.