SIGGRAPH Asia 2022 Papers

We present a motion in-betweening framework to generate high quality, physically plausible character animation when we are given temporally sparse keyframes as soft animation constraints. More specifically, we learn imitation policies for physically simulated characters by using deep reinforcement learning where the policies can access limited information only. Once learned, the physically simulated characters are capable of adapting to external perturbations while following given sparse input keyframes. We demonstrate the performance of our framework on two different motion datasets and also compare our results with the the results generated by a baseline imitation policy.

In this work, we propose a novel Metric-K Nearest Neighbor (-KNN) to facilitate topology aware learning in point clouds. Topology aware learning is achieved by accumulation of local features in deep-learning model. Recent work rely on Ball queries or K-Nearest-Neighbor (KNN) for local feature extraction of point clouds and finds challenges in retaining topological information. -KNN employes a generalised Minkowski distance in the KNN search algorithm for topological representation of point clouds. -KNN enables state-of-the-art point cloud methods to perform topology aware downstream tasks. We demonstrate the performance of -KNN as plugin towards point cloud classification, part-segmentation, and denoising using benchmark dataset.

In recent years, video content that enables users to experience a simulated fall using an head mounted display (HMD) has become popular. However, the visual factors that cause the sensation of falling have not been clarified. In this study, we focused on visual stimuli, which are known to affect the intensity of vection, and examined their relationship to the sensation of falling.The results confirmed that stimulus presented from the peripheral visual field area and increased spatial frequency amplified the sensation of falling. In addition, we were able to produce video content that produced a greater sense of falling by using multiple visual stimuli that had the property of increasing vection.

We introduce MMGrip, a handheld multimodal haptic device that simultaneously presents vibration, impact, and shear for realistic and immersive haptic feedback to virtual collision events. The three types of haptic stimulus mimic a damped vibration, a collision impulse, and a skin slip deformation, respectively, occurring at physical contact. Controlling the three stimuli, as well as their onset time differences, provides a vast design space for offering diverse and realistic haptic experiences. We present the design of MMGrip and demonstrates its application to virtual sports.

We address the problem of 3D human motion estimation from original MoCap optical markers. The original markers are noisy, disordered, and unlabeled, hence recovering 3D human motion from them is non-trivial. Existing works are either time-consuming or assuming the knowledge of the marker labels. We address these problems by presenting an end-to-end method for 3D human motion estimation by leveraging the capability of Transformer to model long-range dependencies. The method takes original markers as inputs and learns joint poses with a Transformer-like architecture. Experimental results show that our method is able to achieve better than centimeter-level errors.

Content-aware image resizing could automatically retarget an image to different aspect ratios while preserving visually salient contents. However, it is difficult for users to interact with the retargeting process and control the results. In this paper, we propose a language-driven diversified image retargeting (LDIR) method that allows the users to control the retargeting process by providing additional textual descriptions. Taking the original image and user-provided texts as inputs, LDIR retargets the image into the desired resolution while preserving the content indicated by texts. Following a self-play reinforcement learning pipeline, a multimodel reward function is proposed by considering both the visual quality and language guidance. Preliminary experiments manifest that LDIR can achieve diversified image retargeting guided by texts.

Skill improvement in sports is an essential factor in keeping the motivation to continue. Previous studies showed that kinesthetic illusion enhances observational learning. However, such studies have only dealt with the learning of movements using a single part of the body, and whether kinesthetic illusion can be induced in observational learning of whole-body movements has not been clarified. In this study, we conducted an experiment involving human subjects and confirmed that synchronized visuo-tactile stimuli can induce kinesthetic illusion even in whole-body movements. Moreover, we also demonstrated a complete mediation model in which the synchronization of visuo-tactile stimuli influences kinesthetic illusion mediated by body ownership.

Recognizing transparent objects such as window and glassware is a challenging task in 3D Reconstruction from color images. In recent years, transparent object recognition methods have focused on feature extraction from boundary region of transparent objects. Our observation is that, unlike non-transparent objects, transparent objects can be characterized and located better by looking at their external and internal boundaries separately. We propose a new internal-external boundaries attention module in which internal and external boundary features are separately recognized. We add an edge-body fully attention module that supervises the segmentation of the generated transparent objects body using semantic information in the external boundaries. We employ contour loss to perform distance-weighted supervision on the inner and outer boundaries separately. Extensive experiments show that proposed method outperforms existing methods on Trans10k dataset.

Figure 1: (a) the proposed fabrication process in which the bi-stable thermochromic materials are mixed with UV curable resin and directly printed from 3D printer. After printing the 3D object, we use the laser irradiation to modify the color and patterns of 3D printed objects. (b) (d) the application scenario of InfiniteShader. We can control and change the patterns of (b) a figure, (c) a smartwatch cover, and (d) a bracelet after its fabrication. (white dashed lines are added for visibility.)

The paper presents different strategies for fusing Bounding Volume Hierarchies (BVHs) of varying level of detail (LODs). Our method makes it possible to traverse BVHs of varying LODs in a cache-friendly way for path tracing. Considering BVH of the finest level (we call it the Base LOD BVH), the method defines a strategy to fuse subtrees from BVH of other LODs to the Base LOD BVH. Our method proposes a heuristic to find such subtrees in non-Base LOD BVHs and an algorithm to find fusion or insertion points for these subtrees in the Base LOD BVH without much increasing the overall the volume of the Base LOD BVH and the number of nodes traversed as a result of this fusing.

We propose an autonomous drone exploration system (ADES) with a lightweight and low-latency saliency prediction model to explore unknown environments. Recent studies have applied saliency prediction to drone exploration. However, these studies are not sufficiently mature. The ADES system proposes a smaller and faster saliency prediction model and adopts a novel drone exploration approach based on visual-inertial odometry (VIO) to solve the practical problems encountered during exploration, i.e., exploring salient objects without colliding with them and not repeatedly exploring salient objects. The system not only has a performance comparable to that of the state-of-the-art multiple-discontinuous-image saliency prediction network (TA-MSNet) but also enables drones to explore unknown environments more efficiently.

This study focused on font type and weight and we examined their effects on the readability of long texts in VR. In the experiment, we changed the font weight of three font types, Gothic, Mincho and Antigothic, to evaluate the readability of text with a length of 600 characters. Our results showed a tendency for Antigothic to be more readable and less fatiguing than Mincho and Gothic.

We propose a lens aberration correction method for holographic displays via a light wave propagation simulation and optimization algorithm. Aberration correction is an important technology to obtain noise-less hologram images in holographic displays. We optimized phase holograms with an automatic differentiation technique and Adam optimizer, and aberration corrected images were achieved. Given an aberrated lens with a focal length of 20mm, the optimized holographic image has a PSNR value of 32.

We propose an application that combines the benefits of augmented reality (AR) and virtual reality (VR) experiences for realistic and effective fire response training. Our analysis of recent literature reveals that there are different advantages to AR versus VR applications, and combining these experiences can generate new compound benefits in fire drills. In this study, a high-level training environment is automatically generated using three-dimensional object recognition technology, and the physical training tools are linked to the VR environment to improve immersion and training effectiveness. The combined advanced AR and VR experience demonstrates the potential use of this method in various industries.