SIGGRAPH Asia 2022 Papers

Rhythm is important to improve skills in various sports. In this study, we create a rhythm game for ski training. It synchronizes music and turn timing, and has various types of feedback for training. We conducted a pilot study to verify the effectiveness through a comparison of three conditions.

Biofeedback is a well-known form of therapy in which the patient receives sensory and auditory input on their physiology to help them reflect on, recognize, and be aware of their own state in order to improve their cognitive and emotional functioning. In our approach, a user’s self-avatar is generated based on their physiological state, specifically their heart rate and electrodermal activity. Moreover, based on self-reported anxiety symptoms, we investigate if the biofeedback self-immersive avatar can help reduce stress levels.

We explore the application of a time-dependent machine learning framework to art direction of volumetric simulations. We show the benefit of the time dependency inherent to the ODE-net model when used in conjunction with simulation sequences. Unlike other machine learning methods which maintain a uniform timestep constraint during evaluation, the ODE-net framework is able to generate results for arbitrary time samples. We demonstrate how this non-uniform time step evaluation can be leveraged for use in artistic direction tasks. We specifically apply the model to the retiming of volumetric simulations to showcase the ability of the machine learning method to properly predict arbitrary time steps. We show that with minimal training data, the model is able to generalize over several simulation sequences with similar parameters.

We introduce an image super-resolution technique for high-quality texture mapping in this poster. We first get upscaled textures from an existing image super-resolution (SR) method. We then perform a post-color correction algorithm to restore color tones and details lost in the SR algorithm. Finally, we compress the textures with variable compression ratios to reduce storage and memory overheads caused by the increased resolution. As a result, TexSR can improve the image quality of a state of the art, Real-ESRGAN.

In many sports, rhythmic skills are considered important. In this paper, we take juggling as an example and propose a VR system that simplifies the acquisition of a sense of rhythm. The proposed system uses temporal and spatial distortion and other functions to assist the training. A pilot study is conducted to validate the effectiveness of each function.

Figure 1: Comparison of reconstruction results of large-scale indoor scene with high ambiguity and repeated structures. (a) Result of COLMAP [Schönberger and Frahm 2016], (b) result of [Su et al. 2020], and (c) result of our method. Both (a) and (b) indicate failure to reconstruct some ambiguous areas and floor collapse. By contrast, our method could successfully construct a complete model, and all parts were connected correctly. ACM Reference Format: Chia-Hung Yu and Kuan-Wen Chen. 2022. Robust and Efficient Structurefrom-Motion Method for Ambiguous Large-Scale Indoor Scene. In SIGGRAPH Asia 2022 Posters (SA ’22 Posters), December 06-09, 2022. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3550082.3564199

In this work, we propose a robust pipeline to create vectorized models from LiDAR point clouds without the assumption of watertight polygonal surfaces. The core idea behind our method is combining the hierarchical 2D projection with the pairwise intersections of clipped planes to precisely hypothesize faces of an object inside and outside while enhancing the robustness of different scenes. Moreover, to generate concise polygon meshes efficiently, Adaptive K-means++ (AKM++) is used to smooth the 2D rasterized height map followed by the Levenberg-Marquardt (LM) algorithm to optimize the projection primitive with planar intersections jointly. The final model could be easily obtained by merging the derived neighbouring facets above. Our experimental results show that the proposed method obtains compact models with high fidelity and efficiency on either precise or defect-laden data compared with other state-of-the-art approaches.

We use existing machine learning neural style transfer model, differential rasterizer, for colored font design. The input of the proposed system is an existing TrueType font and the output is an neural style transferred OpenType-SVG color font. Each character glyph contains a series of vector graphics path elements. A neural style transferred glyph consists of dozens of random Bézier curves initially distributed among the glyph space, gradually converging to glyph stokes of a given character. In contract, the process of designing a font manually requires specialized skills for designers. They use mouse cursor to adjust strokes and glyph kerning for each character. It require hours to achieve a specific style. A Chinese font contains thousands of characters. Neural style transfer techniques can be used to speed up the process for designing decorative fonts. In addition, we add different colors to the curves in a an neural style transfer character for visual appealing. In order to support colors in a font, we choose OpenType-SVG format, allowing displaying colors in software like Illustrator or Inkscape. We open-source our code at https://github.com/su8691/ribbon.

In recent years, most productions of 3D video games dealing with cars have represented automobile damage due to collisions. However, the cracks peculiar to FRP materials have not been reproduced in previous works. Therefore, in this research, we devised a method of expressing cracks in FRP materials. In addition, we will conduct experiments to determine the realism of the visual representation of breakage as well as the processing cost and consider whether it is possible to incorporate it into actual video game works.

Image-based pose estimation generally needs 3D map generation, which is costly in practice. In this paper, we propose a method to accurately estimate the pose by recursively rendering 2D images from a generic 3D city mesh map and updating the pose.

In this study, we developed a flow to simulate anisotropy in multi-color printing of a FFF 3D printer. There are previous methods for simulating layered marks from gcode, which is a 3D printer control statement, and multi-color printing by switching filament colors and their simulators. However, there was no simulator for the case where color anisotropy exists in a single path, which is the target of this study. Since the coloring of this method is based on a completely different principle from existing methods, it is difficult to predict the results. Our simulator tool was very useful in rendering multi-color 3D printed models. Furthermore, a more sophisticated simulation of the placement of objects in real space was performed based on the highly realistic images created by physical-based rendering. The simulation also worked in VR space and showed an optimal preview at the architectural scale.

In light of the COVID-19 pandemic, wearing a mask is crucial to avoid contracting infectious diseases. However, wearing a mask is known to impair communication functions. This study aims to address the communication difficulties caused by wearing a mask and provide a strategy for aiding in understanding the speaker’s speech through facial animation. Facial animation is generated in real-time, and upper facial information is processed to detect the speaker’s emotions, generating a lower facial expression. In addition, the system detects the mask’s shape and enables accurate registration in the proper position. This technology can improve communication and alleviate challenges associated with communication between persons wearing face masks.

Color vision tests are conducted based on subjective responses; however, since subjective responses can be biased and are not uniformly applicable to all individuals, an objective method is required. In this study, we developed a quantitative and user-friendly color vision test, which utilized a psychophysical method based on pupillary responses to color flicker stimuli sampled from the confusion line. The results revealed that power spectral density of pupillary oscillations increased corresponded to the color differences. Our system could eventually lead to a feasible color testing method.

Figure 1: System overview. A remote worker captures the area with a camera and streams the image (we call it rst-person view) to our system. Then, the system estimates the camera’s position and direction from the image and the pre-scanned 3D geometry. Finally, our system generates a third-person view by overlapping the captured image and the location information on the geometry. The receiver can switch these views freely. (a) First-person view: The image captured by remote worker is displayed in the center, with the pre-scanned geometry around it. This view moves with the movement of the remote worker. (b) Switching between rst-person and third-person view: The receiver can switch between the two views. (c) Third-person view: The user can move the viewpoint freely in the geometry model. The rst-person view and the camera’s position are also displayed so that the receiver can know the location of the remote worker.

We propose procedural modeling of crystal clusters. Based on crystallography, we model single crystals and then distribute single crystals on a base rock using a hierarchical sampling approach. Users can interactively generate various crystal clusters, including amethyst and citrine clusters. Furthermore, our approach offers a high-level specification, i.e., target shape so that the generated clusters approximate the shape; such controllability is crucial for realizing artists’ intentions. Additionally, we focused on fabrication-oriented procedural modeling. We identify the possible parameter spaces based on physical experiments, i.e., test the removability of resin casts from silicone molds produced by the 3D printed crystal clusters for making replicas.

We propose a novel image search system with color palettes. By querying color palettes, users can search for inspiring images, which is helpful for design exploration. Although few such systems can accept palettes as queries, they have several constraints on query palettes (e.g., limited palette size) or search results. Our system accepts palettes with color weights and any sizes as the inputs and returns results with enough diversity that can stimulate users’ design inspiration. To achieve these, we extract color themes from our database with a perceptually-based model and calculate the palette similarity based on a weighted palette distance. Visual comparisons demonstrate that our system has more potential for helping users’ design inspiration than the existing systems.

In this poster, we present Flow Human, a no-code system that generates conversational behavior of digital humans from the text. Our users only need to build a conversation flow they want to talk to customers using the flow-based authoring tool we developed. Our system then automatically generates the verbal and non-verbal behavior of digital humans along the conversation flow, interacts with customers, and collects feedback. We believe that this work can serve the potential to be distributed to various services that have not been introduced because of the challenging task of controlling multiple factors in digital humans (e.g., conversation flow, co-speech gestures, and facial animation).