SIGGRAPH 2021 Papers

Gait is a key barometer to analyze human body conditions. We propose a personalized gait analysis framework which can diagnose a possible muscularskeletal disorders with a single gait cycle. Our framework built over a gait manifold which reveals the principle kinematic characteristics in the temporal pose sequence. Body parameters such as muscle, skeleton, and joint limits for an arbitrary gait cycle can be approximated by measuring similarity in the small latent space. We present a physical gait simulator to enrich the gait space paired with the body conditions.

The Qiang is an ancient minority in western China. However, the number of people who can speak the Qiang language is decreasing due to a lack of written text. Although the protection of intangible cultural heritage has been widely discussed, there is still a dearth of interactive design for the Qiang people’s language and culture. This research aims to determine the efficacy of tangible interactive games to encourage Qiang people’s interest in learning the Qiang language and increase their cultural awareness. To better understand the current state and challenges of Qiang language and culture, we conducted a three-day field investigation in the Qiang villages. Based on the field study’s key findings, we created ”WhiteStone,” a tangible interactive projection device based on the heroic epic of Qiang. This poster examines the design opportunities for tangible interactive games to revitalize the Qiang language and culture.

We extend Neural Radiance Fields (NeRF) with a cylindrical parameterization that enables rendering photorealistic novel views of 360° outward facing scenes. We further introduce a learned exposure compensation parameter to account for the varying exposure in training images that may occur from casually capturing a scene. We evaluate our method on a variety of 360° casually captured scenes.

We present a learning-based method for generating animated 3D pose sequences depicting multiple sequential or superimposed actions provided in long, compositional sentences. We propose a hierarchical two-stream sequential model to explore a finer joint-level mapping between natural language sentences and the corresponding 3D pose sequences of the motions. We learn two manifold representations of the motion –- one each for the upper body and the lower body movements. We evaluate our proposed model on the publicly available KIT Motion-Language Dataset containing 3D pose data with human-annotated sentences. Experimental results show that our model advances the state-of-the-art on text-based motion synthesis in objective evaluations by a margin of 50%.

In this paper, we introduce TeraFoils, a method for designing and fabricating material-based structures using binary holograms in the terahertz region. We outline the design, fabrication, imaging, and data processing steps for embedding information inside physical objects and exploring a method to create holographic structures with silver-foiled paper. This paper is a sheet on which silver foil is pasted where the ink is printed, using a home-use laser printer and an electric iron. Wave propagation calculations were performed to design a binary-amplitude hologram. Along with the designed pattern, we fabricated silver-foiled binary holograms in the sub-terahertz range (0.1 THz) and confirmed their functions using a two-dimensional THz sensor.

We propose a novel deep neural network-based learning framework that understands acoustic information in the variable-length sequence of vocal tract shaping during speech production, captured by real-time magnetic resonance imaging (rtMRI), and translate it into text. In an experiment, it achieved a 40.6% PER at sentence-level, much better compared to the existing models. We also performed an analysis of variations in the geometry of articulation in each sub-regions of the vocal tract with respect to different emotions and genders. Results suggest that each sub-regions distortion is affected by both emotion and gender.

Screenshot is a frequently used tool in our daily life, while the screenshot capturing techniques are not much discussed in computer graphics and image processing researches. Capturing a screenshot is not always as easy as it seems. Firstly, the target devices for screenshot capturing must have screenshot software installed or featured in their operating systems. Secondly, the users must have input access to control the screenshot software within the target devices. Thirdly, the target devices must have Internet access or other hardware interfaces (such as USB ports) so that the users can take their screenshots out. When these requirements are not met, people often need to use their smartphones to take photographs in front of the screens as a substitute of screenshots. This allows direct sharing of the screen content, but the fidelity of the obtained content is apparently not as good as software screenshots. Might we be able to achieve a computer graphic solution to directly convert a screen photography to a screenshot, which looks like as if it was taken using software?

Fracture of solid objects produces debris. Modelling the physics that produces the broken fragments from the original solid requires an increase in the number of degrees of freedom. This causes a huge increase in computational cost for FEM based methods used to model such phenomena. We present a graph-based FEM method that tackles this issue by relabeling the edges of the graph induced in a volumetric mesh, using a damage variable. We reformulate the system dynamics for this relabelled graph in order to simulate the fracture mechanics using FEM without an explosion in the computation cost. Our method therefore requires no remeshing of the volumetric mesh used for computation and this makes it very scalable to high-resolution meshes. We demonstrate that the method can simulate both brittle and ductile fracture.

We present a method to infer physically-based material properties for free-viewpoint video. Given a multi-camera image feed and reconstructed geometry, our method infers material properties, such as albedo, surface normal, metallic and roughness maps. We use a physically based, differentiable renderer to generate candidate images which are compared against the image feed. Our method searches for material textures which minimise an image-space loss metric between candidate renders and the ground truth image feed. Our method produces results that approximate state of the art reflectance capture, and produces texture maps that are compatible with common real-time and offline shading models.

ACM Reference Format: Masaki Takahashi,MakikoAzuma, TakuyaHanda, Taichi Ishiwatari, Masanori Sano, and Yuko Yamanouchi. 2021. Real-time sports video analysis for video content viewing with haptic information . In Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters (SIGGRAPH ’21 Posters), August 09-13, 2021. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3450618.3469135

This paper describes an OmniProcam system, which enables 360 degree horizontal projection by a fisheye lens with a coaxial procam in which the optical axes of the camera and projector are exactly matched. Combined with 2D marker recognition, the OmniProcam can display images onto screens at arbitrary positions in 3D space. As an example application, we developed a system which projects lip animation onto the user’s face masks for better communication at the physical meeting in current COVID-19 situation. The system recognizes the user’s speech, generates the lip animation using Lipsync, and projects the animation onto the user’s face masks.

Quantum Nodes is a Blender add-on that introduces the integration of quantum algorithms into the 3D creation process. Our work focuses on allowing users to experiment both new forms of creation and approaching the concepts of quantum computing through 3D creation.

The appearance of a real-world feather is the result of light interactions with complex, patterned structures of varying scale; however, these have not yet been modeled for accurate rendering of feathers in computer graphics. Previously published works have presented simplified curve models for feather appearance. Using imaging from real feathers, we suggest why these approaches are not sufficient and provide motivation for building an appearance model specific to feathers. In that vein we demonstrate a new technique that takes into account the substructures of feathers during shading calculations to produce a more accurate far-field appearance.

We present Pockets, a simple means of organizing and carrying 3D tools and other objects in virtual environments. Previous examples exist of using 3D tools with visually obvious affordances in virtual immersive environments instead of more traditional menus, however, in these applications a 2D menu is still necessary to select 3D tools from. Pockets make use of a belt with physical buttons, that objects can be assigned to. The Pockets design not only enables users to use their muscle memory to store and retrieve objects, thereby making tool use more efficient, but also solves the occlusion problem associated with state of the art approaches such as 2D menus tied to the body or to world space.

We introduce a method to automatically convert a single panoramic input into a multi-cylinder image representation that supports real-time, free-viewpoint view synthesis for virtual reality. We apply an existing convolutional neural network trained on pinhole images to a cylindrical panorama with wrap padding to ensure agreement between the left and right edges. The network outputs a stack of semi-transparent panoramas at varying depths which can be easily rendered and composited with over blending. Initial experiments show that the method produces convincing parallax and cleaner object boundaries than a textured mesh representation.

Figure 1: An example of sequence of non-destructive effects transforming a simple editable base mesh (e.g. a cube) into a more complex asset. Operations like bevel, extrusion or boolean intersection are available in most of common mesh-based 3D modeling suites, but the effect applied at the second step is less common. By implementing it as an OpenMfx plug-in, it is available in any supporting host software and thus the asset is interexchangable in its non-destructive form.

ACM Reference Format: Toshikatsu Nakamura, Ryusuke Miyazaki, Kouga Yoshida, and Toshiki Sato. 2021. Omnidirectional display that presents information to the ambient environment with optical transparency. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters (SIGGRAPH ’21 Posters), August 09-13, 2021. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3450618.3469139