SIGGRAPH 2022 论文列表

Virtual Production fulfills George Lucas's early dream of having an engulfing “space-opera in the sky” (1). Epic Games’ focus on realistic interactive 3D game environments using Unreal Engine, revolutionized the field of film-making, by replacing rear film projections with large format, curved, high resolution, immersive LED video screens, allowing backdrops to adapt in real time to the narrative needs of each scene by tracking the movement of the camera. Cinematographers and Art Directors are adapting to the challenges of virtual and real lighting and props, recruiting animators and new media developers who create, usually in very little time, virtual and real props, and metahuman actors and characters, enhancing the production value, optimizing and reducing costs in unparalleled ways. This poster presents the results of the first Virtual Production class offered by the Film Animation and New Media Department at the University of Tampa. In a very short time span, students working in interdisciplinary teams have seen the possibilities of these new technologies for science fiction, fantasy and experimental films that otherwise would have been impossible to create with very limited student budgets.

When learning a dexterous skill such as playing the piano, people commonly watch videos of a teacher. However, this conventional way has some downsides such as limited information to be retrieved and less intuitive instructions. We propose a virtual training system by visualizing differences between hands to provide intuitive feedback for skill acquisition. After synchronizing the data, two visual cues are proposed including a hand-overlay manner and a two-keyboards visualization. A pilot study confirm the superiority of the proposed methods over conventional video-viewing.

In this study, we propose a stereoscopic transparent display that can be viewed with the naked eye. Existing methods for generating realistic high-quality stereoscopic images require wearable devices and the presence of a display, which degrades the sense of presence. Our method increases the sense of presence by making the stereoscopic images blend into the surrounding environment.

In everyday life, localizing a sound source entails more than the sole extraction of auditory cues to define its three-dimensions: spatial hearing also takes into account the available visual information (e.g. cues to sound position) and resolves perceptual ambiguities through active listening behavior (e.g. exploring the auditory environment with head movements). We introduce a novel approach to sound localization in 3D named SPHERE which exploits a commercially available Virtual Reality Head-mounted display system with real- time kinematic tracking to combine all of these elements (controlled positioning of a real sound source, recording of participants’ responses in 3D, controlled visual stimulations and active listening behavior). SPHERE allows accurate sampling of 3D spatial hearing abilities, and allows detection and quantification of the contribution of active listening.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). SIGGRAPH ’22 Posters, August 07-11, 2022, Vancouver, BC, Canada © 2022 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-9361-4/22/08. https://doi.org/10.1145/3532719.3543245 2015] and hardware-related appropriation [Ludwig et al. 2019]. On-desktop projection have been proposed for learning support systems for math development [Beşevli et al. 2019] and typing on a physical keyboard [Sono and Hasegawa 2019]. On-body projection systems were developed for education [Hoang et al. 2017][Ferdous et al. 2019]. Although projection-based human-machine interfaces have attracted attention, there have been very few works on integration with sensor-based activity recognition. Therefore, in this paper we propose an SAR assistance system with accelerometer and projection mapping. We considered cleaning activities [Yaegashi et al. 2022] for recognition to achieve wellbeing during and after the pandemic. The contribution of the proposed idea is to integrate SAR and real-time activity recognition technologies. The sensor-based activity recognition is advantageous in terms of privacy compared with well-known gesture recognition systems using a camera. The fixed floor-projection also contributes for disuse of a camera by avoiding dynamic calibration. A small and light-weight sensor device mounted on a cleaning tool does not affect the action. Low-latency feedback is achieved by an edge computing based architecture.

. Spatially varying bidirectional reflectance distribution functions (SVBRDFs) play an important role in appearance modeling of real-world surfaces. Automatic capture of these surface properties is highly desirable, but many techniques only partially capture these properties or use complicated setups to do so. Micro surface roughness variations are especially difficult to capture using image-based methods. In this paper, we propose a novel approach towards estimating the complete SVBRDFs of surfaces using a portable projector-camera system made of standard consumer-grade components. Our approach uses insights about the relations between the illumination and viewing geometry and captured image statistics to estimate surface reflectance properties. Our technique should be of great value to practitioners seeking to model and render the geometric and reflectance properties of complex real-world surfaces.

Recently, many methods have been proposed to realistically render various materials. The realism of the synthetic images can be improved by rendering small-scale details on the surfaces of 3D objects. We focus on the efficient rendering of the scratches on the transparent objects. Although a fast rendering method using precomputed 2D BRDFs for a scratched material has been proposed, the method is limited to the opaque materials such as metals. We extend this method to the transparent objects. On the surface of the transparent object, rays are split into specular reflections and refractions. We therefore precompute Bidirectional Scattering Distribution Functions (BSDFs). We use a 2D ray tracer as in the previous method to accelerate the precomputation. We show several examples to demonstrate the effectiveness of our method.

We introduce a novel method that describes the functionality and characteristics of an efficient VR recorder with replay capabilities, implemented in a modern game engine, publicly available for free.

The lens distortion is essential for displaying VR contents on a head-mounted display (HMD) with a distorted display surface. We propose a novel lens distortion algorithm on an embedded GPU system. To minimize the memory access overhead, we propose a compressed form of a lookup table. We also utilize the integrated memory architecture of the edge GPU system (e.g., NVIDIA’s Jetson devices) to reduce data communication overhead between host and device. As a result, our method shows up to 1.72-times higher performance than prior lookup table-based lens distortion approaches while it consumes up to 28.93% less power. Finally, our algorithm achieved real-time performance for high-resolution images on edge GPU systems (e.g., 94 FPS for 8K image on Jetson NX). These results demonstrate the benefits of our approach from the perspectives of both performance and energy.

Stylized Novel View Synthesis is an emerging technique that combines style transfer and view synthesis. However, none of the existing works explore their applications in Virtual Reality (VR). This work devises a novel application for stylized novel view synthesis. We propose to replace actual 3D scene models or 360 images with stylized stereoscopic images for the areas outside the major play area but are still visible to the user. User study results reveal that users can feel 3D sense and tell them from plane texture. Codes and other materials are available at: kuan-wei-tseng.github.io/ArtNV

We present ProjecString, a touch-sensitive string curtain projection display that encourages novel interactions via touching, grasping, and seeing and walking through the display. We embed capacitive-sensing conductive chains into an everyday string curtain, turning it into both a space divider and an interactive display. This novel take on transforming an everyday object into an interactive projection surface with a unique translucent property creates novel interactions that are both immersive and isolating.

The goal of our research is to control a physics-based character that learns several dynamic motor skills using conditional Generative adversarial imitation learning(GAIL). We present a network-based learning algorithm that learns various motor skills and changing motions between the motor skills from disparate motion clips. The overall framework for our controller is composed of a control policy which generates a character’s behavior, and a discriminator which induces the policy to produce proper motions from a user’s commands. The discriminator and the policy take outputs from each other as input and improve each performance through an adversarial training process. Using this system, when a user commands a specific motion to the character, the character can design a motion plan to perform the motion from the current pose. We demonstrated the effectiveness of our approach through examples with an interactive character that learns various dynamics motor skills and follows a user command in the physics simulation.

Translucent materials are ubiquitous in our daily lives, from organic materials such as food, liquids or human skin, to synthetic materials like plastic or rubber. In these materials, light penetrates inside the surface and scatters in the medium before leaving it. While the physical phenomena responsible for translucent appearance are well known, understanding how human observers perceive this type of materials is still an open problem: The appearance of translucent objects is affected by many dimensions beyond the optical properties of the material, including shape and illumination. In this work, we focus on the effect of illumination on the appearance of translucent materials. In particular, we analyze how static and dynamic illumination impact the perception of translucency. Previous studies have shown that changing the illumination conditions results in a constancy failure, specially in media with anisotropic phase functions. We extend this line of work, and analyze whether motion can alleviate such constancy failure. To do that, we run a psychophysical experiment where users need to match the optical density of a reference translucent object under both dynamic and static illumination. Surprisingly, our results suggest that in most cases light motion does not impact the perceived density of the translucent material. Our findings can have implications for material design in predictive rendering and authoring applications.

Transient imaging methods often analyze time-resolved light transport for applications such as range imaging, reflectance estimation and, especially, non-line-of-sight (NLOS) imaging, which targets the reconstruction of hidden geometry using measurements of indirect diffuse reflections emitted by a laser. Transient rendering is a key tool for developing such new applications. In this work, we introduce a set of simple, yet effective subpath sampling techniques targeting transient light transport simulation in occluded scenes. We analyze the usual capture setups of NLOS scenes, where the light and camera indirectly aim at hidden geometry through a secondary surface. We leverage that configuration to reduce the integration path space. We implement our techniques in our modified version of Mitsuba 2, adapted for transient light transport, allowing us to support parallelization, polarization, and differentiable rendering.