SIGGRAPH 2019 Papers

We present an educational virtual reality (VR) puzzle game set in an archaeological context. We digitally documented the site architecture and a selection of excavated artefact using structure from motion (SfM) mapping, reconstructed the site during Classic Period (AD 250-900) based on the current state and archaeological findings, and created the natural environment using procedural modeling. With this collection of resources, we created a holistic landscape of the Mayan site of Cahal Pech. The player can link the Mayan ruin between its current state and the past through collecting artefact and evidence, and discover the architectural beauty and historical richness of this site.

To develop a graphics project with ease and confidence, the reliability and extensibility of the underlying framework are essential. While there are existing options, e.g., pbrt-v3 [Pharr et al. 2016] and Mitsuba [Jakob 2010], they either focus on education or not being updated for a long time. We would like to present an alternative solution named Photon.

When presbyopic people use digital devices, they often zoom in the display, because it is not in focus when they move it close to face. We have proposed the automatic display zoom system for presbyopic people [Fang and Funahashi 2018]. However, some of the information on the small display has gone out of it after zooming-in (Fig. 2(a) to (b)). It is necessary to scroll it frequently, and a bother. On the other hand, a conventional partial zoom means like a magnifying glass is also usually provided (Fig. 2(a) to (c)). The part around a zoomed area is cut off, and it is necessary to move the glass frequently too. People sometimes want to skim through sentences and understand an overview. By the way, although it is difficult to read blurry words (Fig. 3(a)), you can guess and read a sentence includes the blurry words when some other words are clear (Fig. 3(b)). Therefore, we reconsider the zoom-in method for presbyopic people. For example, the area paid attention is zoomed in to read clearly, and the magnification rate of the area around it is gradually reduced to zoom-out rate so that all information is displayed in the small display even though some words are zoomed out. It is expected that you can guess and read also the unzoomed-in words like blurred words around the clear zoomed-in words. We propose a suitable partial zoom-in function that allows you to skim a document.

Physics-based models for ocean dynamics and optical raytracing are used extensively for rendering maritime scenes in computer graphics [Darles et al. 2011]. Raytracing models can provide high-fidelity representations of an ocean image with full control of the underlying environmental conditions, sensor specifications, and viewing geometry. However, the computational expense of rendering ocean scenes can be high. This work demonstrates an alternative approach to ocean raytracing via machine learning, specifically Generative Adversarial Networks (GANs) [Goodfellow et al. 2014]. In this paper, we demonstrate that a GAN trained on several thousand small scenes produced by a raytracing model can be used to generate megapixel scenes roughly an order of magnitude faster with a consistent wave spectrum and minimal processing artifacts.

For cytology, pathology or histology image analysis, whether performed by computer-aided algorithms or human experts, a general issue is to exclude the disturbance caused by noisy objects, especially when appeared with high similarities in shape, color and texture with target cell or tissues. In this paper, we introduce a novel model to reduce such type of noisy objects with large quantity and distribution in the microscope images based on deep learning and hand-craft features. The model experimentally reduces the false positives without effect on objects of interest for cancer detection. Moreover, it also provides much distinct images for human experts for the final diagnosis.

In this poster, we propose a new haptic rendering algorithm that dynamically modulates wave parameters to convey distance, direction, and object type by utilizing neck perception and the Hapbeat-Duo, a haptic device composed of two actuators linked by a neck strap. This method is useful for various VR use cases because it provides feedback without disturbing users' movement. In our experiment, we presented haptic feedback of sine waves which were dynamically modulated according to direction and distance between a player and a target. These waves were presented to both sides of the users' necks independently. As a result, players could reach invisible targets and immediately know they had reached the targets. The proposed algorithm allows the neck to become as important a receptive part of body as eyes, ears, and hands.

High resolution facial capture has received significant attention in computer graphics due to its application in the creation of photorealistic digital humans for various applications ranging from film and VFX to games and VR. Here, the state of the art method for high quality acquisition of facial geometry and reflectance employs polarized spherical gradient illumination [Ghosh et al. 2011; Ma et al. 2007]. The technique has had a significant impact in facial capture for film VFX, recently receiving a Technical Achievement award from the Academy of Motion Picture Arts and Sciences [Aca 2019]. However, the method imposes a few constraints due to the employment of polarized illumination, and requires the camera viewpoints to be located close to the equator of the LED sphere for appropriate diffuse-specular separation for multiview capture [Ghosh et al. 2011]. The employment of polarization for reflectance separation also reduces the amount of light available for exposures and requires double the number of photographs (in cross and parallel polarization states), increasing the capture time and the number of photographs required for each face scan.

We propose an effective method to solve multiple characters audio-driven facial animation (ADFA) problem in an end-to-end fashion via deep neural network. In this paper each character's ADFA considered as a single task, and our goal is to solve ADFA problem in multi-task setting. To this end, we present MulTaNet for multi-task audio-driven facial animation (MTADFA), which learns a cross-task unified feature mapping from audio-to-vertex that capture shared information across multiple related tasks, while try to find within-task prediction network encoding character-dependent topological information. Extensive experiments indicate that MulTaNet generates more natural-looking and stable facial animation, meanwhile shows better generalization capacity to unseen languages compare to previous approaches.

Meet in Rain is a serious game on Chinese poetry. While invoking various events, players must complete given tasks, which help them better appreciate the poems, by exploring imaginary sceneries that depict Chinese poems. Its visual design also mimics Chinese paintings in the era when the poems were created. As only a few serious games exist for Chinese poetry and they mostly focus on knowledge acquisition, our work provides a rare design exemplar of a serious game that is designed with the intention to foster aesthetic appreciation for a cultural subject.

We propose MagniFinger, a fingertip-worn microscopy device that augments the limited abilities of human visual and tactile sensory systems in micrometer-scale environments. MagniFinger makes use of the finger's dexterous motor skills to achieve precise and intuitive control while allowing the user to observe the desired position simply by placing a fingertip. To implement the fingertip-sized device and its tactile display, we have built a system comprising a ball lens, an image sensor, and a thin piezoelectric actuator. Vibration-based tactile feedback is displayed based on the luminance of a magnified image, providing the user with the feeling of touching the magnified world.

This study proposes a tabletop projection device that can be implemented by combining physical objects with interactive projections. Users can interact on kraft papers using daily tools, such as marker pens, toothbrushes, colored blocks, and square wooden blocks. The input of the proposed device is a multifunction sensor, and the output is a tabletop projector. Using MagicPAPER, four types of interactions are implemented, namely drawing, gesture recognition, brushing, and building blocks. The abstract and poster discuss the design motivations and system descriptions of MagicPAPER.

There is increasing interest in using robots in simulation to understand and improve human-robot interaction (HRI). At the same time, the use of simulated settings to gather training data promises to help address a major data bottleneck in allowing robots to take advantage of powerful machine learning approaches. In this paper, we describe a prototype system that combines the robot operating system (ROS), the simulator Gazebo, and the Unity game engine to create human-robot interaction scenarios. A person can engage with the scenario using a monitor wall, allowing simultaneous collection of realistic sensor data and traces of human actions.

Given a point set S ⊆ R2, reconstruction refers to the process of identifying a vector shape that best approximates the input. Although this field was pioneered since 1983 by Edelsbrunner [Edelsbrunner et al. 2006] and has been heavily studied since then, the general problem still remains open, ill-posed and challenging. Solving it is essential for a wide range of applications, from image processing, pattern recognition and sketching to wireless networks.

The authors developed a VR orchestral application for interactive music experience, allowing virtual musical instruments in an orchestral piece to be repositioned spatially, dynamically and interactively in VR space. This can be done for changing environments where 3D audio technology is used to restructure traditional orchestral pieces into a new music art form. User experience surveys were undertaken on two kinds of users, with the results showing that the VR orchestral system developed in this paper could bring some special advantages in the musical experience.

Cinematic scientific visualizations turn complex scientific phenomena and concepts into stunning graphics and make them easier for the general public to comprehend. Adding interactivity to cinematic scientific visualizations is highly beneficial especially for educational purposes, as it keeps the viewers engaged and promotes active learning [Cano et al. 2017]. Although there are existing software tools such as VisIt that are capable of handling large data sets and allow for interactive exploration, they are usually designed for scientists and not meant for producing cinematic visualizations for the general public. Creating aesthetically pleasing visualizations of scientific data helps to better communicate the scientific concepts, increase impact, and reach a broader audience [Borkiewicz et al. 2018]. As existing examples of visualizations that are both interactive and cinematic have mainly been produced with custom software, there is a lack of easily accessible tools for developing this type of scientific visualization.

We present a collaborative immersive technology effort, InNervate AR and InNervate VR. These applications meet the need to expand on existing anatomy education platforms by implementing a more dynamic and interactive user interface. This user interface allows for exploration of the complex relationship between motor nerve deficits and their effects upon the canine anatomy's ability to produce movement. Preliminary AR user studies provided us with positive feedback in the quality of learning. The studies show that the dynamic touch interactions in AR definitely benefit students' critical reasoning and spatial visualization in learning motor nerve and muscle relationships. However, users seek a more immersive VR-based learning environment, without the distractions that an AR experience may offer. Based on this feedback, a VR version of this learning experience was created. Preliminary responses show that users are satisfied with this VR environment which allows them to manipulate and control the anatomical content with full-body interactions.