SIGGRAPH 2020 论文列表

1 EVALUATION OF A VIRTUAL REALITY-BASED BUFFET TO ADDRESS CHALLENGES IN HEALTH RESEARCH AND PRACTICE The Immersive Virtual Environment Testing Area (IVETA) is a behavioral science laboratory offering tools and expert support for virtual reality (VR)-based health research at the National Institutes of Health. We design VR tools to tackle methodological and communication challenges in the domain of health research and practice, with a specialty in genomics. One of our most useful VR tools is a custom virtual reality application intended to simulate a buffet food choice experience in a typical cafeteria setting. Food choice practices are a crucial research topic, but it remains difficult to assess as few methods exist to measure choices and their nutritional content accurately in real-time. The simulation is typically used to conduct experiments on child feeding, involving parents of children between the ages of 3 and 7. Participants walk freely around the buffet, navigating food choices in an environment designed specifically to fit in the bounds of our lab space. The VR buffet was designed initially for legacy virtual reality hardware but has since been through several iterations and is primarily to be used with the readily available HTC Vive system. Given that most of our study participants lacked any experience with virtual reality, and our goal was to simulate a familiar, real world activity, important design decisions for ease of use needed to be made from the start. Therefore, the input mapping for the Vive controllers was made to be dead simple: with a single click, participants could either add or subtract one portion of food to or from their plate.

We present a novel method to construct compatible surface meshes with bounded approximation errors. Given two oriented and topologically equivalent surfaces and a sparse set of corresponding landmarks, our method contains two steps: (1) generate compatible meshes with bounded approximation errors and (2) reduce mesh complexity while ensuring that approximation errors are always bounded. Central to the first step is a parameterization-based remeshing technique, which is capable of isotropically remeshing the input surfaces to be compatible and error-bounded. By iteratively performing a novel edge-based compatible remeshing and increasing the compatible target edge lengths, the second step effectively reduces mesh complexity while explicitly maintaining compatibility, regularity, and bounding approximation errors. Tests on various pairs of complex models demonstrate the efficacy and practicability of our method for constructing high-quality compatible meshes with bounded approximation errors.

Lighting has been used to enhance emotion and appeal of characters for centuries, from paintings in the Renaissance to the modern-day digital arts. In VFX and animation studios, lighting is considered as important as modelling, shading, or rigging. Most existing work focuses on either empirical best-practice created by artists of the centuries or on lighting perception with basic shapes. In contrast, our work focuses on the effect of lighting on emotional characters. Our study presents an extensive set of novel perceptual experiments designed to investigate the effects of brightness levels (key light brightness) and the proportion of light intensity illuminating the two sides of a character’s face (key-to-fill ratio). We are particularly interested in the effect of lighting on the recognition of emotion, emotion intensity, and the overall appeal, as these are crucial factors for audience engagement. Our results have implications for artists and developers wishing to increase the appeal and emotional expression of their characters, ranging from cartoon to realistic styles. Our key finding is that lighting can be used to effectively alter the intensity of emotion of a character and that brighter conditions increased appeal across all of our experiments.

Single Photon Avalanche Photodiodes (SPADs) have recently received a lot of attention in imaging and vision applications due to their excellent performance in low-light conditions, as well as their ultra-high temporal resolution. Unfortunately, like many evolving sensor technologies, image sensors built around SPAD technology currently suffer from a low pixel count. In this work, we investigate a simple, low-cost, and compact optical coding camera design that supports high-resolution image reconstructions from raw measurements with low pixel counts. We demonstrate this approach for regular intensity imaging, depth imaging, as well transient imaging. Our method uses an end-to-end framework to simultaneously optimize the optical design and a reconstruction network for obtaining super-resolved images from raw measurements. The optical design space is that of an engineered point spread function (implemented with diffractive optics), which can be considered an optimized anti-aliasing filter to preserve as much high-resolution information as possible despite imaging with a low pixel count, low fill-factor SPAD array. We further investigate a deep network for reconstruction. The effectiveness of this joint design and reconstruction approach is demonstrated for a range of different applications, including high-speed imaging, and time of flight depth imaging, as well as transient imaging. While our work specifically focuses on low-resolution SPAD sensors, similar approaches should prove effective for other emerging image sensor technologies with low pixel counts and low fill-factors.

We propose a novel method to efficiently compute bijective parameterizations with low distortion on disk topology meshes. Our method relies on a second-order solver. To design an efficient solver, we develop two key techniques. First, we propose a coarse shell to substantially reduce the number of collision constraints that are used to guarantee overlap-free boundaries. During the optimization process, the shell ensures the Hessian matrix with a fixed nonzero structure and a low density, thereby significantly accelerating the optimization. The second is a triangle inequality-based barrier function that effectively ensures non-intersecting boundaries. Our barrier function is C∞ inside the locally supported region and its convex second-order approximation is able to be analytically obtained. Compared to state-of-the-art methods for optimizing bijective parameterizations, our method exhibits better scalability and is about six times faster. The performance of our bijective parameterization algorithm is comparable to state-of-the-art methods of locally flip-free parameterizations. A large number of experimental results have shown the capability and feasibility of our method.

In this paper we propose a fully automatic method for shape correspondence that is widely applicable, and especially effective for non isometric shapes and shapes of different topology. We observe that fully-automatic shape correspondence can be decomposed as a hybrid discrete/continuous optimization problem, and we find the best sparse landmark correspondence, whose sparse-to-dense extension minimizes a local metric distortion. To tackle the combinatorial task of landmark correspondence we use an evolutionary genetic algorithm, where the local distortion of the sparse-to-dense extension is used as the objective function. We design novel geometrically guided genetic operators, which, when combined with our objective, are highly effective for non isometric shape matching. Our method outperforms state of the art methods for automatic shape correspondence both quantitatively and qualitatively on challenging datasets.

Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices. While hand-designed features on point clouds have long been proposed in graphics and vision, however, the recent overwhelming success of convolutional neural networks (CNNs) for image analysis suggests the value of adapting insight from CNN to the point cloud world. Point clouds inherently lack topological information, so designing a model to recover topology can enrich the representation power of point clouds. To this end, we propose a new neural network module dubbed EdgeConv suitable for CNN-based high-level tasks on point clouds, including classification and segmentation. EdgeConv acts on graphs dynamically computed in each layer of the network. It is differentiable and can be plugged into existing architectures. Compared to existing modules operating in extrinsic space or treating each point independently, EdgeConv has several appealing properties: It incorporates local neighborhood information; it can be stacked applied to learn global shape properties; and in multi-layer systems affinity in feature space captures semantic characteristics over potentially long distances in the original embedding. We show the performance of our model on standard benchmarks, including ModelNet40, ShapeNetPart, and S3DIS.

Figure 1: Left: Orientation-based redirection in Virtual Reality (VR) using dynamic-foveated rendering which leverages the effect of inattentional blindness induced by a cognitive task. The blending (e.g. parallax of green alien) in the transition zone between the rotated virtual environment (VE) (non-foveal) and the non-rotated VE (foveal) is imperceptible by the users due to inattentional blindness. Angular gain of the non-foveal zone for the frame shown is at its maximum i.e. 13.5°. Right: The path walked in physical tracked space (PTS) up to that point is shown in orange color. The corresponding path in the VE is shown in blue color. The cyan-colored box indicates the 4 × 4m2 of available PTS and the camera icon inside the box indicates the location of the user w.r.t. the PTS. Even with room-scale PTS such as the one used, the users were able to walk distances in the virtual environment (VE) which are up to almost 18 times higher than the longest distance in the PTS; maximum recorded in experiments was 103.9m.

Geometry processing of surface meshes relies heavily on the discretization of differential operators such as gradient, Laplacian, and covariant derivative. While a variety of discrete operators over triangulated meshes have been developed and used for decades, a similar construction over polygonal meshes remains far less explored despite the prevalence of non-simplicial surfaces in geometric design and engineering applications. This paper introduces a principled construction of discrete differential operators on surface meshes formed by (possibly non-flat and non-convex) polygonal faces. Our approach is based on a novel mimetic discretization of the gradient operator that is linear-precise on arbitrary polygons. Equipped with this discrete gradient, we draw upon ideas from the Virtual Element Method in order to derive a series of discrete operators commonly used in graphics that are now valid over polygonal surfaces. We demonstrate the accuracy and robustness of our resulting operators through various numerical examples, before incorporating them into existing geometry processing algorithms.

Common acoustic sources, like voices or musical instruments, exhibit strong frequency and directional dependence. When transported through complex environments, their anisotropic radiated field undergoes scattering, diffraction, and occlusion before reaching a directionally-sensitive listener. We present the first wave-based interactive auralization system that encodes and renders a complete reciprocal description of acoustic wave fields in general scenes. Our method renders directional effects at freely moving and rotating sources and listeners and supports any tabulated source directivity function and head-related transfer function. We represent a static scene's global acoustic transfer as an 11-dimensional bidirectional impulse response (BIR) field, which we extract from a set of wave simulations. We parametrically encode the BIR as a pair of radiating and arriving directions for the perceptually-salient initial (direct) response, and a compact 6 × 6 reflections transfer matrix capturing indirect energy transfer with scene-dependent anisotropy. We render our encoded data with an efficient and scalable algorithm - integrated in the Unreal Engine™ - whose CPU performance is agnostic to scene complexity and angular source/listener resolutions. We demonstrate convincing effects that depend on detailed scene geometry, for a variety of environments and source types.

Children's National Hospital is a leading pediatric teaching hospital with medical students, residency programs, fellowships, and research initiatives. Children's National uses online e-learning in its training, including interactive courses with 3D animated virtual simulations. Infants and young children can be hard to diagnose since they cannot easily tell doctors where it hurts or what they are feeling. Thorough diagnosing is important because diseases may often resemble other issues. Medical trainees of all levels require practice in the process of forming a diagnosis through organized steps in information gathering via medical history, lab results, and physical exam findings. The creation of educational videos of ill infants pose unique problems since children cannot self-consent, have rapid decompensations from minute to minute, and video capture can put the infant at risk. For example, a videographer would need to be in close proximity to a small child and could lead to increased exposures or prevent the care team from giving timely care. In addition, children often present with only one or two findings and these expand with time. As we work to ensure diversity in education and learning, the ability to create cases with diverse patient populations is important. While in a clinic setting, it is unethical to "allow" disease progression. Using 3D virtual simulation, the diagnostic process can be taught without risk to the patient.

This work concerns the computation and approximation of developable surfaces --- surfaces that are locally isometric to the two-dimensional plane. These surfaces are heavily studied in differential geometry, and are also of great interest to fabrication, architecture and fashion. We focus specifically on developability of heightfields. Our main observation is that developability can be cast as a rank constraint, which can then be plugged into theoretically-grounded rank-minimization techniques from the field of compressed sensing. This leads to a convex semidefinite optimization problem, which receives an input heightfield and recovers a similar heightfield which is developable. Due to the sparsifying nature of compressed sensing, the recovered surface is piecewise developable, with creases emerging between connected developable pieces. The convex program includes one user-specified parameter, balancing adherence to the original surface with developability and number of patches. We moreover show, that in contrast to previous techniques, our discretization does not introduce a bias and the same results are achieved across resolutions and orientations, and with no limit on the number of creases and patches. We solve this convex semidefinite optimization problem efficiently, by devising a tailor-made ADMM solver which leverages matrix-projection observations unique to our problem. We employ our method on a plethora of experiments, from denoising 3D scans of developable geometry such as documents and buildings, through approximating general heightfields with developable ones, and up to interpolating sparse annotations with a developable heightfield.

Designing robust mutation strategies for primary sample space Metropolis light transport is a challenging problem: poorly tuned mutations both hinder state space exploration and introduce structured image artifacts. Scenes with complex materials, lighting, and geometry make hand-designing strategies that remain optimal over the entire state space infeasible. Moreover, these difficult regions are often sparse in state space, and so relying exclusively on intricate—and often expensive—proposal mechanisms can be wasteful, whereas simpler inexpensive mechanisms are more sample efficient. We generalize Metropolis–Hastings light transport to employ a flexible two-stage mutation strategy based on delayed rejection Markov chain Monte Carlo. Our approach generates multiple proposals based on the failure of previous ones, all while preserving Markov chain ergodicity. This allows us to reduce error while maintaining fast global exploration and low correlation across chains. Direct application of delayed rejection to light transport leads to low acceptance probabilities, and so we also propose a novel transition kernel to alleviate this issue. We benchmark our approach on several applications including bold-then-timid and cheap-then-expensive proposals across different light transport algorithms. Our method is applicable to any primary sample space algorithm with minimal implementation effort, producing consistently better results on a variety of challenging scenes.

Filipe Antonio de Barros Reis Dept. of Computer Engineering and Industrial Automation School of Electrical and Computer Engineering University of Campinas (UNICAMP) Campinas, Brazil Paula Dornhofer Paro Costa Dept. of Computer Engineering and Industrial Automation School of Electrical and Computer Engineering University of Campinas (UNICAMP) Campinas, Brazil José Mario de Martino Dept. of Computer Engineering and Industrial Automation School of Electrical and Computer Engineering University of Campinas (UNICAMP) Campinas, Brazil

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.

Recently, deep generative adversarial networks for image generation have advanced rapidly; yet, only a small amount of research has focused on generative models for irregular structures, particularly meshes. Nonetheless, mesh generation and synthesis remains a fundamental topic in computer graphics. In this work, we propose a novel framework for synthesizing geometric textures. It learns geometric texture statistics from local neighborhoods (i.e., local triangular patches) of a single reference 3D model. It learns deep features on the faces of the input triangulation, which is used to subdivide and generate offsets across multiple scales, without parameterization of the reference or target mesh. Our network displaces mesh vertices in any direction (i.e., in the normal and tangential direction), enabling synthesis of geometric textures, which cannot be expressed by a simple 2D displacement map. Learning and synthesizing on local geometric patches enables a genus-oblivious framework, facilitating texture transfer between shapes of different genus.

We present a deep generative scene modeling technique for indoor environments. Our goal is to train a generative model using a feed-forward neural network that maps a prior distribution (e.g., a normal distribution) to the distribution of primary objects in indoor scenes. We introduce a 3D object arrangement representation that models the locations and orientations of objects, based on their size and shape attributes. Moreover, our scene representation is applicable for 3D objects with different multiplicities (repetition counts), selected from a database. We show a principled way to train this model by combining discriminative losses for both a 3D object arrangement representation and a 2D image-based representation. We demonstrate the effectiveness of our scene representation and the network training method on benchmark datasets. We also show the applications of this generative model in scene interpolation and scene completion.