SIGGRAPH 2019 Papers

We proposed a detail refinement method to enhance the visual effect of turbulence in irrotational vortex. We restore the missing angular velocity from the particles and convert them into linear velocity to recover turbulent detail due to numerical disspation.

We present a variety of new compositing techniques using Multi-plane Images (MPI's) [Zhou et al. 2018] derived from footage shot with an inexpensive and portable light field video camera array. The effects include camera stabilization, foreground object removal, synthetic depth of field, and deep compositing. Traditional compositing is based around layering RGBA images to visually integrate elements into the same scene, and often requires manual 2D and/or 3D artist intervention to achieve realism in the presence of volumetric effects such as smoke or splashing water. We leverage the newly introduced DeepView solver [Flynn et al. 2019] and a light field camera array to generate MPIs stored in the DeepEXR format for compositing with realistic spatial integration and a simple workflow which offers new creative capabilities. We demonstrate using this technique by combining footage that would otherwise be very challenging and time intensive to achieve when using traditional techniques, with minimal artist intervention.

"Biodigital" is a Sci-Fi interactive story set in the year 2117 that combines VR film, immersive 3D environments, and VR data visualization. It turns data into a cinematic experience where a user is enmeshed as a character in the story. This VR storytelling tells the tale of humanity a hundred years from now. It also encourages us to think "How should we live in the future?"

We propose a system that automatically generates layouts for magazines that require graphical design. In this system, when images or texts are input as the content to be placed in layouts, an appropriate layout is automatically generated in consideration of content and design. The layout generation process is performed by randomized processing in accordance with a rule set of minimum conditions that must be satisfied for layouts (minimum condition rule set), where a large number of candidates are generated. The appearance, style, design, and composition of the work are evaluated by a learning-to-rank estimator, top scores are returned to the user. Users can greatly improve the efficiency of layout creation/editing by selecting from among automatically generated candidate layouts.

For most mammals and vertebrate animals, tail plays an important role for their body providing variant functions to expand their mobility, or as a limb that allows manipulation and gripping. In this work, Arque, we propose an artificial biomimicry-inspired anthropomorphic tail to allow us alter our body momentum for assistive, and haptic feedback applications. The proposed tail consists of adjacent joints with a spring-based structure to handle shearing and tangential forces, and allow managing the length and weight of the target tail. The internal structure of the tail is driven by four pneumatic artificial muscles providing the actuation mechanism for the tail tip. Here we highlight potential applications for using such prosthetic tail as an extension of human body to provide active momentum alteration in balancing situations, or as a device to alter body momentum for full-body haptic feedback scenarios.

In recent years, there has been a trend in teaching programming in elementary schools around the world. When teaching programming to lower grader students, robots and puzzles are often used to learn programming for easy understanding. However, those tools have limitations in execution results. Higher graders often use visual programming as a learning material. However, there is a problem that visual programming requires one computer for each individual student, and many students have to learn how to use a computer first. Therefore we propose a programming tool using tangible blocks and AR. This makes it possible to learn programming intuitively with fewer restrictions. Our tool is operated using a smartphone and tangible blocks without using a computer. By using AR, it is possible to create an intuitive programming that can interact with reality. We asked teachers who have experience teaching programming to children to assess the usefulness of our tool within programming education in school. As a result, there was an opinion that it might be suitable for multi-person programming.

In this article, we present a procedural approach to capture a variety of appearances of American Second Empire houses, which are well known for their mansard roofs and their inspired ornamentation. To develop this procedural approach, we have identified the set of rules and similarities of Second Empire houses. Our procedural approach, therefore, captures the style differences of Second Empire houses with a relatively few number of parameters. Using our interface, we are able to generate virtual houses in a wide variety of styles of American Second Empire architecture. We have also developed a method to break up these virtual models into slices in order to efficiently and economically 3D-print them. Using this method, we have printed miniatures of two landmark buildings in Savannah, GA and Baltimore, MD: The Hamilton-Turner Inn and Enoch Pratt House, respectively. We observe that the virtual models still provide more details because of the limited resolution of the 3D printing process as shown in Figures 1 and 2.

We propose a method for transforming inclined panoramic images into upright posture as if they were captured up straight. The method rectifies images automatically using neither additional information nor external instructions, while it enables panoramic images of both indoor and outdoor scenes to stand upright robustly. It helps unskilled people to make high quality panoramic images used for widespread applications including city navigation and real estate easier.

While there are many professional examples of successful character designs, there seems to be little academic formalization in standardizing a process to achieve consistent visual results. In this work, we present such a formal process to construct visual designs for character archetypes that are given by "verbal descriptions". This process is based on visual semiotics that are used for creating clear meaning behind design choices while still retaining a sense of aesthetic through principles of artistic design. Using this process, we have developed a set of encyclopedic references for a wide variety of psychology and literary archetypes to demonstrate the power of this approach. We also used this process successfully in a visual storytelling class.

Time-resolved imaging has made it possible to look around corners by exploiting information from diffuse light bounces. While there have been successive improvements in the field since its conception, so far it has only been proven to work in very simple and controlled scenarios. We present a public dataset of synthetic time-resolved Non-Line-of-Sight (NLOS) scenes with varied complexity aimed at benchmarking reconstructions. It includes scenes that are common in the real world but remain a challenge for NLOS reconstruction methods due to the ambiguous nature of higher-order diffuse bounces naturally occurring in them. With over 300 reconstructible scenes, the dataset contains an order of magnitude more scenes than what is available currently. The final objective of the dataset it to boost NLOS research to take it closer to its real-world applications.

Monte Carlo methods for transient rendering have become a powerful instrument to generate reliable data in transient imaging applications, either for benchmarking, analysis, or as a source for data-driven approaches. However, due to the increased dimensionality of time-resolved renders, storage and data bandwidth are significant limiting constraints, where a single time-resolved render of a scene can take several hundreds of megabytes. In this work we propose a learning-based approach that makes use of deep encoder-decoder architectures to learn lower-dimensional feature vectors of time-resolved pixels. We demonstrate how our method is capable of compressing transient renders up to a factor of 32, and recover the full transient profile making use of a decoder. Additionally, we show how our learned features significantly mitigate variance on the recovered signal, addressing one of the pathological problems in transient rendering.

This work presents a combination of 3D printing with mixed reality to use the results in the context of museum exhibitions or for cultural heritage. While now priceless artefacts are encased in glass, kept safe and out of reach of the visitors, we present a new pipeline which would allow visitors hands-on interaction with realistic 3D printed replicas of the artefacts which are then digitally augmented to have the genuine artefacts' appearances.

We propose a new aerial imaging display in which autostereoscopic objects with horizontal and vertical parallax appear as if they are floating in the air. This system operates by displaying an integral photography image in which depth is reversed beforehand and by observing the image from the other side of a micro mirror array.