SIGGRAPH Asia 2023 Papers

A 3D digital scene contains many components: lights, materials and geometries, interacting to reach the desired appearance. Staging such a scene is time-consuming and requires both artistic and technical skills. In this work, we propose PSDR-Room, a system allowing to optimize lighting as well as the pose and materials of individual objects to match a target image of a room scene, with minimal user input. To this end, we leverage a recent path-space differentiable rendering approach that provides unbiased gradients of the rendering with respect to geometry, lighting, and procedural materials, allowing us to optimize all of these components using gradient descent to visually match the input photo appearance. We use recent single-image scene understanding methods to initialize the optimization and search for appropriate 3D models and materials. We evaluate our method on real photographs of indoor scenes and demonstrate the editability of the resulting scene components.

The kinematic motion of a robotic character is defined by its mechanical joints and actuators that restrict the relative motion of its rigid components. Designing robots that perform a given target motion as closely as possible with a fixed number of actuated degrees of freedom is challenging, especially for robots that form kinematic loops. In this paper, we propose a technique that simultaneously solves for optimal design and control parameters for a robotic character whose design is parameterized with configurable joints. At the technical core of our technique is an efficient solution strategy that uses dynamic programming to solve for optimal state, control, and design parameters, together with a strategy to remove redundant constraints that commonly exist in general robot assemblies with kinematic loops. We demonstrate the efficacy of our approach by either editing the design of an existing robotic character, or by optimizing the design of a new character to perform a desired motion.

Modeling human behaviors in contextual environments has a wide range of applications in character animation, embodied AI, VR/AR, and robotics. In real-world scenarios, humans frequently interact with the environment and manipulate various objects to complete daily tasks. In this work, we study the problem of full-body human motion synthesis for the manipulation of large-sized objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework that can generate full-body manipulation behaviors from only the object motion. Since naively applying diffusion models fails to precisely enforce contact constraints between the hands and the object, OMOMO learns two separate denoising processes to first predict hand positions from object motion and subsequently synthesize full-body poses based on the predicted hand positions. By employing the hand positions as an intermediate representation between the two denoising processes, we can explicitly enforce contact constraints, resulting in more physically plausible manipulation motions. With the learned model, we develop a novel system that captures full-body human manipulation motions by simply attaching a smartphone to the object being manipulated. Through extensive experiments, we demonstrate the effectiveness of our proposed pipeline and its ability to generalize to unseen objects. Additionally, as high-quality human-object interaction datasets are scarce, we collect a large-scale dataset consisting of 3D object geometry, object motion, and human motion. Our dataset contains human-object interaction motion for 15 objects, with a total duration of approximately 10 hours.

We estimate the three Herschel-Bulkley parameters (yield stress σY, power-law index n, and consistency parameter η) for shear-dependent fluid-like materials possibly with large-scale inclusions, for which rheometers may fail to provide a useful measurement. We perform experiments using the unknown material for dam-break (or column collapse) setups and capture video footage. We then use simulations to optimize for the material parameters. For better match up with the simple shear flow encountered in a rheometer, we modify the flow rule for the elasto-viscoplastic Herschel-Bulkley model. Analyzing the loss landscape for optimization, we realize a similarity relation; material parameters far away within this relation would result in matched simulations, making it hard to distinguish the parameters. We found that by exploiting the setup dependency of the similarity relation, we can improve on the estimation using multiple setups, which we propose by analyzing the Hessian of the similarity relation. We validate the efficacy of our method by comparing the estimations to the measurements from a rheometer (for materials without large-scale inclusions) and show applications to materials with large-scale inclusions, including various salad or pasta sauces, and congee.