paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Learning Physics-based Forward Model Corrections in Unrolled Networks for Diffuser-based Imaging

Yoko Sogabe, Shiori Sugimoto, Shoichiro Saito, Masaki Kitahara

Diffuser-based imaging systems replace conventional lenses with optical diffusers to enable compact lensless cameras, but their performance critically depends on computational reconstruction from compressed measurements. Deep unrolled networks have emerged as a leading framework by incorporating physical forward models into iterative optimization. However, existing methods typically rely on simplified assumptions, most notably shift-invariant point spread functions (PSFs), which fail to capture spatially-varying aberrations and geometric distortions in real hardware. In this paper, we propose learning physics-based forward model corrections for diffuser-based imaging. We construct a physically interpretable forward model with learnable correction terms based on Eigen-Kernel decomposition to handle spatially-varying aberrations. This design preserves the structure of the physical model while improving its fidelity to real measurements. The corrected forward model is integrated into an unrolled reconstruction framework (EK-DUN), leading to consistent improvements in reconstruction accuracy. Experimental results on real-world data demonstrate that our method achieves stateof-the-art performance compared to existing unrolled baselines.

ECCV 2026

Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching

Ruizhi Yang, Xingqiang Li, Jiajun Bai, Jinsong Du

Self-supervised stereo matching holds great promise by eliminating the reliance on expensive ground-truth data. Its dominant paradigm, based on photometric consistency, is however fundamentally hindered by the occlusion challenge—an issue that persists regardless of network architecture. The essential insight is that for any occluders, valid feedback signals can only be derived from the unoccluded areas on one side of the occluder. Existing methods attempt to address this by focusing on the erroneous feedback from the other side, either by identifying and removing it, or by introducing additional regularities for correction on that basis. Nevertheless, these approaches have failed to provide a complete solution. This work proposes a more fundamental solution. The core idea is to transform the fixed state of one-sided valid and one-sided erroneous signals into a probabilistic acquisition of valid feedback from both sides of an occluder. This is achieved through a complete framework, centered on a pseudo-stereo inputs strategy that decouples the input and feedback, without introducing any additional constraints. Qualitative results visually demonstrate that the occlusion problem is resolved, manifested by fully symmetrical and identical performance on both flanks of occluding objects. Quantitative experiments thoroughly validate the significant performance improvements resulting from solving the occlusion challenge. The code is available at https://github.com/qrzyang/PseudoStereo.

ECCV 2026

SlowBA: An efficiency backdoor attack towards VLM-based GUI agents

Junxian Li, Tu Lan, Haozhen Tan, Yan Meng, Haojin Zhu

Modern vision-language-model (VLM) based graphical user interface (GUI) agents are expected not only to execute actions accurately but also to respond to user instructions with low latency. While existing research on GUI-agent security mainly focuses on manipulating action correctness, the security risks related to response efficiency remain largely unexplored. In this paper, we introduce SlowBA, a novel backdoor attack that targets the responsiveness of VLM-based GUI agents. The key idea is to manipulate response latency by inducing excessively long reasoning chains under specific trigger patterns. To achieve this, we propose a two-stage reward-level backdoor injection (RBI) strategy that first aligns the long-response format and then learns trigger-aware activation through reinforcement learning. In addition, we design realistic pop-up windows as triggers that naturally appear in GUI environments, improving the stealthiness of the attack. Extensive experiments across multiple datasets and baselines demonstrate that SlowBA can significantly increase response length and latency while largely preserving task accuracy. The attack remains effective even with a small poisoning ratio and under several defense settings. These findings reveal a previously overlooked security vulnerability in GUI agents and highlight the need for defenses that consider both action correctness and response efficiency. Code can be found in https://github.com/tu-tuing/SlowBA.

ECCV 2026

PIAvatar: Physically Interactive Avatars via Deformation Gradient Decoupling

Sang-Hun Han, Min-Gyu Park, Jisu Shin, Seunghyun Shin, JINHWI PARK, Hae-Gon Jeon

3D human avatars have shown impressive visual fidelitydriven by pose-conditioned models, yet they still lack the physical abil-ity required for interactions with each other and environments. Althoughrecent studies have made various attempts to incorporate physical char-acteristics into 3D avatars, they only exhibit limited physical deforma-tions, often leading to constrained interaction behaviors. To resolve thisissue, we present PIAvatar, a framework to simultaneously enable physi-cally aware interactions between avatar-avatar and avatar-environment,and a non-rigid deformable human body simulation. In this work, ourkey insight is to decouple kinematic velocity from deformation gradi-ent. When external forces act on avatars, the kinematic velocity inducesstress which hinders the avatar’s ability to achieve a desired pose. Inaddition, we integrate a skeletal framework within the avatar. It allowsestimating its poses and real-time tracking in a closed form, even duringnon-rigid physical interactions. Our approach is implemented within aconventional Material Point Method framework to ensure physically con-sistent dynamics. We lastly evaluate the method on both human-objectand human-human interaction scenarios to assess its behavior under di-verse interaction settings.

ECCV 2026

Em-Garde: A Propose-Match Framework for Proactive Streaming Video Understanding

Yikai Zheng, Xin Ding, Yifan Yang, Shiqi Jiang, Hao Wu, Qianxi Zhang, Weijun Wang, Ting Cao, Yunxin Liu

Recent advances in Streaming Video Understanding has enabled a new interaction paradigm where models respond proactively to user queries. Current proactive VideoLLMs rely on per-frame triggering decision making, which suffers from an efficiency-accuracy dilemma. We propose Em-Garde, a novel framework that decouples semantic understanding from streaming perception. At query time, the InstructionGuided Proposal Parser transforms user queries into structured, perceptually grounded visual proposals; during streaming, a Lightweight Proposal Matching Module performs efficient embedding-based matching to trigger responses. Experiments on StreamingBench and OVO-Bench demonstrate consistent improvements over prior models in proactive response accuracy and efficiency, validating an effective solution for proactive video understanding under strict computational constraints. Code and model are available at https://air-embodied-brain.github.io/ Em-Garde/.

ECCV 2026

SparseCtrl-HOI: Sparse Temporal Control for Human-Object Interaction Video Generation

Shenbo Xie, Mingrui Cai, Xu Yang, Yifei Liu, Changxing Ding

Human-Object Interaction (HOI) video generation aims to synthesize realistic videos of humans manipulating diverse objects, serving as a promising avenue for AI-driven live streaming e-commerce. A primary obstacle in this domain lies in the complexity of modeling fine-grained physical dynamics and the intricate spatial-temporal coordination between human hands and objects. Existing approaches to this problem typically rely on dense temporal guidance, e.g., frame-wise hand-object pose sequences, to strictly control the interaction process. However, such dense guidance incurs high annotation costs and affects motion synthesis diversity. To overcome these limitations, we introduce SparseCtrl-HOI, a novel sparse temporal control framework for HOI video generation. It requires only a few keyframes that capture interaction states at designated timestamps. Specifically, we employ a TimeControlled Rotary Positional Embedding (TiRoPE) mechanism to temporally anchor these keyframes while preserving their spatial integrity. Subsequently, to govern the dynamics across intermediate frames, we propose a Motion Prior Injection Module that leverages Multimodal Large Language Models (MLLMs) to extract high-level motion priors. This empowers the model to hallucinate logically and physically plausible transitions. Furthermore, we build SparseHOI-5K, a high-quality and richly annotated dataset for HOI video generation with sparse temporal control. Comprehensive evaluations confirm that our method substantially reduces annotation overhead while synthesizing superior live-streaming e-commerce videos. Both our code and dataset are publicly available at https://mpi-lab.github.io/SparseCtrl-HOI.

ECCV 2026

SVCBench: A Streaming Video Counting Benchmark for Spatial-Temporal State Maintenance

Pengyiang Liu, Zhongyue Shi, Hongye Hao, Qi Fu, Xueting BI, Siwei Zhang, Xiaoyang Hu, Zitian Wang, Linjiang Huang, Si Liu

Video understanding requires models to continuously track and update world state during playback. Although existing benchmarks have advanced video understanding evaluation across multiple dimensions, they provide limited visibility into how models maintain world state over time. We propose SVCBench, a Streaming Video Counting Benchmark that repositions counting as a minimal, controlled probe for diagnosing models’ world-state maintenance capability. We decompose this capability into object counting and event counting, forming 8 fine-grained subcategories. Object counting covers tracking currently visible objects and cumulative unique identities, while event counting covers detecting instantaneous actions and tracking complete activity cycles. SVCBench contains 406 videos with frame-by-frame annotations of 10,071 event occurrences and object state changes, yielding 1,000 streaming QA pairs with 4,576 query points distributed along video timelines. By observing state maintenance trajectories through streaming multipoint queries, we design three complementary metrics to diagnose numerical precision, trajectory consistency, and temporal awareness. Evaluations of mainstream video-language models show that current models still exhibit significant deficiencies in spatial-temporal state maintenance, with especially poor performance on periodic event counting. SVCBench provides a diagnostic framework for measuring and improving state maintenance in video understanding systems. Our code and data are available at https://buaa-colalab.github.io/SVCBench.

ECCV 2026

CRISP: Calibration-Aware Visual State Space Duality for Remote Sensing Image Segmentation

Kangning Wang, Haopeng Zhang, Zhiguo Jiang

State space models, especially Visual State Space Duality (VSSD), have emerged as efficient linear-time alternatives to Transformers for dense visual tasks. However, we observe that VSSD compresses spatial context into a global aggregation that suppresses high-frequency responses, causing excessive boundary smoothing in remote sensing semantic segmentation. To address this, we propose CRISP, a calibration framework with two components. Its core, the Duality Calibration Operator (DCO), restores local contrast and boundary responses through residual injection and frequency calibration within the VSSD backbone, without altering its linear complexity. To retain the recovered detail, an Orthogonal Multi-Prototype (OMP) head assigns multiple orthogonally constrained prototypes per class to model large intra-class variance. Extensive experiments on Potsdam, Vaihingen, and LoveDA show that, with only ∼30M parameters, CRISP achieves consistent gains in mean F1 (mF) and mIoU while remaining competitive with state-of-the-art methods. Code is available at https://github.com/crazylifeha/crisp.

ECCV 2026

Quick ViTs: Speeding up Vision Transformers through Equivariance

David Nordström, Johan Edstedt, Fredrik Kahl, Georg Bökman*

Natural images exhibit strong geometric regularities: local structures, such as edges, corners, and textures, appear in many orientations and mirror configurations. Since Vision Transformers (ViTs) operate on square image patches, these transformations naturally correspond to the dihedral symmetry group D 8, also known as the octic group. Recent work has shown that ViTs can be made reflection equivariant and more e!cient than standard ViTs simultaneously by implementing the linear layers in the Fourier domain of the reflection group. In this work, we extend the equivariance to reflections and rotations and analyze the scalability of the resulting networks. Our Quick ViTs, based on octic equivariant linear layers, achieve 5.33x reductions in FLOPs and up to 8x reductions in memory compared to ordinary linear layers. By analyzing the arithmetic intensity of these layers, we identify theoretical limits on how much the FLOP savings translate into throughput improvements on modern GPUs. However, these limitations disappear as the embedding dimensions increase. Enabled by their computational e!ciency, we conduct a broader empirical evaluation of equivariant ViTs than in previous work. Upon training supervised (DeiT-III) and self-supervised (DINOv2) on ImageNet-1K, we find that our Quick ViTs match or exceed baseline accuracy while at the same time providing substantial e!ciency gains.

ECCV 2026

Unified Panoramic–Gaussian Representation for Monocular 4D Scene Synthesis

Yuankun Yang, Yi Wei, Wenyang Zhou, Li Zhang

4D scene synthesis from monocular videos has made significant progress in recent years. However, existing methods are typically constrained by view interpolation. As a result, they struggle to infer unseen regions beyond the observed views. In this paper, we reformulate the task as 4D scene synthesis with unseen regions, which extends beyond traditional interpolation settings. Camera-conditioned video generation enables unseen region synthesis by guiding generation along specified cameras. However, these methods lack explicit 3D priors and are optimized with random camera trajectories. This design leads to severe inconsistencies under large trajectory deviations. To address this limitation, we build a unified training and inference framework with panoramic trajectory guidance. While this design improves cross-view consistency, the panoramic representation alone fails to model dynamic content effectively. Object motion in panoramic space introduces scale and shape distortions. To address this, we propose PanoGaussian, a unified PanoramicGaussian representation that distills the panoramic representation into an explicit dynamic Gaussian representation to capture dynamic physical priors of the 4D scene. Experiments demonstrate that PanoGaussian achieves consistent 4D scene synthesis even under large viewpoint variations.

ECCV 2026

When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Models

Dasol Choi, Jihwan Lee, Minjae Lee, Minsuk Kahng

While prior research on text-to-image generation has predominantly focused on biases in human depictions, demographic bias in generated objects remains relatively underexplored. We introduce SODA (Stereotyped Object Diagnostic Audit)1, a novel framework for systematically measuring these biases through automated attribute discovery and three standardized metrics: Base vs. Demographic Divergence (BDS), Cross-Demographic Disparity (CDS), and Visual Attribute Concentration (VAC). Applying SODA to 8,000 images across five state-of-the-art models and eight object categories (e.g., cars), we find that “neutral” prompts produce outputs most visually similar to middle-aged and White people, suggesting these groups are implicitly over-represented in model defaults. Furthermore, demographic cues trigger highly skewed stereotypical outputs: 26.6% of object-model-demographic combinations produce results where all 20 generated images share the exact same attribute value (e.g., rose gold laptops for women). Finally, prompt-level debiasing reduces inter-group disparity but paradoxically collapses within-group diversity, replacing one stereotype with another. SODA offers a practical pipeline for making these implicit associations measurable, serving as a step toward more responsible AI development.

ECCV 2026

AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World

Zhongqiang Song, Guanying Chen, Yuqi Zhang, Yin Zou, Chuanyu Fu, Zhiyuan Yuan, Chuan Huang, Shuguang Cui, Xiaochun Cao

This paper addresses the problem of monocular metric depth estimation in aerial UAV imagery. Although recent data-driven methods have achieved remarkable progress in ground-level scenarios, models trained primarily on street-view and indoor datasets exhibit significant domain gaps when applied to aerial viewpoints. To tackle these challenges, we introduce AerialMetric, a benchmark dataset designed to evaluate and facilitate the adaptation of monocular metric depth estimation under UAV aerial viewpoints. The dataset consists of four complementary subsets collected from different sources, jointly covering real-world photogrammetry data, controlled aerial acquisition settings, photorealistic synthetic scenes, and in-the-wild Internet imagery. In total, AerialMetric provides 52K real-world and 16K synthetic image–depth pairs with reliable metric ground truth. Based on this dataset, we conduct systematic evaluations of existing state-of-the-art models under aerial settings and investigate the impact of viewpoint, altitude, and camera parameters on metric depth prediction. In addition, by fine-tuning representative metric depth model on our dataset, we establish a comprehensive aerial benchmark and achieve state-of-the-art performance across diverse aerial imagery. Our dataset, code, and model weight are publicly available at https://kuieless.github.io/AerialMetric-ECCV2026-page/.

ECCV 2026

OmniSch: A Multimodal PCB Schematic Benchmark For Structured Diagram Visual Reasoning

Taiting Lu, Kaiyuan Lin, Yuxin Tian, Yubo Wang, Muchuan Wang, Sharique Khatri, Akshit Kartik, Yixi Wang, Amey Rane, Yida Wang, Yifan Yang, Yi-Chao Chen, yincheng jin, Mahanth Gowda

Recent large multimodal models (LMMs) have made rapid progress on visual grounding, document understanding, and diagram reasoning tasks. However, their ability to convert Printed Circuit Board (PCB) schematic diagrams into machine-readable, spatially weighted netlist graphs that capture component attributes, connectivity, and geometry remains largely underexplored, even though such graph representations are the backbone of practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniSch, the first comprehensive benchmark designed to assess LMMs on schematic understanding and spatial netlist graph construction. OmniSch contains 1,854 real-world schematic diagrams and supports four tasks: (1) visual grounding for schematic entities, with 109.9K grounded instances aligning 423.4K diagram semantic labels to their visual regions; (2) diagram-to-graph reasoning, which requires understanding topological relationships among diagram elements; (3) geometric reasoning, constructing layout-dependent edge weights for each connection; and (4) tool-augmented agentic reasoning for visual search, which invokes external tools to accomplish (1) - (3). Our results reveal substantial gaps in current LMMs’ ability to interpret schematic engineering artifacts, including unreliable fine-grained grounding, brittle layout-to-graph parsing, inconsistent global connectivity reasoning, and inefficient visual exploration.

ECCV 2026

GeoV2V: Geometry-Grounded Video Diffusion Model for Driving Scene Generation

Yuchen Xi, tippy guo, Chenwei Hou, Zixu Liu, Jin Fang, Jason Liu, Ruigang Yang

Novel view synthesis under trajectory changes is essential for autonomous driving simulation, yet existing methods struggle to generate consistent videos when extrapolating to unseen viewpoints such as lane shifts. Reconstruction-based approaches maintain geometric consistency but degrade in visual quality under large viewpoint changes, while reconstruct-then-restore methods often produce visually plausible frames yet suffer from geometric and temporal inconsistencies. We present GeoV2V, a geometry-grounded video-to-video diffusion framework that synthesizes temporally coherent driving videos along novel trajectories by integrating LiDAR and depth-derived geometric priors with full-video conditioning in the Wan 2.1 backbone. A key challenge in learning cross-lane transformations is the lack of synchronized multitrajectory supervision in real-world datasets. To address this, we also introduce Para4D, a synthetic dataset capturing synchronized multi-lane driving videos within dynamic scenes, which provides explicit supervision for lane-shift generation and serves as a strong training prior for geometry-consistent view synthesis. Experiments on Waymo, nuScenes, and Para4D demonstrate state-of-the-art performance in visual quality, geometric consistency, and robustness to complex lighting dynamics under large trajectory shifts. Project page: https://xiyuche.github.io/ GeoV2V/

ECCV 2026

GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation

Nicolas von Lützow, Barbara Roessle, Katharina Schmid, Matthias Niessner

Most recent advances in 3D generative modeling rely on diffusion or flow-matching formulations. We instead explore a fully autoregressive alternative and introduce GaussianGPT, a transformer-based model that directly generates 3D Gaussians via next-token prediction, thus facilitating full 3D scene generation. We first compress Gaussian primitives into a discrete latent grid using a sparse 3D convolutional autoencoder with vector quantization. The resulting tokens are serialized and modeled using a causal transformer with 3D rotary positional embedding, enabling sequential generation of spatial structure and appearance. Unlike diffusion-based methods that refine scenes holistically, our formulation constructs scenes step-by-step, naturally supporting completion, outpainting, controllable sampling via temperature, and flexible generation horizons. This formulation leverages the compositional inductive biases and scalability of autoregressive modeling while operating on explicit representations compatible with modern neural rendering pipelines, positioning autoregressive transformers as a complementary paradigm for controllable and context-aware 3D generation.

ECCV 2026

RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video

Ulrich Prestel, Stefan Andreas Baumann, Nick Stracke, Bjorn Ommer

Self-supervised novel view synthesis (NVS) remains challenging to scale, despite the abundance of video data, largely due to the brittleness of training on realistic videos and the hard-to-predict scaling behavior of multi-network system designs. We introduce RayDer, a unified, feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone, turning self-supervised NVS into a well-posed single-model scaling problem. A minimal dynamic state, treated as a nuisance factor, absorbs time-varying content and enables stable training on unconstrained real-world video. Importantly, RayDer keeps static-scene NVS as its target task: dynamic content is leveraged purely as scalable supervision, not reconstructed as in dynamicscene (4D) NVS. Across multiple model sizes and orders of magnitude in data, RayDer exhibits clean power-law scaling with data and compute, and outperforms static-scene data mixtures. On a large number of benchmarks, RayDer achieves strong zero-shot open-set performance competitive with state-of-the-art supervised approaches. Project Page: https://compvis.github.io/rayder.

ECCV 2026

MemLearner: Learning to Query Context Memory for Video World Models

Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang, Quande Liu, Xintao Wang, Pengfei Wan, Kun Gai, Xihui Liu

Video World Models are interactive video generation modelsthat predict future world states based on user actions and history videoframes. A critical challenge in video world models is the lack of memory,causing inconsistent generated scenes over extended durations. Previousmethods explored rule-based context frame retrieval as memory, but theyfail to generalize in scenarios with scene occlusions and dynamic objects.We propose MemLearner, a learning-based adaptive context querymethod using query tokens to bridge context and predicted tokens. Byleveraging the video generation model itself for context querying, Mem-Learner exploits pre-trained visual priors without training additionalmodules from scratch, and incorporates e_x001E_cient strategies for trainingand inference. We collect a dataset of long videos with scene occlusionsand dynamic objects, paired with camera pose annotations, and proposea multi-dataset training strategy leveraging both annotated renderedand unannotated real-world videos. Extensive experiments demonstratethat MemLearner signi_x001C_cantly outperforms prior video world modelsin terms of scene consistency and memory, particularly under challengingocclusion and dynamic scenarios.

ECCV 2026

Seek to Segment: Active Perception for Panoramic Referring Segmentation

Song Tang, Shuming Hu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang

Existing referring image segmentation (RIS) models passively process static images captured from fixed perspectives, limiting their applicability in Embodied AI, where agents must perform active perception in the continuous 360◦ environments. To bridge this gap, we introduce a novel task: Active Panoramic Referring Segmentation (APRS). In this setting, an agent is required to adjust its viewing direction (∆θ,∆ϕ) to explore the 360◦ environment, seeking the object specified by a user instruction for segmentation. To tackle this challenging task, we propose PanoSeeker, a memory-augmented agent for efficient APRS. Rather than relying on heuristic scanning, PanoSeeker integrates a Vision-Language Model (VLM) with EgoSphere—an explicit spatial visual memory. By progressively integrating sequential local observations into a unified 360◦ representation, EgoSphere enables the agent to plan efficient and non-redundant search trajectories. Once the target is found, the agent performs active viewpoint alignment and outputs the segmentation mask. Furthermore, we curate an expert-annotated search trajectory dataset with memory timelines for Supervised FineTuning, followed by Reinforcement Learning post-training to explicitly optimize PanoSeeker’s exploration efficiency. Extensive experiments on our newly established APRS benchmark demonstrate that PanoSeeker achieves superior search efficiency and segmentation accuracy, significantly outperforming adapted state-of-the-art baselines.

ECCV 2026

Mode-Conditioned Residual Calibration for Multi-Object Tracking

Muyu Li, Henan Hu, Deepak Jain, Xudong Zhao

Multi-object tracking degrades in crowded scenes. The reliability of geometry and appearance shifts under occlusion, uniform appearance, and fast motion. Consequently, association scores suffer from mode-dependent miscalibration. We observe that these failures concentrate into a small set of recurring error modes. This pattern renders single global thresholds ineffective. To address this issue, we propose a diagnose-then-correct strategy via Mode-Conditioned Residual Calibration (MCRC). It is a lightweight auxiliary module that identifies the dominant failure mode from pairwise and set-level geometric cues. The module then applies a mode-conditioned residual correction to the association cost matrix. For highly ambiguous tracking pairs, MCRC selectively activates an uncertainty-gated semantic refiner. This refiner incorporates simple acceptance checks to ensure representation stability, thereby avoiding global computational overhead. Extensive evaluations on DanceTrack, SportsMOT, MOT17, and MOT20 confirm its effectiveness. MCRC consistently improves identity preservation when inserted into diverse trackers, maintaining minimal added overhead.

ECCV 2026

NeLU3D: Neural Inverse Structured Light without Modeling the Projector

Giancarlo Pereira, David Fouhey, Claudio Silva, Daniele Panozzo

Structured Light (SL) is a popular technique that delivers precise 3D shape acquisition across diverse materials and object sizes. SL requires, however, careful modeling of the set-up, including geometric and radiometric calibration of the camera-projector pair; without it, reconstruction quality degrades. We propose NeLU3D, a neural inverse SL method without explicitly modeling the projector. We calibrate the camera–projector system using a neural network that maps 3D positions to a set of projected patterns. Then, with as few as four monochromatic images (or two RGB images), our approach uses differentiable volume rendering to fit a surface to match SL captures. We scan over twenty-five objects of different shapes and reflectances to demonstrate the feasibility and quality of our method in a handful of projector-camera set-ups, including an extremely low-cost projector and an analog projector with fixed RGB pattern. We also showcase sub-millimeter accuracy with suboptimal patterns, where previous methods recover noisy 3D surfaces. We release an open-source implementation at https://github.com/ geometryprocessing/neural-lookup.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.