NeurIPS 2026 论文列表

While equivariant architectures are standard for processing symmetric data, there is growing interest in achieving equivariance by applying group averaging or canonization to non-equivariant backbones. However, the theoretical generalization properties of these alternative strategies remain poorly understood. We introduce a theoretical framework to analyze the generalization error of these methods by bounding their covering numbers. We establish a rigorous generalization hierarchy: the error bounds of canonized models are at best equal to the error bounds of structurally equivariant and group-averaged models, and at worst equal to the bounds of non-equivariant baselines. Furthermore, we show that there exist "optimal" canonizations which attain the optimal error bounds, and "poor" canonizations which attain the non-equivariant error bounds, and that this depends on the regularity of the canonization. Finally, applying this framework to permutation groups in point cloud processing, we rigorously prove that the covering number of lexicographical sorting grows exponentially with point cloud dimension, whereas Hilbert curve canonization guarantees polynomial growth. This provides the first formal theoretical justification for the empirical success of Hilbert curve serialization in state-of-the-art point cloud architectures. We conclude with experiments which support our theoretical claims.

Room acoustics modeling requires capturing intricate wave phenomena beyond direct sound propagation, including reflections, refractions, and diffractions. Recent neural acoustic synthesis methods have progressively incorporated richer scene representations and physics-informed priors, yet typically learn acoustic behavior without considering wave propagation dynamics, which inherently capture diffraction and interference phenomena. We propose WavNAF, a framework that bridges wave dynamics and neural acoustic modeling by leveraging Finite-Difference Time-Domain (FDTD) simulations, which solve the wave equation on discrete spatiotemporal grids to capture these phenomena. However, directly applying FDTD at full scale is computationally prohibitive under CFL stability constraints. Inspired by traditional acoustic scale models, WavNAF constructs a geometry-preserving, scaled digital replica from NeRF-derived scene geometry and acoustically probes it with temporally compressed FDTD simulation. The resulting pressure maps are encoded not as direct full-scale Room Impulse Response (RIR) estimates, but as compressed-domain wave signatures that provide the neural acoustic field with a structured inductive bias beyond static geometry. To bridge the mismatch between these scaled simulation features and full-scale acoustic responses, we introduce a Neural Acoustic Scaling Module that adaptively recalibrates pressure-map features at the feature level for full-scale RIR prediction. Experiments show that WavNAF yields consistent gains over prior neural baselines across standard acoustic metrics.

This paper introduces Walk3D, a novel 3D Gaussian Splatting (3DGS) framework explicitly designed for 3D referring segmentation, particularly excelling at interpreting complex, long-sentence descriptions. Our primary motivation stems from observing that existing 3DGS-based open-vocabulary methods mainly focus on simple, category-level object segmentation. These methods struggle to precisely segment targets described by highly detailed, multi-sentence queries due to weak compositional language understanding and a lack of structural scene awareness. To tackle these dense and complex linguistic constraints, we propose a spatial routing network that decodes these queries into sequential chain-of-thought instructions upon a scene graph. Specifically, we construct a static supernode graph based on the geometric and semantic properties of the 3DGS space, where a spatial Graph Neural Network (GNN) dynamically pathfinds using the decomposed linguistic chain to accurately segment the referred target. Extensive experiments on complex 3D referring segmentation benchmarks demonstrate the superiority of our proposed method in interpreting lengthy, multi-conditional queries. Our code and trained models will be publicly released.

Recent advances in brain decoding have achieved impressive visual reconstruction under subject-specific settings, yet they fail to generalize to unseen subjects without retraining---hindering practical zero-shot cross-subject applications. The core challenges stem from inter-subject variability in voxel dimensionality and misaligned spatial response patterns. While existing methods rely on surface-based alignment or complex feature disentanglement, they often operate as black boxes and overlook the potential of volume-based fMRI data. To address this, we propose \textbf{Voxel as Token (VoxTok)}, a conceptual framework that treats each fMRI voxel as an independent token endowed with a learnable functional embedding. This formulation naturally handles variable-length inputs and subsumes existing adapter-based models as special cases where linear projections approximate voxel functionality. Guided by the hypothesis that large-scale spatial distributions of neural activity are consistent across subjects despite fine-grained variability, we introduce a KNN-based estimator to predict functional embeddings for unseen subjects. This approach achieves state-of-the-art zero-shot retrieval performance and comparable reconstruction quality, while enabling plug-and-play adaptation for existing models (e.g., MindEye2). Integrating multi-scale ROIs further boosts performance, with our best model achieving $40.4\%$ zero-shot image retrieval accuracy, rivaling early supervised methods. Crucially, we find that decoding success correlates strongly with ROI size, indicating that global co-activation patterns drive cross-subject generalization.

Reconstructing the complete geometry of a scene from a single RGB image remains challenging—especially when inferring hidden structures where visual evidence is incomplete. We introduce VolFill, a generative framework that predicts the 3D structure of the complete scene rather than relying on traditional pixel-aligned regression. Our method utilizes a hybrid 3D VAE to compress sparse truncated unsigned distance function grids into a compact latent space, paired with a latent Diffusion Transformer that denoises this representation to recover the complete scene. We condition the generation on geometry foundation models, leveraging rich spatial priors for robust reasoning. Unlike existing methods limited by per-ray constraints or unstructured point-cloud queries, VolFill provides a structured representation that supports direct surface extraction and occupancy queries at scale. Extensive experiments on the 3D-FRONT and NRGBD datasets demonstrate that our approach significantly outperforms current baselines, providing a robust foundation for holistic spatial understanding.

Procedural generation is well-suited for modeling 3D buildings, as architectural structures are naturally composed of repetitive and parameterized components. It provides a compact and editable representation by encoding geometry through executable rules. However, constructing such procedural representations typically relies on manually designed grammars, making them difficult to obtain in practice. We study image-based procedural facade reconstruction, where the goal is to recover an editable building procedural representation from an in-the-wild building image. We propose a hierarchical neuro-symbolic framework that decomposes the reconstruction into two coupled levels: a global structure layout tree for split-and-repeat facade organization, and a component parameterization representation for local architectural. A fine-tuned layout model recovers and refines the global structure through rendered visual feedback, while a tile-level predictor maps representative component crops to structured procedural attributes. The two representations are then deterministically compiled into executable CGA programs. Experiments show that our method achieves strong performance on both regular facades and challenging in-the-wild images, while maintaining high structural consistency and procedural executability. In addition, the tile-level predictor improves fine-grained architectural detail, enabling accurate and controllable component synthesis.

Embodied agents are expected to assist humans by actively exploring unknown environments and reasoning about spatial contexts. When deployed in real life, agents often face sequential tasks where each new task follows the completion of the previous one and may include infeasible objectives, such as searching for non-existent objects. However, most existing research focuses on isolated goals, overlooking the core challenge of sequential tasks: the ability to reuse spatial knowledge accumulated from previous explorations to guide subsequent reasoning and exploration. In this work, we investigate this underexplored yet practically significant embodied AI challenge. Specifically, we propose 3DSPMR, a 3D SPatial Memory Reasoning framework that utilizes Field-of-View (FoV) coverage as an explicit geometric prior. By integrating FoV-based constraints, 3DSPMR significantly enhances an agent’s memory, reasoning, and exploration capabilities across sequential tasks. To facilitate research in this area, we further introduce SEER-Bench, a novel Sequential Embodied Exploration and Reasoning Benchmark that spans two foundational tasks: Embodied Question Answering (EQA) and Embodied Multi-modal Navigation (EMN). SEER-Bench uniquely incorporates both feasible and infeasible tasks to provide a rigorous and comprehensive evaluation of agent performance. Extensive experiments verify that 3DSPMR achieves substantial performance gains on both sequential EQA and EMN tasks.

Figure-ground organization in the human visual system relies on several shape-based cues, including surroundedness, convexity, and symmetry. While these cues have been extensively studied using abstract stimuli, little is known about how they operate under natural conditions or how they arise from the statistics of natural scenes. Deep neural networks offer a promising path forward: a model that relies on the same figure-ground cues as humans would provide tractable experimental access to the underlying mechanisms. In this study, we evaluate shape-based figure-ground organization in Vision Transformers (ViTs), for which prior work has demonstrated the emergence of object-based grouping. We test 25 ViTs spanning supervised and self-supervised training objectives, by fitting linear probes to predict figure-ground assignment from intermediate patch representations using both natural images and controlled artificial stimuli that isolate individual cues. Our results show that ViTs robustly encode surroundedness and convexity, and that probes trained on natural images generalize zero-shot to artificial stimuli across several models. For symmetry we observe mixed results: the cue is encoded for uniformly colored but not for textured regions. Taken together, our findings demonstrate that Gestalt-like figure-ground cues can be learned from natural scene statistics and position ViTs as a compelling model system for studying the computational mechanisms of perceptual organization. All code and data will be published.

Modern visual backbones, despite their architectural diversity, largely reduce relational reasoning to pairwise message passing on graphs and therefore primarily capture 2-order interactions. However, visual understanding often depends on latent yet crucial higher-order semantic correlations among multiple regions, which cannot be adequately expressed by pairwise modeling alone. We first formalize this limitation by proving an irreducibility result: on a fixed vertex set, there exist nonlinear hypergraph message passing layers that cannot be represented exactly by any graph message passing layer unless the hypergraph satisfies highly restrictive structural conditions. This shows that higher-order correlation modeling offers a fundamental expressive advantage rather than a simple approximation to graph-based backbones. Therefore, we propose Vision Correlator (ViC), a general-purpose visual backbone built on multi-order hypergraphs. ViC consists of correlation induction and correlation propagation. In the correlation induction stage, we propose an Anchor-based Hypergraph Generation strategy that uses anchors as semantic centers and incorporates topological information to generate scalable hypergraph structures. In the correlation propagation stage, we further design Differential Mixed Aggregation to achieve efficient hyperedge representation and vertex feature updating, and introduce hyperedge dropout to improve the stability of structure learning. Extensive experiments with isotropic and pyramid variants show that our ViC achieves a better accuracy-efficiency trade-off than strong Transformer and graph-based baselines. Specifically, compared to the Vision Transformer baseline, ViC achieves up to 76\% parameter reduction, 93\% FLOPs reduction, and delivers up to a 5.1\% increase in accuracy on ImageNet.

Vision Transformers (ViTs) based vision foundation models (VFMs) have achieved remarkable performance across diverse vision tasks, but suffer from quadratic complexity that limits scalability to long sequences. Existing linear attention approaches for ViTs are typically trained from scratch, requiring substantial computational resources, while linearization-based methods developed for large language model decoders do not transfer well to ViTs. To address these challenges, we propose ViT-AdaLA, a novel framework for effectively adapting and transferring prior knowledge from VFMs to linear attention ViTs. ViT-AdaLA consists of three stages: attention alignment, feature alignment, and supervised fine-tuning. In the attention alignment stage, we align vanilla linear attention with the original softmax-based attention in each block to approximate the behavior of softmax attention. However, residual approximation errors inevitably accumulate across layers. We mitigate this by fine-tuning the linearized ViT to align its final-layer features with a frozen softmax VFM teacher. Finally, the adapted prior knowledge is transferred to downstream tasks through supervised fine-tuning. Extensive experiments on classification and segmentation tasks demonstrate the effectiveness and generality of ViT-AdaLA over various state-of-the-art linear attention counterpart.

Learning semantic representations for 3D Gaussian Splatting has recently emerged as a promising direction for 3D scene understanding. However, existing 2D-to-3D semantic lifting approaches rely primarily on 2D segmentation supervision and lack object-level priors, causing the learned representation to be driven by local mask evidence rather than holistic object structure. In this paper, we present VLSplat, a vision-language guided approach for object-centric semantic lifting in 3D Gaussian Splatting. Our method augments mask-based lifting with object-level semantic priors derived from vision-language models and refines Gaussian representations using an object-group scene graph. The refinement is performed through two language-guided modules: intra-group refinement suppresses semantic noise within object groups, while inter-group acquisition expands object support regions by incorporating structurally relevant candidates from neighboring groups. Extensive experiments on indoor scene datasets demonstrate that VLSplat improves semantic consistency and structural completeness. Our code and models are available at https://vlsplat.github.io/

Geometry estimation from perspective images has greatly advanced, maturing to the point where off-the-shelf foundation models are able to reconstruct 3D scene structure not only from multi-view imagery, but even from a single view. A natural extension is 3D reconstruction from panoramas, with the exciting prospect of recovering a full $360^\circ$ scene from a single panoramic image. In this work, we introduce PaGeR (Panoramic Geometry Reconstruction), a framework to lift powerful 3D foundation models, designed for perspective imagery, to the panorama domain. Our strategy is to start from a pretrained transformer for 3D reconstruction and turn it into a unified high-performance model that predicts scale-invariant depth, metric depth, sky masks and surface normals from both perspective and omnidirectional images, a single forward pass. By keeping architectural changes to a minimum and mixing perspective and panoramic images during training, Pager retains the rich 3D prior of the underlying foundation model while learning to also estimate geometrically consistent $360^\circ$ scenes from single panoramas. We extensively test our method in both indoor and outddor environments and find that it delivers state-of-the-art performance and excellent zero-shot performance across a wide range of scenes.

The efficiency and fidelity of 3D Gaussian Splatting critically depend on adaptive density control, which grows and suppresses Gaussian primitives throughout optimization. However, the standard pipeline still relies on heuristic view-space gradients, scale thresholds, and global opacity resets, rather than explicitly assessing the loss-reducing effect of each split, clone, or opacity update. This mismatch can allocate primitives to already-sufficient regions, miss under-reconstructed structures, and destabilize training through blanket opacity changes. In this work, we recast adaptive density control as a collection of local loss-reduction decisions. We derive efficient Taylor-based per-Gaussian surrogates that predict the objective change caused by three primitive-level operations: splitting, cloning, and opacity decay. This yields unified loss-aware framework with three components. First, we extend second-order splitting to a normalized joint position-scale space, enabling children to adapt not only their locations but also their anisotropic extents. Second, we formulate duplicate cloning as a surrogate minimization problem, selecting only Gaussians whose duplicated contribution is predicted to improve reconstruction and assigning each child a principled opacity. Third, we replace global opacity reset with selective soft opacity decay that weakens harmful primitives while preserving useful ones. Across standard 3DGS benchmarks, our method produces more targeted primitive allocation and achieves improved rendering quality with comparable or smaller Gaussian budgets compared to baselines.

We identify, for the first time, a new model modification attack—the cross-task model repurposing attack—that can render fingerprints generated by existing model fingerprinting approaches inapplicable. To address this challenge, we propose UniReFP—a novel unified model fingerprinting framework applicable to diverse vision models across different tasks—that utilizes ownership evidence in a shared class-level semantic space rather than original task-specific output space, enabling effective ownership verification when a protected model is repurposed across tasks with heterogeneous output formats. Specifically, UniReFP constructs a surrogate classifier by repurposing the protected model, enabling fingerprints to be generated independently of the original task-dependent output space of the protected model. Moreover, it optimizes grouped fingerprints to encode ownership evidence: fingerprints within the same group are encouraged to induce consistent semantic responses and intermediate representations on the surrogate classifier, while producing dispersed responses on independently trained reference classifiers. During verification, UniReFP uses a task-aware semantic projection mechanism to map heterogeneous suspect-model outputs into unified semantic-presence vectors, and determines ownership by measuring inner-group consistency. Extensive experimental results validate the effectiveness of UniReFP, consistently showing superior performance under cross-task repurposing attacks, commonly studied model modification attacks, and even their combinations.

In this paper, we propose UniGS, a unified map representation and differentiable framework for high-fidelity multimodal 3D reconstruction based on 3D Gaussian Splatting. Our framework integrates a CUDA-accelerated rasterization pipeline capable of rendering photo-realistic RGB images, geometrically accurate depth maps, consistent surface normals, and semantic logits simultaneously. We redesign the rasterization to render depth via differentiable ray-ellipsoid intersection rather than using Gaussian centers, enabling effective optimization of rotation and scale attribute through closed-form analytic depth gradients. Furthermore, we derive the analytic gradient formulation for surface normal rendering, ensuring geometric consistency among the reconstructed 3D scenes. To improve computational and storage efficiency, we introduce a learnable attribute that enables differentiable pruning of Gaussians with minimal contribution during training. Quantitative and qualitative experiments demonstrate the state-of-the-art reconstruction accuracy across several datasets, validating the efficacy and robust optimization stability of our geometry-aware paradigm.

Multi-view transformers solve 3D vision tasks in a single feed-forward pass, yet their internal mechanisms remain poorly understood. We view their residual layers as progressively updating a latent state, and study how this state evolves during 3D reconstruction. To make the state geometrically interpretable, we train layer-wise probes that decode intermediate tokens into 3D pointmaps, yielding intermediate 3D reconstructions throughout the network. This view recasts feed-forward 3D reconstruction as progressive refinement rather than a single reconstruction step, with most layers improving reconstruction. Convergence speed varies across models and sample difficulty, with slower convergence on difficult samples. We further analyze these updates and find consistent, complementary roles for single-view and cross-view attention. Cross-view attention improves multi-view alignment, while single-view attention refines intra-view geometry and compensates for points visible in only one view. Finally, we identify cross-view attention heads that specialize in correspondence matching. We show that their correspondences are jointly refined with the reconstructed geometry, and that selected heads can be partially replaced by analytical correspondences. Our analysis provides a direct view of the reconstruction process inside multi-view transformers, and suggests a first step toward replacing emergent mechanisms with analytical operations. The code used for the analysis will be made available.

We present UltraVoxelGS, a 3D ultrasound reconstruction method that achieves both fast reconstruction and physically faithful rendering, bringing live spatial feedback during 2D ultrasound scanning within reach. This is made possible by a feed-forward design that predicts a complete Gaussian scene representation from posed ultrasound images directly, entirely bypassing the per-scene optimization bottleneck that has confined prior methods to offline use. Realizing such feed-forward prediction in this setting is non-trivial: unlike natural images, ultrasound slices share at most one-dimensional intersections and exhibit view-dependent appearance due to acoustic propagation and attenuation, violating the dense overlap and photometric consistency assumed by existing feed-forward Gaussian methods. To achieve fast reconstruction, we introduce a feed-forward voxel-to-Gaussian pass that decouples representation size from input count by lifting posed slices into a fixed-size voxel support and predicting Gaussian parameters directly. For faithful rendering, we propose an ultrasound-aware appearance rendering strategy that factorizes reflection from cumulative attenuation during training and exploits appearance continuity among spatially adjacent slices for efficient Render-time Appearance Adaptation. On both real-world and simulated datasets, UltraVoxelGS achieves the best PSNR/SSIM compared with per-scene optimization baselines, while reducing inference from 10--20 minutes to approximately 10 seconds, over 60× faster, and scaling to sequences exceeding 1,000 frames.

Multi-view transformers for feed-forward 3D reconstruction must jointly solve two tasks. They need to extract precise per-view features and reason about cross-view geometric correspondences. Existing architectures apply cross-view attention throughout the network, leaving each layer to handle both objectives together. We revisit this design through a layer-wise probing analysis of feed-forward geometry transformers and observe that trained models tend to divide these two roles across layers. Early layers tend to specialize in per-frame feature extraction, while spatial cognition and cross-view reasoning emerge predominantly in later layers. This suggests that separating the two roles across the network can allocate compute more effectively. Building on this observation, we propose U-MVP, a U-shaped multi-view transformer for feed-forward 3D Gaussian Splatting that reflects this structure in the architecture itself. The encoder applies frame-wise attention to preserve per-view detail, the decoder introduces multi-view attention for cross-view geometric reasoning, and skip connections fuse the two streams so that appearance and geometry are recombined at decoding time. To scale to dense view regimes, we replace several global interaction blocks at the bottleneck with a grouping and swapping scheme that preserves cross-view information flow at lower cost. U-MVP performs competitively with feed-forward and optimization-based baselines from 16 to 256 views, generalizes to unseen datasets, and achieves strong results on 3D reconstruction and novel-view synthesis in posed and unposed settings.

Current video-to-4D methods struggle with complex topology changes, transparent materials, thin structures, and inner surfaces. We present Trellis4D, a dynamic mesh generation framework by inheriting the expressive representation of Trellis2, adapting it from image-to-3D to video-conditioned 4D generation. Our design arises from two key questions: (a) how to enable Trellis2's frame-local attention to share information across frames while preserving its pretrained quality on rare cases such as transparent objects and inner surfaces, and (b) how to inject temporal information into a purely 3D positional encoding without breaking pretrained capabilities. We address (a) with a sliding-window cross-frame attention and anchor on the first frame. The first frame is generated by the base Trellis2 model and injected into our model, letting it inherit Trellis2's quality in rare cases through cross-frame attention. We address (b) with a 4D temporal encoding that repurposes redundant low-frequency spatial RoPE bands for time, extending the encoding from 3D with no additional parameters. Extensive experiments show the effectiveness of Trellis4D for high-quality dynamic mesh generation on ActionBench and our own challenging complex dynamics set.