NeurIPS 2026 Papers

Unsupervised learning in Vision Transformer models, such as DINO, enables robust visual perception without requiring labeled data by leveraging large-scale image datasets. However, these models lack a focusing mechanism during training, resulting in perceptual representations that are often ambiguous or overly similar. In this work, we introduce two visual focusing loss functions designed to establish correspondences between the model's attention and specific regions within images. Specifically, we leverage the multi-head attention mechanism within the model to selectively steer the model's focus, leading to the emergence of multiple, diverse, and focused perceptual capabilities without requiring supervision. Through both qualitative and quantitative evaluations, we demonstrate that our method substantially increases the spatial selectivity and diversity of attention heads, improving both explainability and fine-grained recognition performance.

Dense 3D tracking from monocular video is fundamental to dynamic scene understanding. While recent 3D foundation models provide reliable per-frame geometry, recovering object motion in this geometry remains challenging and benefits from strong motion priors learned from real-world videos. Existing 3D trackers either follow iterative paradigms trained from scratch on synthetic data or fine-tune 3D reconstruction models learned from static multi-view images, both lacking real-world motion priors. Pre-trained video diffusion transformers (video DiTs) offer rich spatio-temporal priors from internet-scale videos, making them a promising foundation for 3D tracking. However, their frame-anchored formulation, which generates each frame's content, is fundamentally mismatched with reference-anchored dense 3D tracking, which must follow the same physical points from a reference frame across time. We present TrackCraft3R, the first method to repurpose a video DiT as a feed-forward dense 3D tracker. Given a monocular video and its frame-anchored reconstruction pointmap, TrackCraft3R predicts a reference-anchored tracking pointmap that follows every pixel of the first frame across time in a single forward pass, along with its visibility. We achieve this through two designs: (i) a dual-latent representation that uses per-frame geometry latents and reference-anchored track latents as dense queries, and (ii) temporal RoPE alignment, which specifies the target timestamp of each track latent. Together, these designs convert the per-frame generative paradigm of video DiTs into a reference-anchored tracking formulation with LoRA fine-tuning. TrackCraft3R achieves state-of-the-art performance on standard sparse and dense 3D tracking benchmarks, while running 1.3x faster and using 4.6x less peak memory than the strongest prior method. We further demonstrate robustness to large motions and long videos. Our code and weights will be publicly released.

Internet-scale photo collections of real-world landmarks have driven progress in 3D computer vision yet remain highly challenging for modern 3D foundation models (3DFMs) that estimate scene structure in a single feed-forward pass. In this work, we introduce ArchWorld, a benchmark of 3D architectural landmarks with high geographic coverage and rich metadata and use it to conduct a thorough error analysis of current 3DFMs to understand where they fall short and how they can be improved. We examine socially relevant disparities in model error and find that model performance varies by geographic region, but this is heavily confounded by scene size. Leveraging insights from model failings, we introduce an intervention scheme that improves 3DFM performance while simultaneously reducing geographic disparities. Our code and data will be publicly available.

3D Gaussian Splatting (3DGS) has emerged as a mainstream representation for real-time novel view synthesis and 3D reconstruction. However, existing 3DGS rendering pipelines are poorly aligned with modern GPU architectures: depth-sorted alpha compositing introduces strong per-pixel sequential dependencies, limiting parallelism and resulting in scalar Fused Multiply-Add (FMA)-dominated execution that underutilizes high-throughput matrix units such as Tensor Cores. This mismatch becomes more severe for multimodal and semantically enriched 3DGS, where high-dimensional features must be accumulated from millions of Gaussians, leading to rapidly increasing computation and memory costs. To address these challenges, we propose a matrix-parallel rendering framework for feature-scalable 3DGS on modern GPUs. We first identify the sequential dependency of alpha compositing as the key obstacle to efficient high-dimensional feature rendering, and derive an order-independent weighted accumulation formulation that transforms feature rendering into matrix-oriented computation. Our framework features three key designs: i: a tile-based splatting pipeline that maps feature accumulation to warp-level matrix-multiply-accumulate execution on Tensor Cores; ii: a warp-granularity Gaussian packing strategy that mitigates visibility irregularity and improves feature reuse; and iii: a lazy channel-batched feature-fetching pipeline that reduces shared-memory pressure, maintains high warp occupancy, and overlaps memory access with matrix execution for high-dimensional features. Extensive experiments on Mip-NeRF 360, LERF, and 3DOVS demonstrate that our approach enables both high performance and strong scalability for feature-rich 3DGS rendering. Particularly, our method achieves up to 3.8$\times$ speedup over 3DGS and 2$\times$ over TC-GS while preserving rendering quality and semantic accuracy, and further reduces kernel render time by up to 36.7$\times$ for high-dimensional features.

Creating explorable 3D environments from text is important for immersive content creation, simulation, and interactive design. Recent methods often rely on image or video generative priors by synthesizing views along fixed or manually specified trajectories and lifting them into 3D. However, such passive pipelines cannot adapt view generation to the evolving scene memory, often leaving disoccluded regions incomplete and producing holes, floaters, and inconsistent geometry under free-viewpoint exploration. We argue that the key challenge is not to generate more views, but to decide where to generate useful views according to the current scene state. To this end, we propose ActPano3D, an active text-to-3D scene generation framework based on memory-guided panoramic exploration. ActPano3D formulates trajectory selection as spatial deliberation over the evolving scene memory, where candidate actions are evaluated by their expected utility for exploring unknown regions, repairing uncertain geometry, and improving the future 3D world model. The framework integrates expansion-refinement active planning, memory-routed panoramic generation, and reliability-aware scene memory fusion into a closed-loop generation-and-reconstruction process. The accumulated memory is further optimized into a renderable 3D Gaussian Splatting scene for free-viewpoint exploration. Experiments show that ActPano3D improves scene coverage, reduces rendering holes, and achieves better rendering quality and text alignment than existing text-to-3D scene generation baselines.

Existing 3D Gaussian Splatting (3DGS) segmentation methods produce accurate rendered masks yet often fail to select a complete set of object Gaussians: removing the predicted Gaussians leaves the rendered depth in the target region nearly unchanged. We trace this gap to the contribution-weighted supervision path of rendering-based feature learning. Because Gaussian geometry and opacity are pre-optimized for RGB reconstruction, rendered-feature losses deliver strong gradients only to high-contribution Gaussians, leaving other Gaussians in the same object region weakly supervised and ultimately unselected. We propose VODA, a plug-and-play supervision objective that addresses this bias. After RGB optimization, Gaussians of the same object portion concentrate at similar depths along the camera ray. A depth-aware rasterizer exploits this property, grouping per-pixel contributing Gaussians into depth-coherent clusters and selecting the cluster that best explains each object pixel. The rendered feature of the selected cluster is then applied as a target directly to every Gaussian within it, bypassing the contribution-weighted gradient path. A complementary pixel-level loss further sharpens boundaries between adjacent objects. To evaluate volumetric completeness, we introduce Removal-3D and the Removal Depth Error (RDE), which measure whether removing the predicted Gaussians induces the expected depth change in 3D. Across four rendering-based baselines, VODA consistently improves volumetric segmentation on Removal-3D while maintaining or improving standard performance on LERF and ScanNet.

Vision--language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly confident under paraphrase and omitted details. We aim to learn representations whose matching is stable across caption views and whose confidence reflects how strongly text constrains an image. We propose Text as Partial Constraint (TPC), a core--residual alignment framework that treats multi-view captions as incomplete supervision: it distills a consensus semantic core as the alignment target, learns a single-view core predictor for standard inference with one query, and explicitly discourages vision--language similarity from depending on the orthogonal ``unsaid'' residual. An uncertainty-aware contrastive objective further softens alignment when caption views disagree, reducing overconfident updates under weak language constraints. Across zero-shot recognition and adversarial robustness, TPC achieves 81.42/64.05 Top-1 clean/robust accuracy on ImageNet and 76.19/52.03 on an Avg-14 transfer suite, while improving LVLM transfer with 85.16 POPE F1 and 59.57 OKVQA accuracy under an LLaVA-1.5-7B stack. These results suggest that modeling text as a partial constraint is a practical and principled route to more reliable vision--language representations under underspecified language supervision.

Methods that operate on Vision Transformer features almost universally rely on Euclidean distance to judge feature similarity. Yet Euclidean distance weights every direction in feature space equally, ignoring that downstream tasks concentrate their sensitivity on a low-dimensional submanifold. The metric that respects this structure is the pullback Riemannian metric $g(F) = J(F)^\top J(F)$ induced by the task decoder's Jacobian, but materializing it is prohibitively expensive; a single ViT-B/14 probe point requires storing $4\times 10^{10}$ scalars, and per-token scoring provably fails when task-sensitive directions are spread across tokens. We show that whether a low-rank approximation of $g(F)$ is \emph{learnable} at all depends on a precise structural property of the model-decoder pair, which we characterize via a single scalar diagnostic $\kappa_{cap}(r)$, the captured-energy fraction of the top-$r$ singular directions of $J$, computable matrix-free in $\mathcal{O}(m+rq)$ Jacobian-vector products. When the diagnostic licenses it, we train the \emph{Spectral Pullback Network} (SPN) on directions found by randomized power iteration on $J^\top J$, and distill the resulting per-token signal into a 310K-parameter feature-only \emph{conformal head}, never materializing $J$. When the original spectrum is too rich for any practical rank-$r$ budget, a VAE reparameterization of the decoder recovers a latent space in which the same machinery applies, converting intractable dense pipelines into tractable ones. Across DPT, DINOv2, CLIP, and VGGT backbones the diagnostic correctly predicts which architecture succeeds: the conformal head reaches Spearman $\rho=0.998$ against the Jacobian-derived target on DINOv2 CLS, and geometric token pruning yields a 25\% relative improvement over ToMe at aggressive prune ratios on DPT depth. These results suggest that learning \emph{which} directions matter is often more valuable than learning better features.

Diffusion models have achieved remarkable success in image, video, and 3D content creation. However, extending these successes to native 4D model generation remains challenging due to the intricate coupling between temporal geometry and appearance dynamics. Existing approaches typically generate 4D content by synthesizing videos from novel viewpoints, yet they often suffer from poor spatio-temporal consistency across views and time. To address these limitations, we introduce TESLA, a novel feedforward approach for generating native 4D Gaussian Splatting (4DGS) representations. Our key insight is to decouple temporal geometry and appearance generation through a temporally structured latent representation. This decomposition allows us to first establish coarse temporal geometry that captures fundamental structural dynamics, and then synthesize fine-grained appearance details conditioned on this geometric scaffold. To bridge this decoupled representation to the final output, we design a native 4DGS decoder that directly transforms our temporal features into native 4DGS representations, enabling flexible temporal rendering across arbitrary viewpoints and timesteps. Trained on a large collection of curated animated sequences, we extensively evaluate TESLA on the 4DGS generation task, showing superior performance over existing baselines.

Triangle-based splatting has emerged as a promising bridge between high-quality novel view synthesis and traditional graphics pipelines. However, naively applying unstructured splatting mechanisms to polygonal primitives fails to exploit their inherent topological advantages, leading to over-tessellation and degraded high-frequency appearances. To address, we present T$^2$-Splat, a mesh splatting framework with adaptive topology and texture optimization. We introduce progressive edge-collapse and vertex-split operations to adaptively allocate triangle primitives, reducing face count while preserving well-conditioned connectivity. We further propose a scale-aware residual Mip-texture model to decouple appearance from geometry, for providing the topology with flexibility needed to safely collapse redundant triangles without introducing blurring or aliasing. Experiments on the Mip-NeRF 360, Tanks \& Temples, and DTU benchmarks demonstrate that T$^2$-Splat improves PSNR by +0.87 dB and maintains robust visual fidelity even under aggressive mesh compression to 10\% of the original face count.

3D Gaussian Splatting enables high-fidelity, real-time novel view synthesis, yet adapting it to dynamic, evolving scenes remains challenging. Existing methods rely on fixed static-dynamic partitioning, which suffers from persistent partitioning errors and cannot adapt during optimization. To tackle this issue, we introduce SynGS, a unified 3D Gaussian framework for continual scene updating and multi-view change detection. We cast static-dynamic decomposition as a learnable task, jointly optimizing per-Gaussian change probabilities and scene representations for iterative refinement. We further propose a change-guided update strategy that modulates illumination-aware photometric corrections via learned probabilities, separating transient appearance variations from authentic scene changes. Projecting Gaussian-level probabilities to the image plane yields 3D-consistent change masks, enabling mutual optimization between reconstruction and change localization. An illumination-decoupled refinement step further improves mask quality. Evaluations on standard benchmarks show that SynGS outperforms baseline methods on scene update and change detection tasks, while fully retaining the real-time rendering capability of 3D Gaussian Splatting. Code will be released upon acceptance.

With impressive progress in architectures, training recipes, and quantity of high-quality data, recent monocular geometry estimation methods estimate global 3D geometry remarkably well. However, they still exhibit surprisingly low quality on surface details, which is clearly visible in qualitative predictions but not reflected in common metrics. To improve this shortcoming, we first formulate a metric that is capable of measuring the surface quality of the estimated geometry via normals derived from the predicted point map. Furthermore, we propose two manners to improve fine-grained local geometry in monocular geometry estimation models: (i) we present an improved point-based gradient loss, and (ii) we introduce a novel decoder based on neighborhood attention. We evaluate our proposed method on eight common zero-shot monocular geometry estimation benchmarks and achieve state-of-the-art results, particularly improving local surface geometry. Extensive ablations validate our design choices. Code and models will be open-sourced.

We present StereoSplat, a feed-forward 3D Gaussian Splatting architecture designed explicitly for stereo videos to achieve reliable, metric-scale novel-view synthesis. Unlike monocular approaches that neglect the nature of binocular pairs, StereoSplat adopts a native stereo architecture that extracts dense multi-scale latent features and disparity from a foundation stereo backbone and feeds them into a dedicated Gaussian decoder. Our decoder utilizes cross-view feature fusion and gated spatial refinement to directly translate these rich representations into per-pixel 3D Gaussians. To resolve the high redundancy of per-pixel predictions across overlapping views, we propose a learnable depth-adaptive aggregation mechanism that clusters and aggregates primitives in inverse-depth space while preserving fine-grained details. To support robust training and generalization, we introduce XRStereo, a large-scale synthetic dataset of stereo video featuring various rig configurations tailored to match the physical hardware of real-world platforms. Trained exclusively on synthetic data and evaluated across challenging benchmarks, StereoSplat yields substantial improvements in the depth accuracy of rendered novel views over established baselines while maintaining high-fidelity novel view synthesis. By overcoming prior geometric limitations, our framework enables robust zero-shot transfer to real-world domains, offering a scalable geometric foundation for spatial computing in mixed reality, autonomous driving, and robotics.

Existing 3D point cloud tracking methods have not fully exploited temporal information in point cloud sequences, making it difficult to effectively capture the continuous evolution of target states and thereby limiting tracking stability in scenarios with large cross-category geometric variation. To address this issue, we propose TETrack3D, a framework for category-agnostic 3D point cloud tracking. The core idea of TETrack3D is to elevate temporal information from an auxiliary cue to an explicit learning constraint for cross-frame association. Specifically, we introduce a historical feature reuse mechanism to preserve and reuse target-related features accumulated over multiple frames, enabling the association process in the current frame to access richer and finer-grained historical context. We further design a state evolution supervision module, which guides the model to learn temporally consistent target motion patterns from historical observations by predicting future target states. In addition, to alleviate cross-frame feature drift caused by target-state changes in point cloud sequences, we propose a temporal distribution alignment strategy based on optimal transport theory, which constrains target features across adjacent frames at the distribution level and improves the stability of temporal target modeling. Extensive comparative experiments on KITTI, nuScenes, and Waymo demonstrate that explicitly modeling the target-state evolution process effectively improves the localization robustness of 3D object tracking.

Streaming 3D reconstruction has become a critical component for real-time applications. While recent advances have adapted large-scale offline feedforward models into streaming architectures via causal attention and KV-caching, they suffer from fundamental instability: the irreversible bias in early frames. This limitation stems from two coupled factors: (1) the existing KV-cache-based causal architecture strictly prohibits access to future information, resulting in information-poor early caches; and (2) current methods typically anchor the pose constraints to the initial frame, amplifying the early bias throughout the sequence. To address these challenges, we propose Stable3R, a holistic framework that establishes stable geometric references by enriching historical caches and anchoring poses to prefixes rather than a single initial frame. Specifically, we first introduce a lookahead-augmented attention mechanism, integrating future observations into past caches without violating causal constraints. Second, we adopt a frame-equivariant architecture with prefix-relative supervision, alleviating the reliance on the biased initial frame. Extensive experiments across multiple benchmarks show that our method consistently boosts reconstruction quality within a strictly causal inference schedule.

Latent 3D representations have significantly expanded the capabilities of modern shape modeling, enabling compact encoding of 3D scenes and supporting powerful 3D generative models. Despite this progress, current representations struggle to capture the fine details of complex objects, fundamentally limited by the capacity of their latents. However, scaling latent capacity at training time is computationally expensive and often infeasible under memory limits, and naively increasing the number of latents at inference fails to make effective use of the added capacity. In this paper, we address both challenges with split-and-scale, a method that effectively expands latent capacity at inference while keeping training cost tractable. Our key observation is that by representing 3D scenes as continuous functions of 3D coordinates, a single pretrained tokenizer can be applied at inference to arbitrarily small sub-regions of a scene, allocating the same latent capacity to a smaller spatial extent and thus capturing far higher detail. We validate this insight by conducting extensive ablations to study the trade-offs between compute, memory, latent capacity, and reconstruction quality, demonstrating our inference-time scaling method achieves quality competitive with much larger models that exceed single-GPU memory limits. We further demonstrate that split-and-scale supports high-quality generative modeling, training an autoregressive 3D generator that performs competitively with state-of-the-art baselines.

Novel view synthesis methods such as 3D Gaussian Splatting degrade under sparse-view settings, suffering from an inability to extrapolate beyond observed regions. To enable high fidelity reconstruction and rendering from sparse views, inpainting of sparsely observed areas and completion of occluded regions is required. Unlike existing approaches that perform these tasks using 2D inpainting diffusion models or geometric-only 3D completion models, we propose a fully 3D scene completion framework, called InvisiSplat, that performs geometry and appearance completion directly in 3D. Our method consists of a Gaussian Splat VAE which learns a sparse point cloud latent space that predicts Gaussian Splat parameters from the latents, as well as a two-step flow model that learns to generate samples conditioned on the point cloud latents, different from other approaches that use tokens without 3D coordinates. Our flow model utilizes alternating local 3D attention and global attention, which balances the needs for precise local geometry and large-scale spatial context. Across several datasets, we demonstrate that our method produces 33% - 82% higher fidelity 3D geometry over prior work while attaining competitive performance on appearance metrics when compared to existing approaches.

Spiking Neural Networks (SNNs) offer a promising energy-efficient alternative to Artificial Neural Networks (ANNs) through event-driven, multiplication-free computation. However, when applied to downstream tasks such as Neural Radiance Fields (NeRF), SNNs suffer from significant information loss, resulting in a noticeable performance gap. In this paper, we present \textit{Spik-NeRF v2} to advance SNN-based neural rendering. We propose the $\pm$I-LIF spiking neuron, which extends I-LIF to support signed integer values during training, thereby mitigating information loss. During inference, it is converted to ternary spikes $\{-1, 0, 1\}$, preserving event-driven properties with addition-only operations. Furthermore, we introduce a re-parameterization technique that transforms a trained I-LIF-based Spik-NeRF with $t$ timesteps into an equivalent $\pm$I-LIF-based model with $t/2$ timesteps. This enables faster inference while preserving rendering quality, overcoming the suboptimal performance of directly trained $\pm$I-LIF models with reduced timesteps. Extensive experiments on synthetic and realistic datasets demonstrate that Spik-NeRF v2 surpasses existing SNN-based NeRF methods and achieves rendering quality comparable to ANN-based approaches.

Active view selection for 3D Gaussian Splatting (3DGS) requires information-theoretic grounding, computational efficiency, and strong reconstruction quality, yet existing methods satisfy these only partially. Information-theoretic approaches quantify gain over the full parameter space via costly Fisher computation under a Laplace approximation; faster alternatives sacrifice the grounding they started from. We propose SpheriBED, a Bayesian experimental design framework that defines the information objective over per-Gaussian spherical appearance—the quantity each observation fundamentally measures—rather than the full parameter vector, yielding exact closed-form posteriors without Laplace approximation. Scoring a candidate view requires only a single forward rasterization pass with no backward pass or per-candidate optimization. Extensive experiments across benchmarks demonstrate that SpheriBED consistently improves active reconstruction performance and provides more reliable uncertainty quantification compared to state-of-the-art methods.