NeurIPS 2026 论文列表

Vision-language action (VLA) models excel for end-to-end autonomous driving, yet their tendency to over-invoke chain-of-thought (CoT) reasoning mirrors a human cognitive pitfall: overthinking. Just as deliberate reasoning can slow/mislead human judgment in routine tasks, excessive CoT in VLA incurs unnecessary compute overhead and can paradoxically degrade planning performance. Recent research tackles this via adaptive reasoning, which dynamically modulates inference complexity by selectively triggering CoT for complex scenes while responding directly to routine ones. While prior adaptive methods rely on implicit adaptation, recent work shows that explicitly optimizing the route between CoT and direct responses yields superior performance, assuming scene-difficulty annotations which are labor-intensive, poorly scalable, and prone to error due to the subjectivity of judging driving complexity. We explore, for the first time, whether adaptive reasoning with explicit route optimization can be achieved without any scene-difficulty labeling. We introduce STEER, a generic over-reasoning mitigation strategy featuring two key components: (i) routing uncertainty-aware rollout, which calibrates sampling to align routing diversity with model uncertainty; and (ii) cross-route advantage credit assignment, which introduces a differential metric to reinforce optimal routing decisions based on environmental rewards rather than subjective labels. Extensive experiments on NAVSIM v1/v2 demonstrate state-of-the-art results in both planning quality and the Pareto front of inference efficiency.

3D object grounding localizes referred objects in a 3D scene from natural language. Unified instance-centric 3D-LLMs aim to solve grounding together with dialog, QA, and captioning, yet many rely on a single pointer-style grounding decision that compresses a relational instruction into one selection. This is brittle for fine-grained queries where multiple same-class candidates must be ruled out by context objects and spatial relations. We propose Structured Spatial Reasoning 3D-LLM (SSR3D-LLM), a structured grounding interface for unified 3D-LLMs. Given fixed Mask3D object proposals, the LLM writes a sequence of latent spatial reasoning steps and memory tokens from the query, and a geometry-aware scorer reads these latent steps in order to refine candidate rankings step by step with step-length masking. The latent steps are learned from standard benchmark target supervision with auxiliary referential-cue supervision during training, while inference uses only the input query and Mask3D proposals. Across ReferIt3D, ScanRefer, and Multi3DRef, SSR3D-LLM achieves the strongest results among unified 3D-LLM baselines, with substantial gains over the single-pointer QPG baseline on fine-grained grounding and consistent improvements over prior unified 3D-LLMs, while preserving the default language-task route.

Modern image super-resolution methods generate detailed, visually appealing results, but they often introduce visual artifacts: unnatural patterns and texture distortions that degrade perceived quality. These defects vary widely in perceptual impact—some are barely noticeable, while others are highly disturbing—yet existing detection methods treat them equally. We propose artifact prominence as an evaluative target, defined as the fraction of viewers who judge a highlighted region to contain a noticeable artifact. We design a crowdsourced annotation protocol and construct SR-Prominence, a dataset suite containing 3,935 artifact masks from DeSRA, Open Images, Urban100, and a realistic no-ground-truth Urban100-HR setting, annotated with prominence. Re-annotating DeSRA reveals that 48.2% of its in-lab binary artifacts are not noticed by a majority of viewers. Across the suite, we audit SR artifact detectors, image-quality metrics, and SR methods. We find that classical full-reference metrics, especially SSIM and DISTS, provide surprisingly strong localized prominence signals, whereas no-reference IQA methods and specialized artifact detectors often fail to generalize across datasets and reference settings. SR-Prominence is released with an objective scoring protocol that allows new metrics to be benchmarked on our suite without further crowdsourcing. Together, the data and protocols enable SR artifact evaluation to move from binary defect presence toward perceptual impact.

Recent foundation models are moving toward native multimodal Vision-Language Models (VLMs), making VLMs a central form of next-generation foundation models. However, their large language backbones make edge deployment difficult due to high memory footprint and memory-bound autoregressive decoding. Weight-only post-training quantization is a practical solution, but pushing VLMs to extreme low bit-widths remains challenging: existing rotation-free methods suffer from outliers at 2--3 bits, while rotation-based methods improve accuracy at the cost of additional runtime overhead. We propose SPHQuant, a rotation-free spherical weight-only quantization framework for VLMs. Instead of quantizing weights directly in Cartesian coordinates, SPHQuant decomposes each 8D weight vector into coordinate signs, radius, and a positive unit direction. This representation isolates outlier magnitude into the radius while keeping directions bounded and statistically regular. Based on this insight, SPHQuant allocates extra precision to the radius to mitigate accuracy degradation induced by outliers. It further uses a compact positive-direction codebook and fine-tunes codebook entries through angular parameterization to preserve the unit-sphere constraint. We also design a hardware-friendly GEMV kernel that keeps the direction codebook small enough for shared-memory lookup and packs radial bits efficiently. Experiments show that SPHQuant matches the performance of state-of-the-art extreme low-bit quantization methods while improving decode throughput by \textbf{30.3\%}, providing a new step toward efficient edge deployment of VLMs.

Vision-Language Models for Earth Observation are bottlenecked at the visual encoder. Existing pipelines either compress multi-spectral signals into RGB pixels or collapse them into a single CLIP-style global vector, losing spectral information or spatial structure respectively. We show this is a consequence of coupling contrastive supervision to the dense feature stream, which degrades spectral fidelity. SPEXT resolves this through architectural decoupling. A wavelength-parameterized encoder produces multi-spectral patch tokens for dense tasks and VLM grounding, while a dedicated alignment token absorbs the contrastive gradient for retrieval. SPEXT simultaneously leads the PANGAEA segmentation benchmark (+2.51 mIoU), tops zero-shot retrieval among multi-spectral CLIP baselines (+4.6 mAP@100), and enables a frozen VLM, paired with a small projection adapter and LoRA, to answer free-form Earth-Observation questions directly from its tokens, substantially outperforming an RGB-pixel baseline on a downstream EO-RAG benchmark.

Generalist neural routing solvers have shown great potential in solving diverse vehicle routing problems (VRPs) with a unified model. However, existing solvers are typically limited to symmetric settings or degrade in performance when switching to asymmetric settings due to input inconsistencies or inherent structural differences, substantially limiting their practicality in real-world scenarios that encompass both scenarios. To address this limitation, we define the spatial position of each node based on the relative distances to a specific set of pivots and further propose a Spatial Pivot-Aligned Coordinate-free Embedding (SPACE) framework that unifies node representation and solution generation across symmetric and asymmetric VRPs. Specifically, we construct a bidirectional Fréchet representation using a novel furthest pivot sampling strategy to enable invariant node representations across distinct problem settings. Furthermore, we introduce a weight-decomposed adaptive decoding mechanism that decouples geometric perception from problem representations, mitigating the overfitting of constraint decisions to a specific geometry setting. Extensive experiments on 110 VRP variants, comprising 55 symmetric problems and their asymmetric counterparts, demonstrate that SPACE achieves promising zero-shot generalization in both symmetric and asymmetric VRPs.

In this paper, we introduce SP$^3$, a novel Plug-and-Play algorithm that accelerates Maximum \emph{a posteriori} image restoration by replacing denoisers with Spherical Encoders (SE) as generative priors. SP$^3$ approximates the intractable proximal prior step by utilizing the SE tightly structured latent space as a robust projection onto the natural image manifold. Alternating this projection with a closed-form data-consistency step, via Half-Quadratic Splitting, guarantees stable convergence without requiring gradient computation during inference. This unique formulation unlocks ``anytime'' restoration capabilities, producing sharp, plausible images from the first iteration. Evaluations across a variety of image restoration tasks demonstrate that SP$^3$ achieves perceptual quality comparable to state-of-the-art zero-shot diffusion and flow methods while being $3$-$630\times$ faster.

Speculative Decoding (SD) has shown strong success in accelerating large language model inference, but its efficiency remains limited for Vision-Language Models (VLMs). The diverse input modalities in VLMs make it difficult for a small trained drafter to achieve consistently high throughput across modalities. We address this limitation with an adaptive self-speculative decoding method. In particular, we adaptively select the skipped layers of VLMs to form the draft model that best fits the given query. Our selection algorithm SP$^2$ec utilizes the single-peak structure of the throughput with respect to the number of selected layers. Our algorithmic design explicitly minimizes the novel notion of wall-time regret for the given query, corresponding to the empirical wall-time latency. Theoretical guarantees on the wall-time regret show that SP$^2$ec is adaptive to the per-prompt single-peak structure. We conduct extensive experiments with Qwen3-VL and LLaVA-1.5 models across visual and textual tasks, which demonstrates the efficacy of the proposed SP$^2$ec.

Vision-language models (VLMs) have shown impressive capabilities across diverse tasks, motivating efforts to leverage these models to supervise robot learning. However, when used as evaluators in reinforcement learning (RL), today’s strongest models often fail under partial observability and distribution shift, enabling policies to exploit perceptual errors rather than solve the task. We introduce SOLE-R1 (Self-Observing LEarner), a video-language reasoning model explicitly designed to serve as the sole reward signal for online RL. Given only raw video observations and a natural-language goal, SOLE-R1 performs per-timestep spatiotemporal chain-of-thought (CoT) reasoning and produces dense estimates of task progress that can be used directly as rewards. To train SOLE-R1, we develop a large-scale video trajectory and reasoning synthesis pipeline that generates temporally grounded CoT traces aligned with continuous progress supervision. This data is combined with foundational spatial and multi-frame temporal reasoning, and used to train the model with a hybrid framework that couples supervised fine-tuning with RL from verifiable rewards. Across four different simulation environments and a real-robot setting, SOLE-R1 enables zero-shot online RL from random initialization: robots learn previously unseen manipulation tasks without ground-truth rewards, success indicators, demonstrations, or task-specific tuning. SOLE-R1 succeeds on 24 unseen tasks and substantially outperforms strong vision-language rewarders, including GPT-5 and Gemini-3-Pro, while exhibiting markedly greater robustness to reward hacking.

Vision Transformers dominate modern image benchmarks, however, what trained self-attention computes head by head remains unclear: prior work either injects inductive biases at initialization or clusters attention patterns visually without committing to a closed-form computation. We ask whether the softmax$(QK^\top)$ inside a trained head can be replaced, without further training, by a structured closed-form operator while preserving prediction and the residual-stream trajectory. We answer yes across ten ViTs and seven pretrainings (ImageNet-1k, ImageNet-21k FT, MAE, DINO, DINOv2, CLIP, SigLIP), and package the result as \textbf{SOLA}: a Structured Operator Library for Attention with three entries, each gated by a per-head diagnostic that upper bounds substitution error. The entries are a fixed 2D convolution kernel for shift heads, a per-image broadcast row for fixed-target heads, and a per-image rank-$k$ SVD of the attention matrix for near-global heads. A rank-adaptive extension picks $k$ per head from the SVD spectrum and extends structural coverage to every head on every backbone, with task-fidelity controlled by a single spectral threshold $\tau$ that trades compression for downstream fidelity. The diagnostics carry over to cross-attention: every decoder cross-attention head in BLIP captioning has effective rank $\approx 1$ and admits the broadcast substitution. A minimal reproducer that runs the full SOLA pipeline on a single backbone is included in the Supp. Mat. as a demo; the full codebase will be released open-source upon acceptance.

Regression-based LiDAR relocalization has recently emerged as a promising solution for high-precision positioning in GNSS-denied environments. However, these methods are primarily tailored to autonomous driving, exhibiting significantly degraded accuracy in unmanned aerial vehicle (UAV) scenarios due to arbitrary pose variations and irregular flight paths. In this paper, we propose SOAR, a regression-based LiDAR relocalization framework for UAVs. Specifically, we introduce a locality-preserving sliding window attention module with locally invariant positional encoding to capture discriminative geometric structures robust to viewpoint changes. A coordinate-independent feature initialization module is further designed to eliminate sensitivity to global transformations. Furthermore, most existing UAV datasets are limited to evaluate LiDAR relocalization in real-world, due to the lack of synchronized LiDAR scans, accurate 6-DoF poses, or multiple traversals. Thus, we construct a large-scale UAV LiDAR localization dataset with 4 scenes and 13 irregular paths exhibiting rotation and altitude variations, providing a more realistic benchmark for UAVs. Extensive experiments demonstrate that our method achieves state-of-the-art performance, improving the localization success rate by 40\% and reducing mean error over 10m on UAVLoc. Our code and dataset will be released soon, and the dataset demo is available via the anonymous link.

Existing benchmarks for multimodal memory reasoning largely evaluate systems within pre-assembled contexts, but under-evaluate whether agents can use evidence distributed across independently originated sources. We argue that source-distributed memory composition is an important and under-examined bottleneck in multimodal agent memory, especially when relevant evidence is fragmented across heterogeneous artifacts such as conversations, profiles, screenshots, tables, images, and documents. To address this gap, we introduce Source-distributed Multimodal Memory Benchmark, which measures whether agents can retrieve, align, and compose multimodal evidence scattered across multiple sources rather than reason within a single curated context. SMMBench evaluates four core capabilities: (1) cross-source multimodal reasoning; (2) conflict resolution; (3) preference reasoning; (4) memory-grounded action prediction. The benchmark contains $1,877$ samples grounded in $264$ sources. Experiments on representative memory-style and retrieval-based baselines show that current systems still struggle on these capabilities, positioning source-distributed multimodal memory as an important and still under-evaluated challenge for multimodal agents. Our data and code are available at https://anonymous.4open.science/r/NeurIPSBenchmarksW8kP6gV/.

We introduce Skill Learning from Video Memory (SLVMBench), the first benchmark that jointly evaluates whether video large language models (video-LLMs) can learn skills from long video memory and apply them to real-time tasks. SLVMBench presents models with 2–3 hour video streams that contain a tutorial video embedded in a stream of arbitrary irrelevant videos, resembling real-world human learning practices. Video-LLMs are asked to apply the acquired skill to answer real-time questions about an ongoing video. Unlike long-video understanding benchmarks that emphasize passive comprehension and skill-learning benchmarks that rely on short, immediate demonstrations, SLVMBench tests the full pipeline of memorizing and extracting procedural knowledge, as well as transferring it to real-time tasks. Moreover, rigorous human annotations feature sub-second-level temporal calibration, manually engineered questions eliminating common-sense guessing, and collated tutorials to ensure coverage of the required skills. Evaluations on state-of-the-art proprietary and open-source video LLMs show that video-LLMs struggle substantially with learning and applying skill knowledge from videos. Moreover, performance degrades markedly when the skill knowledge is placed within a long video memory. These results reveal a key limitation of existing video LLMs and position SLVMBench as the first benchmark for studying real-time skill acquisition and application from long-context video memory.

Lifelong cross-modal hashing demands incremental updates as new tasks arrive, yet existing regularization-based methods such as EWC and SI share a fundamental weakness: their effective regularization strength is entangled with task scale, as the empirical Fisher Information Matrix (FIM) scales linearly with sample size. This forces costly per-task hyperparameter tuning in streaming deployments to avoid over- or under-regularizing tasks of varying sizes. We propose \textsc{Sing-CH}, a lifelong hashing framework built on information geometry. At its core is a Gibbs probabilistic reconstruction of the pairwise objective, which gives the parameter space a statistical manifold structure and yields a natural $1/n_t^2$ normalization. We prove this normalization renders the FIM scale-agnostic: its spectral properties remain independent of task size even under diagonal and K-FAC approximations. Beyond this invariance, a Forgetting-Rigidity trade-off theorem shows that natural gradient applied to the composite objective reintroduces scale dependence. This makes our decoupled design, pairing the FIM as regularizer with Adam for optimization, a structural necessity rather than a convenience. Experiments on three benchmarks demonstrate that \textsc{Sing-CH} outperforms 11 baselines by 4.1\% average mAP without per-task tuning. Under extreme task-scale imbalance, degradation is only 2.0\% versus 3.9\% for the strongest baseline, providing the first empirically grounded solution to scale-agnostic lifelong retrieval.

Text-driven image editing has advanced rapidly, but reliably localizing these manipulations requires image manipulation localization (IML) models trained on large pixel-annotated datasets, and there is still no low-cost way to obtain such training data at scale. We observe that these data already exist in disguise: public editing datasets contain millions of structurally identical $\textit{(original, edited)}$ pairs to IML training samples, lacking only pixel-level masks. Recovering these masks automatically is non-trivial: pixel differencing is overwhelmed by diffusion-induced perturbations across all pixels, and instruction-only grounding localizes only what the prompt describes, missing unintended editor side-effects. We propose $\textbf{SIGMA}$ ($\textbf{S}$emantic-difference $\textbf{I}$nstruction-$\textbf{G}$rounding $\textbf{M}$ask $\textbf{A}$nnotator), which performs semantic-feature differencing in a vision foundation backbone and injects an instruction-derived spatial prior into this visual stream via bidirectional cross-modal refinement, amplifying the difference signal at intended-edit regions when the editor faithfully realizes user intent. SIGMA is trained in two complementary stages: Stage I supervises on inpainting masks; Stage II closes the diffusion-domain shift via VAE-roundtrip noise calibration, EMA self-training, and an edit-noise disentanglement loss. SIGMA outperforms existing automatic mask generators on five benchmarks ($\textbf{+12.20\%}$ F1, $\textbf{+11.16\%}$ IoU). When applied to public editing corpora, it produces a $\sim$1.1M IML training set that improves six diverse detectors by $\textbf{+18.34\%}$ F1 across five datasets, turning previously unused editing data into a model-agnostic supervisory resource for IML. For reproducibility, our anonymized code is available at https://anonymous.4open.science/r/SIGMA-14FE.

Multimodal large language models (MLLMs) achieve ever-stronger performance on visual-language tasks. Even as traditional visual question answering benchmarks approach saturation, reliable deployment requires satisfying low error tolerances in real-world out-of-distribution (OOD) scenarios. Precisely, selective prediction aims to improve coverage, i.e.\ the share of inputs the system answers, while adhering to a user-defined risk level. This is typically achieved by assigning a confidence score to each answer and abstaining on those that fall below a certain threshold. To enable reliable generalization, we require reasoner models to produce localized visual evidence while answering, and design a selector that explicitly learns to estimate the quality of the localization provided by the reasoner. We show that SIEVES (Selective Prediction through Visual Evidence Scoring) improves coverage by up to three times on challenging OOD benchmarks (V* Bench, HR-Bench-8k, MME-RealWorld-Lite, VizWiz, and AdVQA), compared to non-grounding baselines. Beyond better generalization to OOD tasks, the design of the SIEVES selector enables transfer to proprietary reasoners without access to their weights or logits, such as o3 and Gemini-3-Pro, providing coverage boosts beyond those attributable to accuracy alone. We highlight that SIEVES generalizes across all five tested OOD datasets and reasoner models (Pixel-Reasoner, o3, and Gemini-3-Pro), without benchmark- or reasoner-specific training or adaptation.

Mathematical reasoning benchmarks for large language models face three persistent limitations. First, datasets scraped from public competitions are vulnerable to training-data contamination and saturate quickly. Second, most existing benchmarks cover only part of the olympiad-to-research spectrum. Third, benchmarks that withhold problems indefinitely to mitigate leakage trade away transparency and reproducibility. We introduce \gls{soohak}, a benchmark of 1{,}141 newly authored problems by 86 mathematicians designed to address all three. \gls{soohak} consists of three difficulty splits, constructed adversarially against panels of small, mid-size, and frontier baseline LLMs, spanning olympiad-style problem solving through research-adjacent material; a separate 99-item Refusal split probes recognition of ill-posed prompts. The dataset is temporarily embargoed and evaluated by request, with a committed public release in late 2026. On \splitthree{}, Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 reach Avg@3 of 30.39\%, 26.37\%, and 10.39\%, respectively; the best Pass@3 is only 44.12\%, and 129/340 problems remain unsolved by any closed model evaluated, indicating substantial room for improvement before LMs can meaningfully assist in mathematical research. By contrast, the \calibrationsplits{} are nearly saturated by both closed and open-weight systems. Notably, while GPT-OSS-120B, Kimi-2.5, and GLM-5 each reach Pass@3 between 87.4\% and 88.3\% on \splitone{}, within 6 percentage points of the leading closed model (Gemini-3-Pro, 93.85\%), they trail by over 20 percentage points on \splitthree{}. This pattern suggests open-weight pipelines are well targeted to olympiad-style benchmarks but transfer poorly to research-adjacent material. Finally, through a timed human study with 25 participants of diverse mathematical expertise, we confirm that the difficulty is genuinely mathematical rather than artifactual, with aggregated teams covering 50.6\% of a 79-problem sample.

Vision-Language Models (VLMs) show great potential for Industrial Anomaly Detection (IAD) but often overlook subtle high-frequency defects due to their semantic bias. Additionally, one-class adaptation risks rank collapse and catastrophic forgetting. To address these issues, we propose a unified framework termed SF-DST. First, we design Spatial-Frequency Dual-Stream Transformer, which explicitly captures lost details via a parallel discrete cosine transform branch. Crucially, it incorporates Asymmetric Cross-Modal Modulation (ACMM), a mechanism that leverages spatial semantics as a context gate to selectively retrieve and inject spectral features, effectively compensating for the texture blindness of VLMs. Second, Anomaly-Aware LoRA (A-LoRA) enforces Orthogonal Regularization on the adaptation subspace. This geometric constraint ensures subspace diversity to prevent mode collapse, while optimizing a hypersphere-based metric for precise anomaly discrimination. Extensive experiments on MVTec-AD, VisA and MMAD benchmarks demonstrate that our method significantly outperforms state-of-the-art approaches, successfully bridging the gap between large-model generalization and industrial-grade precision.

Diffusion transformers (DiTs) have emerged as a dominant architecture for text-to-image generation, yet their performance drops when generating at resolutions beyond their training range. Existing training-free approaches mitigate this by modifying inference-time attention behavior, often through Rotary Position Embeddings (RoPE) extrapolation combined with attention scaling. However, these strategies apply a uniform and content-agnostic scaling across RoPE components with distinct frequency characteristics, inducing a trade-off between preserving global structure and recovering fine detail. We introduce \textbf{SEGA}, a training-free method that dynamically scales attention across RoPE components according to the latent’s spatial-frequency structure at each denoising step. This adaptive scaling improves both structural coherence and fine-detail fidelity. Experiments show that SEGA consistently improves high-resolution synthesis across multiple target resolutions, outperforming state-of-the-art training-free baselines.