NeurIPS 2026 论文列表

Visual context is a powerful cue for object recognition: scenes inform what, where, and at what scale objects are likely to appear. Yet over-reliance on context becomes a failure mode when models predict objects based on their typical neighbours rather than the objects' own visual features, a phenomenon we refer to as typicality bias. While this issue has been extensively studied in the pre-Transformer era, it has often been assumed that architectural and scale improvements would naturally solve it. Despite recent attempts in isolated settings, a systematic diagnostic across the modern vision stack remains absent. In this work, we provide such a diagnostic framework. Specifically, we measure typicality bias via normalized pointwise mutual information (NPMI) over object pairs, and introduce STRANGE-Bench (Synthetic Typicality-RANked Grounded Evaluation Benchmark): a photorealistic, layout-controlled benchmark spanning 11 typicality tiers from typical to never-co-occurring object pairs. Across \five model families (specialist detectors, open-vocabulary detectors, self-supervised, vision-language, and multimodal large language models) and three recognition tasks}(supervised detection, open-vocabulary detection, and classification), we find the bias to be highly prevalent: gaps of 5--11\% persist when objects appear outside their typical company, and they do not consistently shrink with scale. This persistence motivates a complementary question: how to mitigate it? Data augmentation is a natural lever, since it directly shapes the training distribution that produces the bias; yet existing augmentations are designed for visual diversity, not for breaking co-occurrence shortcuts as they sample from the very distribution that produces typicality bias, preserving rather than perturbing it. We therefore propose Anchor-Atypicality Sampling (AAS), a simple context-aware variant of paste-based augmentations that uses the NPMI signal to deliberately tilt the training-time co-occurrence distribution toward atypical partners. AAS reduces the typicality gap by up to 3.3 pp on natural-image COCO splits and 17.4 pp on synthetic STRANGE-Bench across three representative model families (specialist detection, open-vocabulary detection, self-supervised classification), without sacrificing overall performance.

For practical use, diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic preference. That alignment is challenging because the reward is defined for clean output images, but the alignment procedure requires value function estimates at noisy intermediate latents. Existing methods resort to Tweedie-style or Monte Carlo approximations, trading off estimator bias against computational cost: Tweedie estimates are efficient but biased, while Monte Carlo estimates are more accurate but require expensive rollouts. A natural alternative would be a learned value function, but it remains an open question how to effectively train a strong and general value model specifically for noisy latents. Here, we propose StitchVM (Stitched Value Model), a model stitching framework that efficiently transfers reward models pretrained for clean images to the noisy latent regime. StitchVM starts from an existing, truncated pixel-space reward model and attaches a frozen diffusion backbone to it as its head. From the pixel-space model, the resulting hybrid retains a carefully pretrained, robust reward capability; from the diffusion backbone, it inherits its native ability to handle noisy latents. The stitching procedure is exceptionally lightweight, e.g., stitching and finetuning CLIP ViT-L and SD 3.5 Medium takes only $\approx$10 GPU-hours. By lifting powerful pixel-space reward models to latent space, StitchVM opens up a new style of diffusion alignment: instead of rough, yet costly per-sample approximation of the value function, the correct function for the actual, noisy latents is constructed once and then amortized over many samples and iterations. We show that this approach yields improvements across a broad range of downstream steering and post-training methods: DPS becomes $3.2\times$ faster while halving peak GPU memory, and DiffusionNFT becomes $2.3\times$ faster.

Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or restricted to recognizing pre-specified stereotypes. We introduce StereoTales, a multilingual dataset and evaluation pipeline for systematically studying the emergence of social bias in open-ended LLM generation. The dataset covers 10 languages and 79 socio-demographic attributes, and comprises over 650k stories generated by 23 recent LLMs, each annotated with the socio-demographic profile of the protagonist across 19 dimensions. From these, we apply statistical tests to identify more than 1{,}500 over-represented associations, which we then rate for harmfulness through both a panel of humans (N = 247) and the same LLMs. We report three main findings. \textbf{(i)} Every model we evaluate emits consequential harmful stereotypes in open-ended generation, regardless of size or capabilities, and these associations are largely shared across providers rather than isolated misbehaviors. \textbf{(ii)} Prompt language strongly shapes which stereotypes appear: rather than transferring as a shared set of biases, harmful associations adapt culturally to the prompt language and amplify bias against locally salient protected groups. \textbf{(iii)} Human and LLM harmfulness judgments are broadly aligned (Spearman $\rho=0.62$), with disagreements concentrating on specific attribute classes rather than specific providers. To support further analyses, we release the evaluation code and the dataset, including model generations, attribute annotations, and harmfulness ratings.

Verifiable reward training has improved mathematical and coding reasoning, but these domains capture only part of step-by-step decision making. Many real-world tasks require finding a high-value feasible plan among many valid alternatives. We introduce $\text{OPT}\star$, a scalable family of optimization-style tasks for training and evaluating LLM step-by-step optimization-like reasoning along a complexity axis: each task provides a feasibility checker and evaluator, while a complexity parameter expands the search space without requiring new human labels. This motivates studying these tasks in two regimes: (i) solver-guided online policy optimization, which uses a solver as a value oracle for partial states and applies rank-based reward shaping to reinforce better next steps, and (ii) search-based offline RL when such solvers are unavailable. Theoretically, we relate success in large search spaces to the information a reasoner extracts per unit of search budget. Empirically, we ablate the ingredients that make search efficient on $\text{OPT}\star$ and show that training on $\text{OPT}\star$ improves step-by-step optimization-like reasoning.

Large language models (LLMs) are typically governed by post-training alignment (e.g., RLHF or DPO), which yields a largely static policy during deployment and inference. However, real-world safety is a full-lifecycle problem: static defenses degrade against evolving jailbreak behaviors, and fixed weights cannot adapt to pluralistic, time-varying safety norms. This motivates inference-time governance that steers behavior without costly retraining. To address this, we introduce the Consensus Clustering LinUCB Bandit (CCLUB), a unified framework for adaptive social alignment via system-prompt routing. CCLUB employs a conservative consensus clustering mechanism: it pools data only within the intersection of utility and safety similarity graphs, effectively preventing unsafe generalization across semantically proximal but risk-divergent contexts. Our theoretical analysis gives an expected regret bound $O\left(d\log T/(p_{\min}\gamma^2\lambda_x) + d\sqrt{MT}\log T\right)$, separating the logarithmic cost of identifying safe consensus clusters from the dominant cluster-level exploitation term. Experiments show that CCLUB improves the safety--utility trade-off and offline deployment gap, and improves cumulative reward by 10.98\% over the strongest non-prototype baseline.

As state-of-the-art text-to-image flow models have matured to deliver near-photorealistic quality, controlling what they generate -- e.g., inhibiting harmful content while promoting benign alternatives -- has become a central challenge. The current steering paradigm consists of adding a global steering vector to selected model activations. While functional, a fixed vector applied example-agnostic and uniformly along the entire trajectory cannot adapt to the changing state of the generation, and causes uncontrolled global changes beyond the targeted concepts. We introduce Steering Fields, a generalization of steering vectors that adaptively re-estimates the steering direction at each step of the generative process. Steering Fields operate on latent representations, expose a continuous trade-off between steering strength and content preservation, and are compositional, enabling the simultaneous induction and inhibition of concepts, setting a new state of the art on safety steering benchmarks. Although our formulation prescribes no explicit spatial mask or object-level prior, the trajectory-adaptive estimation naturally preserves local structure, in a manner reminiscent of image editing. In fact, when applied in the VAE latent space, Steering Fields can serve as a structure-preserving image-editing technique that achieves state-of-the-art semantic fidelity (CLIP, VQAScore), while remaining model-agnostic and inversion-free

Activation steering is a practical post-training model alignment technique to enhance the utility of Large Language Models (LLMs). Prior to deploying a model as a service, developers can steer a pre-trained model toward specific behavioral objectives, such as better truthfulness, or reasoning ability, without the need for retraining. Conceptually, these methods implement behavior control through hidden-state interventions, without changing the underlying model parameters. However, this capability unintentionally introduces critical and under-explored safety risks. We identify a phenomenon termed **Steering Externalities**, where steering vectors derived from benign datasets—such as reducing harmless refusals, improving structured-output following, truthfulness, and reasoning performance—inadvertently erode safety guardrails. Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80\% on standard benchmarks by bypassing the initial safety alignment. Ultimately, our results expose a critical blind spot in deployment: benign activation steering can erode the ``safety margin,'' rendering models more vulnerable to black-box attacks and indicating that inference-time utility improvements must be rigorously audited for unintended safety externalities.

We study the complexity of smoothed agnostic learning, in which the learner competes with the best classifier in a target class under slight Gaussian perturbations of the inputs. Specifically, we focus on the prototypical task of agnostically learning halfspaces under subgaussian distributions in this model. The best known upper bound for this problem is based on $L_1$-polynomial regression and has complexity $d^{\tilde O(1/\sigma^2)\log(1/\epsilon)}$, where $\sigma$ is the smoothing parameter and $\epsilon$ is the excess error. Our main result is a Statistical Query (SQ) lower bound showing that this upper bound is close to best possible. In particular, we prove that, even for Gaussian marginals, any SQ algorithm for smoothed agnostic learning of halfspaces requires complexity $d^{\Omega(1/\sigma^2+\log(1/\epsilon))}$. This is the first non-trivial computational lower bound for this task, and it nearly matches the known upper bound. At a conceptual level, we show that the complexity of the problem is governed by the low-degree $L_1$ approximation of the smoothed target $T_\sigma f$, so that applying $L_1$-polynomial regression to the smoothed function is essentially optimal in the SQ model. Our proof proceeds by constructing a moment-matching hard distribution via linear programming duality; the dual program corresponds exactly to finding a low-degree approximating polynomial for $T_\sigma f$, which is the same approximation-theoretic condition underlying the upper bound. To instantiate this framework for halfspaces, we prove explicit lower bounds on the approximation degree of the smoothed sign function. A key ingredient is a new structural result: we construct a distribution that matches moments with a Gaussian while exhibiting periodic structure. This result underlies our $1/\sigma^2$-degree lower bound and may be of independent interest.

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision–language model (VLM) architectures, VLAs predict actions conditioned on visual observations and language instructions, achieving strong performance and generalization across tasks. However, VLAs face two major challenges: a limited context window for input frames and inefficient inference due to the quadratic attention complexity and large parameter counts. To this end, we propose Dysta, a framework that disentangles visual inputs into multi-level static and dynamic tokens, which enables (1) retaining a single copy of static tokens across frames to significantly reduce context length, and (2) reusing the key–value (KV) cache of static tokens through a lightweight recache gate that updates only when necessary. This design enables efficient multi-frame integration and efficient inference. In addition, we introduce a new benchmark that more effectively evaluates the multi-frame integration ability of VLAs. Experiments show that DySta improves multi-frame integration by 24.5\% across metrics on our benchmark and 23.3\% in absolute success rate on real-world memory-dependent tasks, while accelerating inference by 2.0× (with +2.3\% success rate) on simulation benchmarks and 2.2× (with +10.6\% success rate) on real-world general tasks.

Targeted adversarial attacks on Large Vision-Language Models (LVLMs) test whether small image perturbations can steer model responses toward attacker-specified content. Under the standard $L_\infty$ constraint, targeted attacks become a regional perturbation budget allocation problem: attack success depends not only on the perturbation objective, but also on which regions receive updates and in what order. Existing localized attacks improve over global perturbations but rely on stochastic spatial sampling, often updating weakly influential regions. We address this limitation through an attention-based analysis showing that cross-modal attention identifies adversarially sensitive regions and that perturbing high-attention hotspots induces predictable redistribution toward subsequent salient regions. These findings motivate attention-guided region sequencing, which begins from dominant hotspots and progressively moves the update support toward next-salient regions. Based on these principles, we propose Stage-wise Attention-Guided Attack (SAGA), a black-box region-sequencing framework that uses a fixed attention map from an open-source LVLM to guide perturbation updates without accessing target-model parameters, gradients, or attention maps. Across ten closed-source and open-source LVLMs, SAGA achieves state-of-the-art attack success rates and the best overall imperceptibility.

Vector Quantization (VQ) is fundamental to discrete visual tokenizers that power modern autoregressive and masked image generation models. While recent shared-projection codebook methods have substantially advanced codebook utilization, training stability remains a critical and underexplored challenge. We argue that the root cause lies in the entanglement of the Encoder--Decoder and Codebook training: because neither module can reliably fulfill its own responsibility in isolation, the system can only function when the two subsystems happen to cooperate---a fragile condition that breaks down precisely when training is most stressed. We propose StableVQ, which revisits the proper learning objective of each module and resolves the problems that arise when each is trained to fulfill its own role independently. Concretely, (1) Dynamic STE corrects the instability in the Encoder's learning objective, enabling it to robustly optimize the reconstruction space under discrete regularization even when codebook utilization is low. (2) Region VQ Loss reconceives the Codebook's learning objective so that it can independently guarantee full tracking of the encoder output distribution, without relying on encoder oscillations to drive activation. (3) Decoupled Schedule recognizes that the distinct responsibilities of the Encoder--Decoder and the Codebook demand distinct optimization dynamics, and assigns each an independent learning rate schedule to ensure robust system-level behavior. Built on top of shared-projection codebooks, StableVQ is lightweight and introduces no learnable parameters. Experiments on ImageNet demonstrate consistent improvements in training stability, codebook utilization, and reconstruction quality across diverse codebook sizes and initialization settings.

Filtering-based probabilistic numerical solvers for ordinary differential equations (ODEs) have been established as a flexible and efficient simulation framework with built-in numerical uncertainty quantification. However, problems that are both stiff and high-dimensional remain a challenge, as current methods are either stable and have cubic cost in the ODE dimension, or scale linearly at the expense of stability. In this paper, we close this gap and develop probabilistic ODE solvers that are both stable and scalable. We propose two complementary strategies. First, we develop a matrix-free update step that uses Jacobian-vector products, iterative linear solvers, and stochastic covariance estimation to enable linear scaling, all while retaining stability. Second, we propose iterative re-linearization to further improve stability without sacrificing scalability, turning probabilistic ODE solvers into fully implicit methods. We evaluate the proposed approaches on a range of stiff and high-dimensional problems and demonstrate improved stability and scalability over established probabilistic solvers.

Self-training is a practical way to adapt CLIP with limited supervision, but its performance is highly sensitive to pseudo-label reliability. We identify an important failure mode in CLIP adaptation: temporal prediction instability, where a target sample repeatedly flips its prediction across epochs. We provide a perspective based on cross-modal anchoring mismatch: the decision geometry induced by pretrained text anchors can deviate from the local visual cluster structure of the target domain. This mismatch can make ambiguous target samples susceptible to prediction flips during adaptation, whereas target-aligned anchoring produces more stable pseudo-labels. Motivated by this observation, we propose Stability-Aware Self-Training (SAST), centered on Stability-Weighted Image Prototypes (SWIP). SWIP builds class-wise visual anchors from temporally stable target samples to provide more reliable adaptation references. We further add a lightweight text-side refinement to reduce confusion-induced instability, and combine the two streams with an adaptive fusion rule. Across six standard benchmarks and three adaptation settings, SAST consistently outperforms strong baselines. The results suggest that explicitly modeling temporal stability is an effective route to more robust CLIP adaptation under self-training.

Primitive splatting has become a powerful representation for efficient differentiable rendering, but existing density-control strategies remain largely heuristic and often suffer from uncontrolled primitive growth. In this work, we present \textbf{Moment-Preserving Density Control (MPDC)}, a general theoretical framework for densification in primitive splatting. Unlike conventional density-control strategies that \emph{select first and split later}, MPDC follows a \emph{split-then-select} paradigm. Our key insight is that a principled split should first preserve the current rendering, rather than immediately perturb the image. By matching local moments, MPDC decomposes a primitive into multiple offspring while keeping the rendered output, and hence the loss, unchanged up to higher-order error. Although such a split does not directly reduce the loss, it changes the parameter space: a saddle point in the original parameterization may no longer remain a saddle point after splitting. This provides a distinct saddle-escaping mechanism from prior approaches that move primitives along negative-curvature directions. Building on this view, we derive closed-form splitting rules for different primitive parameterizations and then select only primitives whose moment-preserving splits reduce an upper bound of the loss through a splitting-matrix criterion. Experiments on three diverse primitive splatting methods show that MPDC substantially reduces primitive counts without sacrificing rendering quality, while improving memory efficiency and rendering speed. Code will be released upon publication.

Video anomaly detection (VAD) aims to automatically identify events that deviate from normal patterns in untrimmed surveillance videos. Existing methods universally depend on large-scale annotations or task-specific training procedures, severely limiting their rapid deployment to novel scenes. We observe that intermediate-layer features of pre-trained multimodal large language models (MLLMs) already encode rich anomaly semantics, yet existing approaches rely on the language output pathway and fail to exploit the geometric discriminability latent in these representations. Based on this finding, we propose SphereVAD, a fully training-free, zero-shot VAD framework that recasts anomaly discrimination as von Mises–Fisher (vMF) likelihood-ratio geodesic inference on the unit hypersphere, unleashing latent discriminability through principled geometric reasoning rather than learning new representations. Specifically, SphereVAD first applies Fréchet mean centering to unfold feature distributions and eliminate domain biases, then employs Holistic Scene Attention (HSA) to reinforce feature consistency using cross-video priors, and finally performs vMF-guided Spherical Geodesic Pulling (SGP) to align ambiguous segments with directional prototypes on the spherical manifold. This training-free pipeline requires only minimal synthetic images for calibration. SphereVAD establishes new state-of-the-art results among training-free approaches on three major benchmarks and remains competitive with fully supervised baselines. Code will be available upon acceptance.

Omnidirectional or 360° cameras provide embodied AI agents with a holistic, wide field-of-view (FoV) perception of the 3D structure of their surroundings. This capability has generated significant interest in applying Multi-modal Large Language Models (MLLMs) to omnidirectional spatial reasoning. However, most MLLMs are primarily trained on conventional 2D perspective images and therefore struggle with the severe distortions and wrap-around discontinuities introduced by spherical geometry. As a result, enabling MLLMs to generalize effectively to non-Euclidean 3D spaces without retraining remains an open challenge. In this paper, we propose SphMind, a novel, training-free, and plug-and-play framework designed to bridge this gap. The central idea of SphMind is to decouple semantic perception from geometric reasoning. Instead of requiring MLLMs to learn complex spherical geometric principles internally, the framework preserves their strong semantic understanding while handling geometric reasoning externally. To achieve this, we introduce a Spherical Harmonics-based Spatial Graph (SHSG) that models spatial relationships using equivariant transformations on the sphere. We further integrate this with Inference-Time Geometric Grounding (IGG), a model-agnostic closed-loop optimization process that aligns the internal representations of MLLMs with spherical geometric constraints during inference. Extensive experiments on three benchmark datasets demonstrate the effectiveness of SphMind. Without any additional training, the framework achieves more than 21.4% average improvement in directional reasoning on MP3D and Stanford2D-3D, outperforms all prompt-engineering baselines by 8.7% on the real-world ODI-Bench dataset, and achieves 5.9× higher rotational invariance under panorama rotations compared to existing baselines, all without dataset-specific tuning. We also validate the framework on real-world in-the-wild captures, where SphMind successfully resolves directional reasoning queries that baseline vision-language models fail to answer correctly.

Machine unlearning in large language models (LLMs) aims to remove specific learned knowledge, capabilities, or behaviors while preserving model utility. Existing methods have mostly focused on the design of unlearning objectives, while largely overlooking the intrinsic matrix structure of transformer weights. In particular, standard optimizers operate on independent scalar coordinates, thereby ignoring this matrix structure. We propose \emph{spectral unlearning}, an update framework that combines standard unlearning objectives with matrix-structured optimization. We show that the spectral update corresponds to the exact steepest-descent direction for transformers composed of linear layers with normalized inputs and outputs. Our approach is applied on top of existing unlearning methods and evaluated across complementary axes of memorization, privacy, and utility, summarized via their harmonic mean. Spectral unlearning consistently outperforms a broad range of baselines and closes most of the gap to a retain-trained reference.

Model merging enables the integration of multiple expert models with different capabilities into a unified model. In practical deployments, new expert models are often continuously updated and arriving. Meanwhile, due to the training and communication overhead of large language models (LLMs) and multimodal large language models (MLLMs), Low-Rank Adaptation (LoRA) is widely used to adapt large models to specific domains due to its efficiency. This scenario brings a new challenge: continual LoRA merging of deployed MLLMs to load new task capabilities. Naive merging faces catastrophic forgetting and parameter conflicts. We first observe the distinct features of the LoRA direction and layers in MLLM LoRA tuning. Based on this, we propose our Spectral Rank Calibration Merging (SRC-Merging), a data-free and train-free continual LoRA merging method tailored for MLLMs. SRC-Merging regards the deployed LoRA as a compressed spectral memory and performs calibrated rank-level merging for each incoming adapter. It aims to preserve accumulated multimodal knowledge while avoiding over-suppression of new vision-language task updates during continual merging. Extensive experiments on multiple multimodal tasks have demonstrated that our method significantly outperforms state-of-the-art merging methods.

Modular networks pursue specialization through learned routers, gates, and load-balancing losses. However, at matched total-parameter budgets, learned routers can underperform equal-weight No-Routing baselines. Is the bottleneck the routing algorithm, or the alignment between training-signal granularity and the target categories? We show that when each training and inference unit carries one coarse category tag, a fixed parameter-free routing scheme (SpecDrop) induces branch-category specialization and matches or exceeds the learned-routing baselines we evaluate at matched parameter budget; when this alignment breaks, the scheme matches the multi-branch No-Routing baseline. SpecDrop assigns each of $K$ branches weight $p_{\mathrm{a}}$ for its category and a small leakage $p_{\mathrm{i}}{>}0$ otherwise, merged through a category-independent fixed denominator that calibrates merged-branch magnitude to single-branch scale, with no learnable routing parameters or auxiliary losses. Category labels are required at inference, drawn from dataset metadata. On vision tasks where each image has one superclass label, SpecDrop reaches $\mathbf{79.23\%}$ on CIFAR-100 with ResNet-110, outperforming dense by $+4.75$, and $\mathbf{79.89\%}$ on ImageNet-1K with Vision Transformer ViT-S/16, outperforming the matched-supervision No-Routing baseline with a shared expert by $+6.53$. SpecDrop reaches the highest top-1 among the multi-branch routing baselines we evaluate at this parameter budget (Soft MoE and Mod-Squad on ImageNet). On NLP tasks where training units span multiple categories, SpecDrop matches the multi-branch No-Routing baseline on both SlimPajama-6B language modeling with a 30M Transformer and SuperNI instruction tuning over Llama-3.2-1B with LoRA. Furthermore, SpecDrop matches or outperforms multi-branch routing baselines including Demix and LoRAMoE, and is statistically tied with HydraLoRA. Per-branch pruning sensitivity reveals four specialization regimes ranked by category clarity: strongest on ImageNet-1K, strong on CIFAR-100, weakening on SlimPajama, and anti-aligned on SuperNI LoRA. Granularity alignment, not algorithm choice, localizes when routing helps. Code: https://anonymous.4open.science/r/C30862.