NeurIPS 2026 论文列表

Video diffusion transformers rely on self-attention over long spatio-temporal token sequences, making inference expensive even when sparse attention reduces the number of computed blocks. This paper studies a systems bottleneck that appears in dynamic sparse video attention: per-head sparsity can vary widely across denoising steps, layers, and prompts, so equal-head parallel execution leaves some GPUs waiting for dense heads assigned to other GPUs. We propose AsymHP, a runtime system that uses the previous denoising step's per-head sparse density to place a non-uniform number of heads on each GPU. AsymHP combines this lightweight online density estimate, a cost model for mask construction and sparse attention computation, and an asymmetric head redistribution primitive that avoids padding variable-size head shards into symmetric collectives. In our evaluation on H100 GPUs, AsymHP improves sparse attention latency by up to $1.54\times$ without changing the underlying sparse-attention algorithm.

Films introduce characters incrementally as they progress. We study the problem of identifying principal characters and assigning their names directly from the movie itself, without using external cast lists, actor photographs, or pre-built character banks. We introduce the character naming task: given a name mentioned in dialogue and the associated video context, the model must determine which visible character bears that name. This task is challenging because name resolution often depends on cinematic cues such as shot–reverse-shot structure, self-introduction, direct address, third-party reference, gaze, and temporal continuity. We provide a dataset and annotations for evaluating character naming and first character appearances, develop a video-and-dialogue model for the task, and compare it with proprietary multimodal large language models. We further introduce a progressive framework for building a character bank as the film unfolds. Together, these contributions support online, human-like movie understanding in which character identities are inferred from the film itself.

Computer-use agents turn visual predictions into executable GUI clicks, so uncertainty estimates must support rejection, calibration, miss-severity ranking, and spatial safety regions. Existing post-hoc uncertainty quantification (UQ) evidence is fragmented across isolated model and dataset pairs, leaving unclear whether UQ rankings remain stable when the agent, benchmark, or observable interface changes. We present a cross-regime benchmark for post-hoc UQ in single-step executable GUI grounding, covering a 27-method, seven-family open-weight matrix over 4 GUI-grounding VLM agents and 4 datasets, plus an 8-method API-compatible closed-source matrix across 3 frontier vendors where logits, hidden states, and attention maps are unavailable. The main finding is selective transfer: UQ rankings are stable across datasets for a fixed model, but degrade across model classes and observable interfaces. Hidden-state and density methods form the most stable open-weight family, while CoCoA-1MCA, Focus, sampling-based scores, and verbalised self-assessment win in specific regimes. Ranking transfer is strongest within a fixed model across datasets, reaching Spearman $\rho=0.969$ and averaging $\rho=0.705$ over 120 open-weight pairs. In contrast, open-weight to closed-source transfer is the strongest break: cross-tier Spearman $\rho$ averages only $+0.08$ over 12 vendor x dataset pairs on the shared 8-method intersection, indicating that UQ recommendations do not survive removal of internal model signals. Model-class transitions further reshape UQ preferences: attention, verbalised, and VLM-native families lose AUROC on every dataset, while density methods remain stable; a scale-only baseline shows that logit-family degradation on ScreenSpot-Pro is partly confounded by scale rather than grounding fine-tuning alone. Finally, conformal click regions show that score-level discrimination is not enough for deployment: locally weighted disks can shrink radii by 40-60% when the plug-in UQ is calibrated, but coverage can degrade under calibration-test or interface mismatch. We release per-item records, calibration/test splits, UQ scores, and analysis scripts as a reproducible basis for regime-aware UQ selection in GUI agents.

**Faithfulness**—how precisely a generated image aligns with its prompt—is increasingly central to the real-world utility of text-to-image (T2I) models. Existing faithfulness benchmarks, however, rely on simple atomic instructions, where top-tier systems already achieve near-perfect scores. As T2I models enter creative workflows, users increasingly issue multi-faceted requests involving intricate spatial relationships, stylistic constraints, and complex text rendering. In this setting, a single binary VLM-judge score no longer reveals **which** specific constraints the model fails to satisfy. We introduce **Arena-T2I Hard**, a 310-prompt stress benchmark drawn from real arena T2I logs, with approximately 30 decomposed yes/no constraints per prompt spanning six categories, including text rendering. The strongest closed-source system we evaluate reaches **0.855**, with a **33 pp** performance gap across 11 systems, demonstrating substantial discriminative power. Moreover, high public-arena rankings fail to predict faithfulness, confirming that holistic Bradley–Terry (BT) preference scores prioritize aesthetics over fine-grained prompt adherence. We propose a **dependency-aware checklist reward** that decomposes each prompt into a DAG of yes/no questions and zeroes descendants of failed parents, turning faithfulness into a per-constraint training signal. Combined with a BT aesthetic reward via **group-decoupled normalization (GDPO)**, which standardizes each reward within its rollout group so neither collapses, our recipe attains a strictly better faithfulness–aesthetics trade-off on **SD3.5-Medium** and **FLUX.1-dev** under MMRB2 pairwise comparisons than every single-reward, naive weighted-sum, or 4-reward BT-ensemble baseline. We release **Arena-T2I Hard** as a public stress benchmark.

Despite the growing popularity of Multimodal Domain Generalization (MMDG) for enhancing model robustness, it remains unclear whether reported performance gains reflect genuine algorithmic progress or are artifacts of inconsistent evaluation protocols. Current research is fragmented, with studies varying significantly across datasets, modality configurations, and experimental settings. Furthermore, existing benchmarks focus predominantly on action recognition, often neglecting critical real-world challenges such as input corruptions, missing modalities, and model trustworthiness. This lack of standardization obscures a reliable assessment of the field's advancement. To address this issue, we introduce MMDG-Bench, the first unified and comprehensive benchmark for MMDG, which standardizes evaluation across six datasets spanning three diverse tasks: action recognition, mechanical fault diagnosis, and sentiment analysis. MMDG-Bench encompasses six modality combinations, nine representative methods, and multiple evaluation settings. Beyond standard accuracy, it systematically assesses corruption robustness, missing-modality generalization, misclassification detection, and out-of-distribution detection. With $7, 402$ neural networks trained in total across $95$ unique cross-domain tasks, MMDG-Bench yields five key findings: (1) under fair comparisons, recent specialized MMDG methods offer only marginal improvements over ERM baseline; (2) no single method consistently outperforms others across datasets or modality combinations; (3) a substantial gap to upper-bound performance persists, indicating that MMDG remains far from solved; (4) trimodal fusion does not consistently outperform the strongest bimodal configurations; and (5) all evaluated methods exhibit significant degradation under corruption and missing-modality scenarios, with some methods further compromising model trustworthiness. We release MMDG-Bench to enable more rigorous, reproducible, and directly comparable evaluation, addressing current limitations in evaluation practices and providing a stronger foundation for future progress in multimodal domain generalization.

Text-to-image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural language prompts. Yet it remains unclear whether their success reflects genuine causal understanding or sophisticated pattern matching over visual-textual correlations. Inspired by Russell’s inductivist turkey, we introduce Counterfactual-World (CF-World), a counterfactual benchmark designed to probe whether text-to-image models can generate images under rules that systematically contradict real-world priors. CF-World organizes each scenario into three progressive levels: factual generation under ordinary world knowledge, explicit counterfactual generation with direct visual instructions, and implicit counterfactual generation requiring causal deduction from altered rules. We evaluate both open-source and closed-source T2I models using a Vision Language Model(VLM)-based evaluator (CF-Eval), and introduce two metrics: Prior Resistance Rate (PRR), which measures a model’s ability to overcome entrenched real-world priors, and Reasoning Retention Rate (RRR), which assesses whether models can maintain reasoning-dependent counterfactual generation without explicit visual cues. Experiments show that all models exhibit sharp degradation from factual to counterfactual settings. Further analyses suggest that these failures arise because current T2I models encode world knowledge and visual appearances as tightly coupled patterns. They rely heavily on frequent visual co-occurrences in their training data, causing them to default to familiar commonsense priors when tasked with generating counterfactual worlds.

Multimodal benchmarks are central to evaluating whether models can reason over information from multiple modalities. However, existing literature has revealed that some benchmarks may contain shortcut items that can be answered from a single modality or from answer-option regularities alone, thus failing to reflecting models' true cross-modal inference ability. Existing methods for assessing benchmarks typically rely either on intrinsic item structure or extrinsic model response behavior, which are not adequate for a comprehensive diagnosis. To fill in this gap, we propose SRCoD, a structure-response co-calibrated diagnosis framework for item-level diagnosis of modality dependence in multimodal benchmarks. SRCoD estimates sample-wise multimodal information structure from benchmark content and target answers, and uses this signal as a soft structural anchor for modality-conditioned item response modeling. The learned model produces a calibrated item-level diagnostic profile that reflects how strongly an item depends on different forms of modality evidence, enabling fine-grained benchmark analysis beyond aggregate accuracy. SRCoD also provides interpretable attribution of the diagnosis results. Extensive experiments on two complementary protocols, i.e., benchmark-refinement oriented validation and human-labeled item validation, demonstrate that SRCoD outperforms state-of-the-art benchmark diagnosis methods. Our code and data are available at https://anonymous.4open.science/r/SRCoD-61B0.

We study revenue-optimal posted-price design for GPU markets under a finite capacity $B$. We focus on two settings: \emph{sales}, where each purchase permanently consumes a vendor's GPU inventory, and \emph{rentals}, where each job occupies GPUs for a finite duration and then releases them. In both settings, the vendor must commit to an anonymous price menu before observing customers' valuation, GPU demand, and job duration. First, in the sales setting, we show that computing the optimal anonymous static menu is NP-hard, and that the common practice of linear pricing (per-GPU price) can be highly suboptimal. We develop three methods with varying approximation guarantees, all of which outperform linear pricing. Our main method is a convex program (CP) obtained via an ex-ante relaxation of the problem, which obtains a $\tfrac{1-\epsilon}{2}$ approximation to the optimum when the maximum GPU demand $\overline{N}$ satisfies $\overline{N}\leq \epsilon B$. Building on these results, we analyze the \emph{rental} model. We again show that linear pricing (per-GPU-hour price) is highly suboptimal, and design duration-aware posted-price policies based on slot-coupled ex-ante CP. Our \emph{Throttled CP-Menu Posted Pricing} (TCMPP) policy posts the CP-induced menu randomly and achieves a $\frac{1-\epsilon}{4}$ approximation. We further introduce a slackened version of the same convex program, which reserves capacity in each time slot and achieves stronger guarantees in the small job regime. We corroborate our theoretical results with simulations.

Approximate Bayesian inference has driven significant advances in probabilistic machine learning at the basis of modern generative modelling. However, many successful Bayesian methods, such as variational inference or sequential Monte Carlo, often struggle in discrete domains with complex structure. In particular, their application to the challenges of neurosymbolic AI, i.e. learning and reasoning under uncertainty, is largely unexplored. Our contributions are twofold. First, we show that the majority of neurosymbolic problems are problems of posterior inference motivating the use of approximate Bayesian inference for neurosymbolic AI. Second, we adapt variational inference to the neurosymbolic setting by exploiting available symbolic structure. Moreover, we outline what properties a reasonable variational approximation to a neurosymbolic problem should have and how to enforce these properties by connecting to statistical estimation and statistical representation theorems. The result is a principled approach to learning and reasoning at scale that maintains statistical guarantees at test time. This work illustrates the opportunities at the intersection of approximate Bayesian inference, and learning and reasoning under uncertainty.

With the growing number of fine-tuned variants derived from pre-trained models, model merging emerges as a key challenge: effectively integrating task-specific adaptations while preserving performance. However, naive weight averaging often fails when parameter updates are misaligned, revealing the need for a structured space in which merging becomes principled. In this work, we hypothesize that pre-trained models possess an intrinsic low-rank structure. Specifically, we posit that the transformation from input to output is mediated by a latent bottleneck of low intrinsic dimensionality; we leverage this low-dimensional space to construct a model embedding space. This space ensures that fine-tuned updates, i.e., model embeddings, of all adaptations remain confined within the intrinsic structure of the pre-trained model. As model embeddings lie in the same low-dimensional space, model merging via direct averaging becomes more effective. Moreover, we further introduce an aligned subspace of these model embeddings to improve the efficacy of model merging. Experiments across vision and multimodal tasks, including ViT and multimodal LLMs, demonstrate that our approach enables effective model merging and provides a unified geometric view of adaptation and combination in pre-trained models.

Modern video generation models are increasingly hailed as emerging world models with an internalized grasp of physical law. Yet existing benchmarks largely evaluate physical plausibility only at the output level, without verifying whether the model arrives there through a faithful, law-grounded reasoning process. We introduce **Apple-$\pi$**, the first benchmark that anchors video-model evaluation explicitly in physical laws. Apple-$\pi$ comprises three components. **1) Orchard**: a dataset of 400 videos covering ten canonical tasks in classical mechanics. It separates single-law tasks for confounder-free diagnosis from multi-law tasks for probing generalization. **2) Benchmark Protocol**: a three-stage protocol based on scientific reasoning, including Perception, Formulation, and Deduction. It uses *chain-of-frames* prompting on infographic-annotated first frames, treating the generated video as the model's visible reasoning trace. **3) Evaluation Suite**: a hybrid evaluation suite that combines MLLM-based subjective scoring with physics-law-grounded objective measures. This enables stage-resolved diagnosis of not only whether a model fails, but where it fails. Benchmarking 11 models shows that current video models remain far from reliable law-grounded world simulators, with the best video model scoring only 0.473. Our stage-, pillar-, and source-resolved analyses further expose a Perception-to-Formulation-to-Deduction bottleneck, weak multi-law state transfer, and a persistent Sim-to-Real gap. These findings position Apple-$\pi$ as a diagnostic foundation for guiding future video models toward world models with law-grounded physical intelligence.

On-policy self-distillation, where a student is pulled toward a copy of itself conditioned on privileged context (e.g., a verified solution or feedback), offers a promising direction for advancing reasoning capability without a stronger external teacher. Yet in math reasoning the gains are inconsistent, even when the same approach succeeds elsewhere. A pointwise mutual information analysis traces the failure to the privileged context itself: it inflates the teacher's confidence on tokens already implied by the solution (structural connectives, verifiable claims) and deflates it on deliberation tokens (*Wait*, *Let*, *Maybe*) that drive multi-step search. We propose Anti-Self-Distillation (AntiSD), which ascends a divergence between student and teacher rather than descending it: this reverses the per-token sign and yields a naturally bounded advantage in one step. An entropy-triggered gate disables the term once the teacher entropy collapses, completing a drop-in replacement for default self-distillation. Across five models from 4B to 30B parameters on math reasoning benchmarks, AntiSD reaches the GRPO baseline's accuracy in 2 to 10× fewer training steps and improves final accuracy by up to 11.5 points. AntiSD opens a path to scalable self-improvement, where a language model bootstraps its own reasoning through its training signal.

While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it is still unclear VLMs are able to recover temporal dynamics when motion is present. To this end, we introduce the animation-to-code task, as well as a benchmark, \texttt{Animation2Code}, for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. \texttt{Animation2Code} consists of 1,069 web animation videos with diverse visuals and motion patterns, augmented with corresponding HTML/CSS/JavaScript implementations. We further pair the dataset with several novel human-aligned metrics, which allow us to disentangle visual similarity from temporal alignment when comparing rendered animations to ground-truth samples. We benchmark current SOTA model performance on these new samples, and show that current models struggle with temporal consistency, even when achieving high appearance similarity, even in fine-tuning and iterative refinement scenarios. Our benchmark dashboard is available at: \url{https://anim2code-dashboard.vercel.app}.

We propose a new framework for low-precision neural networks based on angular geometry and randomized low-bit estimators. Rather than approximating Euclidean linear operations under limited precision, we reinterpret neural computation in terms of directional similarity and construct quantized random-feature estimators of cosine interactions. Our approach introduces \textit{Angular Layers}, which replace standard linear transformations with low-bit projections that estimate angular similarity in the forward pass, while gradients are computed with respect to the underlying continuous cosine geometry. This decoupling enables stable optimization without straight-through estimators or heuristic surrogate gradients, even under aggressive quantization of both weights and activations. On the theory side, we introduce \emph{G-Gradient} as a gradient proxy for angular layers and show that, in a one-layer planted model, a sufficiently small \emph{G}-gradient can guarantee exact recovery of a target binary solution. We further prove uniform approximation and convergence guarantees that provide a rigorous path from tractable continuous optimization to exact recovery in the original discrete model. Empirically, we instantiate these ideas in \textit{Angular Nets}, including low-precision variants of ResNet, Vision Transformers, and BERT-style encoders, and obtain strong performance across vision and language tasks at substantially reduced precision. Overall, our results suggest that preserving angular structure provides a principled foundation for low-bit deep learning. Our full implementation is available at: \url{https://github.com/GGM2026/GGM}.

Adversarial attacks optimized on a single open-weight LLM can transfer to and jailbreak architecturally different models, allowing an attacker with white-box access to one model to compromise independently deployed systems. This creates a shared vulnerability across models, yet existing defenses are not designed for this cross-model threat. We find that transfer aligns with shared internal representation geometry, making it a natural defense target. We find that cross-model transfer aligns with shared internal representation geometry, making it a natural defense target. \textbf{AnchorRep} targets this geometry directly with a lightweight LoRA adapter that pushes the defended model's internal representations of harmful prompts away from those of a frozen anchor model on the same prompts. Training uses a small set of harmful prompts and no adversarial examples. Across five models and four architectural families, AnchorRep reduces cross-model attack success rate to $\leq$1.1\% on 2{,}000 transferred attacks (0\% on two), including the largest drop on Mistral ($36\% \to 1.1\%$). Existing defenses can reduce transfer, but only at high cost—either inducing up to 77\% degenerate benign output or increasing over-refusal by up to 18\%. Because such degenerate benign outputs are not captured by standard refusal-based metrics, we introduce the \textbf{Benign Garble Rate} to quantify them. Our results suggest that cross-model robustness can be achieved by shaping representation geometry, without requiring attack-specific training. \smallskip \noindent\faGithub~Code, configs and logs: \href{https://anonymous.4open.science/r/AnchorRep/README.md}{anonymous.4open.science/r/AnchorRep}

Echocardiography segmentation plays a central role in quantitative cardiovascular assessment. However, automated echocardiography segmentation remains challenging due to speckle noise, poorly defined anatomical boundaries, acquisition variability, and the limited anatomical coverage of existing benchmarks. Conventional segmentation methods demonstrate limited accuracy under such low image quality and domain shifts. Inspired by the observation that expert cardiologists can maintain reliable anatomical delineation by leveraging their structural understanding of the cardiac anatomy, we propose that incorporating anatomical priors into echocardiography segmentation models can improve both precision and robustness. To this end, we propose AST-Seg, an anatomy-aware spatio-temporal framework for echocardiography segmentation. AST-Seg introduces an anatomy-aware feature encoder that regularizes transformer attention with anatomy-derived spatial-relation priors, and a Deformable Spatio-Temporal Mamba module that captures localized and periodic cardiac motion through adaptive spatial aggregation followed by temporal state-space modeling. In addition, we curate an Echocardiography Anatomy (EA) dataset with pixel-level annotations for 12 clinically relevant cardiac anatomy, enabling comprehensive multi-structure cardiac interpretation. Extensive experiments on in-distribution CAMUS and EA datasets, as well as an out-of-distribution point-of-care echocardiography dataset, demonstrate that AST-Seg provides domain-generalized segmentation with improved precision.

Medical Vision-Language Pre-training (VLP) leverages the semantic richness of radiology reports to learn 3D representations, yet current architectures remain fundamentally anatomy-agnostic. While clinical diagnostics rely on organ-specific context, standard encoders apply a uniform, dense parameterization to all volumetric patches, and existing Mixture-of-Experts (MoE) routers gate on isolated local features without explicit access to the organ composition of the scan or crop. We propose Anatomy-Activated Mixture-of-Experts (A$^{2}$MoE), a framework that incorporates anatomical inductive biases within the model's computational path. A$^{2}$MoE introduces two key components: (1) a Histogram Representation Router (HRR) that conditions expert selection on global visual context and a predicted anatomy-presence histogram, ensuring that parameters specialize in specific anatomical domains; and (2) a mask-free Organ-Query Multi-Scale Projector (OQMSP) that utilizes learnable queries to extract organ-level embeddings for fine-grained alignment without requiring manual annotations at inference. By routing at the patch level rather than the token level, A$^{2}$MoE reduces routing complexity by orders of magnitude while satisfying load balancing by construction. Pre-trained on over 60,000 multi-phase CT scans, A$^{2}$MoE achieves state-of-the-art performance across segmentation, classification, and detection benchmarks. Furthermore, we demonstrate that our anatomically-grounded inductive biases support robust generalization to MRI and ultrasound, establishing a scalable foundation for medical representation learning.

Most black-box optimization methods require extensive hyperparameter tuning, often limiting their ability to generalize across different optimization domains. Foundation models for black-box optimization that learn optimization principles from a large collection of optimization trajectories offer a promising alternative, with the potential to outperform manually designed methods across diverse problem classes. However, prior work has either relied on non-public datasets or on purely synthetic data, limiting reproducibility and generalization to real-world problems. As a result, progress in this area has been constrained by the lack of large-scale, real-world, publicly available pre-training data. We introduce BBO-Pile, the first open-source dataset comprising over 500K optimization trajectories evaluated across 3095 different black-boxes for different optimizers which represents by far the largest public dataset for this task. Using this dataset, we train a family of foundation models at multiple scales, ranging from 2M to 80M parameters and from 200M to 2B training tokens, and study their scaling behavior with respect to compute. Our results demonstrate that large-scale pre-training is a viable and effective approach to imitate black-box optimization methods, paving the way for future research in this direction.

Industrial anomaly detection is evolving from category-specific detectors toward unified models capable of handling multiple categories simultaneously. However, in multi-class setting, greater inter-category diversity not only compels enhanced reconstruction capacity to model normal patterns but also exacerbates the “identical shortcut” problem, wherein anomalies are likewise well reconstructed. Moreover, existing unified frameworks are typically confined to single-modal (RGB) inputs and lack multimodal capability. To address these, we propose an Efficient Cross-modal Feature Reconstruction (ECFR) model for unified multi-class anomaly detection, which harnesses the inherent difficulty of cross-modal reconstruction to alleviate the shortcut issue and amplify anomaly discrimination. The framework is built on two core modules: an Adaptive Feature Interaction and Recalibration (AFIR) module and a Hybrid Attention Convolution (HAC) module. Through feature interaction, fusion, and reconstruction, our model achieves two key outcomes: learning normal patterns of multi-class objects while forcing reconstruction failures for anomalous inputs. Extensive experiments on the MVTec 3D-AD and Eyecandies datasets validate our approach, which achieves state-of-the-art performance in multimodal multi-class anomaly detection while requiring significantly fewer model parameters (44.63 M) and lower computational complexity (15.77 GFLOPs) compared to previous multimodal anomaly detection methods.