NeurIPS 2026 论文列表

Post-hoc image explanations usually reveal *where* a model looks through spatial heatmaps, or *which channels* matter through global channel scores. This separation is limiting for multi-spectral data, where channel relevance is often spatially localized and physically meaningful. We introduce **Spectral-Spatial Interpretation (SSI)**, a full-grid attribution method over position-channel players (g, c). For each spatial position, SSI fixes the observed surrounding context, defines a conditional game over the channels at that position, and applies the Shapley value to this local game. The result is a full spectral-spatial attribution matrix rather than a single heatmap or a global channel ranking. SSI has a clear local semantics: it satisfies Shapley fairness within each position, and its channel aggregation exactly recovers the position-level contextual contrast obtained by replacing that entire position with the baseline. We also distinguish SSI from context-averaged Owen-style hierarchical accounting, which respects the same position-channel grouping but averages over external spatial coalitions. On HLS Burn Scar, SSI outperforms an MC-Owen estimator across retained attribution metrics while using only about 6% of its evaluation budget. Across CUB-200, EuroSAT, and HLS Burn Scar under both per-channel mean and zero baselines, SSI achieves the best joint deletion and insertion AUC over dense, flat, and rank-1 baselines, yields the strongest spectral hit rates and channel deletion, and leads spatial localization on both annotated datasets.

Large language models increasingly use external tools such as web search and document retrieval to solve information-intensive tasks. However, multi-hop tool use in complex tasks introduces substantial latency, since the model must repeatedly wait for tool observations before continuing. We study how to accelerate such trajectories without changing the final trajectory the model would have taken without acceleration, assuming access to faster but less reliable speculator tools. We develop a theoretical framework for lossless speculation in multi-hop tool-use settings, characterizing the optimal achievable latency gain. We propose SpecHop, a continuous speculation framework that maintains multiple speculative threads, verifies predicted observations asynchronously as target tool outputs arrive, commits correct branches, and rolls back incorrect ones. This preserves task accuracy while reducing wall-clock latency. We show that SpecHop can approach the oracle latency gain with sufficient active threads. Empirically, we evaluate SpecHop on retrieval-augmented multi-hop tasks and find that its latency gains closely match theoretical predictions, reaching up to 40% latency reduction in some settings.

Software requirements are usually written in natural language, which is essential for human communication but insufficient as a target for machine-checked correctness. While natural-language descriptions can state intended functionality, verification requires formal specifications with explicit types, relations, quantifiers, guards, witnesses, and edge cases. To bridge this divide, we study the Formal Specification Synthesis task: given a natural-language programming requirement and fixed formal signatures, synthesize a proof-assistant specification for the intended input-output relation. This task is difficult for two primary reasons. First, a large representation gap separates informal natural language from the rigorous structures needed by a proof assistant. Second, generated formal specifications are notoriously hard to evaluate, as proving full equivalence between specifications is often too complex to serve as routine feedback. To address these intertwined challenges, we propose SpecBridge, a reconstruction-guided formalization framework. At its core, SpecBridge introduces a Natural-Language Formalization Plan (NLFP), a semi-structured, readable intermediate representation that captures key formalization choices before decoding to a target specification language. Through reconstruction-guided pattern mining, SpecBridge learns these NLFPs by reconstructing formal specifications to discover reusable patterns, ultimately transferring them to improve natural-language task generation. Furthermore, to tackle the evaluation bottleneck, we introduce a multi-layer evaluation protocol for formal-specification correctness. This protocol utilizes LLM Specification Alignment as an LLM-based judge, alongside Test-Driven Computable Validation and Test-Driven Proof Validation, which approximate "running" testcases on formal specifications by translating propositions into executable pre/post checks and proving testcase obligations in Lean. In our experiments, SpecBridge outperforms the few-shot + CoT baseline by 6.1% on LLM Specification Alignment, 15.2% on Computable Validation Pass Rate, and 28.4% on Proof Validation Pass Rate. These results demonstrate that the NLFP bridge and our multi-layer evaluation protocol make formal specification synthesis significantly more reliable.

Multi-objective Bayesian optimization (MOBO) is commonly approached through specialized acquisition functions or scalarization schemes designed to explicitly account for trade-offs among non-preferential objectives. In this work, we show that such complexity might be unnecessary. We propose a framework that extends standard single-objective acquisition functions directly to the multi-objective setting through a hypervolume-based transformation. We further extend hedge strategies for acquisition functions, which are typically used only in single-objective optimization, to the multi-objective regime. Our approach requires minimal modification to existing Bayesian optimization pipelines and avoids the need for bespoke multi-objective formulations. We demonstrate how a broad class of commonly used single-objective acquisition functions and hedge strategies can be adapted in a principled manner to handle multiple objectives, while preserving their intuitive interpretation and computational efficiency. Empirically, we evaluate the proposed methods across a range of synthetic and real-world multi-objective benchmarks. Despite their simplicity, our extensions consistently match or outperform more complex state-of-the-art MOBO methods in terms of optimization performance and sample efficiency. These results suggest that effective multi-objective Bayesian optimization can be achieved by reusing and carefully extending well-established single-objective acquisition strategies, offering a simpler and more flexible alternative to existing approaches.

Computing accurate geometry from multi-view images is a fundamental problem in computer vision. In this work, we address the Multi-View Stereo (MVS) setting, where camera parameters are assumed to be known. Most existing learning-based methods rely on per-view cost volumes as a camera-induced prior, effectively casting the problem as a sequence-to-one mapping that predicts depth only for a designated reference view. We instead reformulate MVS as a sequence-to-sequence task, enabling the simultaneous prediction of depth maps and point maps for all input views. To this end, we propose a global transformer-based architecture with two key components that explicitly utilize camera-induced priors. First, we employ ray-map embeddings to inject camera parameters into image patch tokens, making the transformer architecture camera-aware for geometry prediction. Second, we replace conventional per-view cost volumes with a unified global cost-volume representation that jointly captures 3D structure across all views. Extensive experiments on multiple public benchmarks demonstrate that our approach achieves state-of-the-art performance, surpassing both multi-view stereo and feed-forward reconstruction baselines.

Multi-modal large language models (MLLMs) achieve strong vision–language reasoning ability but remain vulnerable to jailbreak attacks that exploit subtle cross-modal cues to bypass safety mechanisms. Detecting such attacks is difficult because harmful data are scarce and rapidly evolving, while detectors trained on known patterns often fail to generalize. In this paper, we cast multi-modal jailbreak detection as self-supervised deviation modeling and learn attack-agnostic signals from benign data only. We propose SelfGuard, a self-supervised framework, which models benign multi-modal regularities and learns to score structured violations via controlled pseudo-harmful deviations. SelfGuard synthesizes pseudo-harmful samples via controlled transformations that target three signature cues, cross-modal semantic inconsistency, format manipulation, and semantic toxicity. It then learns complementary deviation statistics through multi-task self-supervised learning, including misalignment discrimination, reconstruction-based toxicity modeling, and contrastive format modeling. At inference, SelfGuard performs multi-view deviation estimation by aggregating task-specific deviation scores to identify departures from benign multi-modal regularities. Extensive experiments on various benchmarks show the effectiveness of our method.

Group-relative RL training (GRPO) samples a small group of parallel rollouts for every training prompt and uses their within-group reward spread to compute per-trajectory advantages. In agentic environments each rollout is a long multi-turn dialogue with one LLM call per step, so this multi-sample multiplier dominates the total training cost. When every rollout of a prompt ends with the same reward, the group has zero reward variance and contributes no gradient, so the extra rollouts add no information; such groups are common in practice (typically around 40% of all groups), so the wasted-compute fraction is substantial rather than marginal. Existing methods filter such groups at the prompt level, either after their rollouts are paid for or before any rollout begins, but both decide without using information that becomes available during the rollout itself. We instead ask whether the in-group divergence between the partial trajectories at an intermediate step can already predict that the group will be zero-variance: when the parallel rollouts have already converged on the same action prefix, the group is on track to produce a single reward, and we can stop early. We propose a one-parameter gate that stops a group when the mean pairwise prefix edit distance between its partial action sequences falls below a threshold. On a 60-iteration on-policy GRPO run on ALFWorld with Qwen2.5-7B, averaged over four random seeds, the gated arm finishes 10.7% faster in wall-clock (bootstrap 95% CI excludes 0) and shifts held-out success rate on 50 unseen tasks by +2.5 pp, with the held-out gain tracing to a measurable reduction in zero-advantage gradient-batch dilution.

Multimodal Large Language Models (MLLMs) have made substantial progress in egocentric video understanding, but their ability to reason cooperatively from multiple embodied viewpoints remains largely unexplored. We study this problem through multi-robot cooperative dynamic spatial reasoning, where a model must answer spatial, temporal, visibility, and coordination questions by integrating synchronized egocentric videos from a team of moving robots. To support this setting, we introduce CoopSR, the first benchmark for this task, together with EgoTeam, a multi-robot egocentric QA dataset. EgoTeam contains 114,227 QA pairs spanning 19 question types, four difficulty tiers, and three team sizes in Habitat and iGibson, along with a real-world test set of around 2,326 QAs collected using two quadruped robots. We further propose SP-CoR (Spectral and Physics-Informed Cooperative Reasoner), an MLLM framework for fine-grained cooperative spatial reasoning. SP-CoR combines dynamics-aware multi-robot frame sampling, spectral- and physics-guided view fusion, and physics-aligned prompt distillation, enabling the model to benefit from privileged robot-pose supervision during training while requiring only egocentric videos at test time. Across 22 MLLM baselines, SP-CoR consistently improves cooperative reasoning, outperforming the strongest fine-tuned baseline by +3.87% on Habitat and +7.12% on iGibson. It also shows stronger generalization to unseen team sizes and real-world robot tests. We will release the benchmark and code.

As multimodal LLMs increasingly emphasize video and audio, a common assumption is that solving such tasks calls for native omnimodal models. We show that this is not always necessary: coding agents equipped with only text+vision and a sandboxed tool-using interface can perform competitively with, and in several settings outperform, SOTA native omnimodal models and predefined multimodal agent scaffolds across multiple audio-video benchmarks. Our trajectory analysis suggests that their advantage comes from coding agents writing code and orchestrating tools to retrieve relevant content from transcripts, frames, and other non-textual modality signals from raw inputs. This effectively converts omnimodal tasks into evidence retrieval and information processing problems, avoiding the inefficiency of ingesting entire videos or audio streams into context. To characterize remaining limitations, we propose a failure taxonomy and a process-level analysis of tool-use traces, and find that simple skill injection, including human-written skills and self-distilled skills from execution logs, can substantially improve performance over the no-skill baseline. To examine whether such capability can be elicited in open-source models, we further introduce Code-X, a complete training recipe with the OmniCoding trajectory dataset and verifiable reward, providing an exploratory baseline on Qwen-3.5-9B and Qwen-3.6-27B. Finally, given the maturity of many-modality understanding, we argue that the more meaningful frontier lies in many-modality processing, and introduce TerminalBench-O, the first process-level benchmark designed for coding agents on real-world omnimodal processing tasks. Together, our findings open new directions for omnimodal content processing and evaluation.

We present a novel method for learning interpretable representations of progressive time series, that is, data capturing irreversible state transitions such as degradation or task completion. Our approach uses a self-supervised contrastive objective to learn a low-dimensional latent space where progression manifests along manifolds anchored by fixed prototype vectors. This structure yields latent-space indicators that quantify progression in a human-meaningful way without proxy labels. We evaluate the approach against the state of the art on diverse domains, including industrial degradation, robotic tasks, and neural activity, validating three key capabilities: (1) end-state prediction, (2) multi-step forecasting, and (3) interpretable phase separation. Our method matches or improves over black-box counterparts on all of these while providing transparency about the underlying mechanisms. A simple linear regressor on top of the learned indicators is competitive with deep architectures, providing direct quantitative evidence that the underlying state is encoded in a geometrically accessible form. Code is available at https://anonymous.4open.science/r/LRPTS-9300/README.md.

Multi-objective optimization aims to solve problems with competing objectives. Evaluating such problems is often slow or expensive, limiting the budget of evaluations. In many applications, historical data from related optimization tasks is available and can be leveraged via meta-learning to accelerate optimization. Bayesian optimization, as a promising technique for expensive black-box problems, has been extended independently to meta-learning and multi-objective optimization, but methods that simultaneously address both settings remain largely unexplored. We propose SMOG—a scalable and modular meta-learning model based on a multi-output Gaussian process—that explicitly learns correlations between objectives. SMOG builds a structured joint Gaussian process prior across meta- and target tasks and, after conditioning on metadata, yields a closed-form prior for the target task. This construction propagates metadata uncertainty into the target surrogate in a principled way. SMOG supports hierarchical, parallel training, achieving linear scaling with the number of meta-tasks. The resulting surrogate integrates seamlessly with standard multi-objective Bayesian optimization acquisition functions. We demonstrate that our method is consistently competitive, delivering strong data efficiency across representative benchmarks and applications.

Lifelong cross-modal hashing demands incremental updates as new tasks arrive, yet existing regularization-based methods such as EWC and SI share a fundamental weakness: their effective regularization strength is entangled with task scale, as the empirical Fisher Information Matrix (FIM) scales linearly with sample size. This forces costly per-task hyperparameter tuning in streaming deployments to avoid over- or under-regularizing tasks of varying sizes. We propose \textsc{Sing-CH}, a lifelong hashing framework built on information geometry. At its core is a Gibbs probabilistic reconstruction of the pairwise objective, which gives the parameter space a statistical manifold structure and yields a natural $1/n_t^2$ normalization. We prove this normalization renders the FIM scale-agnostic: its spectral properties remain independent of task size even under diagonal and K-FAC approximations. Beyond this invariance, a Forgetting-Rigidity trade-off theorem shows that natural gradient applied to the composite objective reintroduces scale dependence. This makes our decoupled design, pairing the FIM as regularizer with Adam for optimization, a structural necessity rather than a convenience. Experiments on three benchmarks demonstrate that \textsc{Sing-CH} outperforms 11 baselines by 4.1\% average mAP without per-task tuning. Under extreme task-scale imbalance, degradation is only 2.0\% versus 3.9\% for the strongest baseline, providing the first empirically grounded solution to scale-agnostic lifelong retrieval.

Cross-city transfer leverages labeled data from well-instrumented cities to improve prediction in label-scarce ones, but remains challenging when cities adopt incompatible partitions with no ground-truth region correspondences. Even with expressive GNN encoders, transfer quality varies dramatically across methods sharing nearly identical backbones---indicating that alignment design, not encoder capacity, is the binding constraint. Existing paradigms exhibit complementary failure modes: heuristic anchor matching collapses to hubness under unequal partitions, while distribution-level matching over-mixes embeddings under heterogeneity. Both stem from a single missing primitive---explicit, mass-controlled soft correspondence between unequal region sets. The challenge intensifies in multi-source transfer, where independent source-to-target alignments yield conflicting gradients and source domination. We propose SCOT, which adapts entropic OT to this regime through three application-specific designs: an OT-weighted contrastive objective that resolves the geometric--semantic tension, a one-sided cycle regularizer respecting the rectangular $n_s\!\neq\!n_t$ geometry, and---as our central contribution---a shared prototype hub coordinated through balanced entropic OT under a target-induced prior, bypassing the source-selection problem in label-scarce regimes. Across real-world cities and tasks, \scot consistently improves transfer accuracy, achieving 5--50\% relative MAE/MAPE reductions over the strongest baseline, with learned couplings and hub assignments quantitatively confirming that the diagnosed failure modes are resolved.

The exponential growth of machine learning submissions has strained the traditional peer review process, resulting in slow feedback loops for authors and an immense burden on reviewers to rigorously audit technical soundness and verify literature. To address this, we introduce ScholarPeer, a multi-agent framework designed to operationalize the rigorous auditing workflow of a senior researcher. Rather than attempting to replace human judgment, ScholarPeer serves as a co-scientist: acting as a mentor for rapid author iteration prior to submission, and as an active verification assistant that augments human reviewers. The framework structurally decouples contextualization from critique by deploying a sub-domain historian to synthesize the field's trajectory, a baseline scout to proactively hunt for omitted state-of-the-art comparisons, and a multi-aspect Q&A engine that deeply audits technical soundness—scrutinizing internal logical consistency, experimental validity, and mathematical rigor—while cross-referencing claims against top-tier academic venues. We comprehensively evaluate ScholarPeer on $\sim$1,800 ICLR submissions spanning 2020 through 2025. Our results show that ScholarPeer achieves significant win-rates against state-of-the-art fine-tuned models and search-augmented agentic baselines.

Agents that perform state estimation using pretrained perception models, e.g., Vision-Language Models (VLMs), face the fundamental challenge that predictions are uncertain. Still, most approaches determinize these predictions into a single best-guess state and make decisions as if it were ground truth. This is brittle because when the guess is wrong, the agent's decisions may be unsafe or infeasible. At the other extreme, maintaining a full distribution over symbolic states is often intractable. We propose a middle path, drawing on the underlying probabilities the perception model already produces (e.g., a VLM's output-layer token distributions) to construct a θ-confidence Minimal Belief Set (MBSθ), a minimal subset of states whose cumulative probability meets a user-defined threshold. The minimality of an MBSθ ensures that the agent considers only the most likely states, enhancing robustness in downstream decision-making while maintaining computational tractability. Using a novel three-layered belief system, we efficiently approximate individual state probabilities from per-feature predictions and task-level constraints, on which we base our MBSθ construction, while maintaining only a lean factored representation of the belief. Our algorithm, PRaVaSS, takes advantage of the unique structure of this belief system to construct an MBSθ with correctness guarantees and polynomial complexity. Using a VLM as the backbone of our state estimator, we validate our method's theoretical properties, scalability with problem size, and improvement of downstream decision-making when used in a task planning pipeline.

Real-world multi-view data always suffer from imperfect information problem, where the view-specific observations are absent (\ie, Incomplete Views, IV) and cross-view correspondences are mismatched (\ie, Noisy Correspondences, NC) for certain instances. As a remedy, numerous IV- and NC-oriented multi-view clustering (MvC) methods have been proposed, which however require either reliable correspondences or sufficiently complete instances, thus stopping short of addressing the imperfect information problem. In contrast, we observe that both IV and NC challenges originate from the same issue of imperfect cross-view counterpart information, where the counterpart of an anchor instance in another view might be either unavailable or unreliable. Based on the observation, we propose a novel robust MvC framework, termed Posterior-guided Latent Counterpart Inference (PLCI), which could handle both IV and NC in a unified manner. Specifically, PLCI formulates the desired cross-view counterpart of each anchor instance as a latent variable, and integrates both instance-level reliability and prototype-level semantic transport to infer the posterior distribution of the latent counterpart. Extensive experiments on six widely-used multi-view datasets against 10 state-of-the-art MvC methods demonstrate the effectiveness of PLCI for tackling the imperfect information problem. The code will be released upon acceptance.

Aligning large language models (LLMs) to diverse user preferences is fundamentally hindered by standard alignment paradigms that optimize for monolithic users. In this work, through empirical studies, we first discover a massive, untapped performance headroom for personalized generation through test-time scaling. We demonstrate that personalized generation is uniquely suited for test-time sampling methods like Best-of-$N$ (BoN) because it can be viewed primarily as a candidate matching problem rather than a generator capability bottleneck. While standard reward models can theoretically exploit this headroom, their massive parameter counts introduce a prohibitive computational bottleneck. To overcome this limitation, we propose a parameter-efficient framework utilizing million-parameter scale Multi-Layer Perceptron (MLP) ranking models. Our personalized ranking model directly reuses the internal embeddings of the base generator with minimal overhead. By scaling train-time data to provide fine-grained personalized preferences, this million-parameter ranking model accurately scores large candidate pools and can seamlessly guide generation to reduce the cost of materializing $N$ candidates. Extensive experiments on nine datasets across three different personalized generation settings demonstrate that our framework effectively exploits the discovered headroom. Remarkably, our million-parameter MLP performs competitively with billion-parameter reward models explicitly finetuned on the same tasks, while requiring only a negligible fraction of the inference cost.

In a 100-body robot co-design benchmark, the height threshold that resets a falling robot changes which bodies appear optimal. We train PPO controllers for DERL-evolved UNIMAL morphologies under Standard (τ = 0.20), Strict (τ = 0.50), and Reset-free (τ = 0) protocols, then evaluate every policy under a common protocol. Strict and Reset-free training share only 5 of their top-10 bodies and have low rank agreement (Spearman ρ = 0.21). The effect is family-specific, not a uniform penalty: averaging per-body retention ratios, Reset-free training reduces Floor bodies' per-step reward by 45% relative to Standard training but reduces variable-terrain bodies by 16%. Under Strict, the losses are 22% for Floor and 34% for variable-terrain bodies, while multi-task bodies are most stable across protocols. Blocked ANOVA and morphology-level permutation tests support the protocol-by-family interaction. A SAC validation panel planned before analysis finds the same Strict-versus-Reset-free family ordering under a different optimizer. Selecting bodies by their minimum reward across the three training protocols recovers ≥91% of every single-protocol oracle's top-k average reward (k ∈ {5, 10, 20}; all bootstrap lower bounds ≥84.3%). Co-design benchmarks should state the reset rule, audit ranking sensitivity, and report protocol-robust selections when choosing bodies.

Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This has been cited as a limit case for CoT monitorability: if the surface tokens carry no information about the reasoning, behavioral oversight has nothing to read. But hidden from the output is not the same as hidden from us. On three task families spanning fact retrieval, parallel numeric composition, and string manipulation, two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way. Attention routes the question through the filler region to the answer, KV-cache transplants at only filler positions causally swap the model's output between examples, and logit-lens readouts reveal a temporal structure in which retrieved facts appear early and their composition crystallizes in late layers. We then introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80–95\% accuracy with the strongest LLM judge across both models and all three tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream. This suggests that monitorability should be understood as a property of the model's full computational trace, not just its surface tokens.