NeurIPS 2026 论文列表

Transformer models have demonstrated a remarkable ability to perform a wide range of tasks through in-context learning (ICL), where the model infers patterns from a small number of example prompts provided during inference. However, empirical studies have shown that the effectiveness of ICL can be significantly influenced by the order in which these prompts are presented. Despite its significance, this phenomenon has been largely unexplored from a theoretical perspective. In this paper, we theoretically investigate how positional encoding (PE) affects the ICL capabilities of linear attention transformer models, particularly in tasks where prompt order plays a crucial role. We examine two distinct cases: linear regression, which represents an order-invariant task, and dynamical systems, a classic time-series task that is inherently sensitive to the order of input prompts. Theoretically, we evaluated the change in the model output when two types of positional encoding (one-hot and RoPE) is incorporated and the prompt order is altered. In all cases, the leading dependence on the permutation size and context length is $k/N$, with constants determined by the positional encoding, model weights, input dimension, and task dependent parameters. These theoretical findings are experimentally validated.

This position paper argues that the research community must act to maintain research pipeline reliability despite increasing threats from GenAI. We start in the pre-GenAI era by highlighting some of the prominent critiques of the research pipeline, consisting of: (a) Authoring, (b) Reviewing, and (c) Evaluating researchers based on bibliometrics. Then, we discuss some of the effects GenAI has had on each research pipeline component based on the best available data, limited though it may be. Importantly, due to opaqueness of the submission pipeline, it is difficult to measure both the impact of GenAI and the efficacy of interventions, which leads us to call for greater transparency. Last, we conclude by offering three alternative views and ten paths forward that may mitigate some of the issues we highlight by keeping human researchers as the primary driver of science

The rapid rise of AI has seen the coexistence of open-weight models and closed-source, API-based deployment, each with its own strengths and limitations. A growing share of deployment scenarios needs both local, customizable execution and owner-side accountability, yet neither extreme serves them well. Such scenarios include on-prem enterprise use with sensitive data, edge and offline robotics, and local fine-tuning or retrieval-augmented generation. This position paper argues that the middle regime between closed APIs and open-weight release deserves a principled deployment-level primitive of its own, and proposes OML—Open, Monetizable, Loyal—as a starting point for that primitive. An OML-formatted artifact aims to be simultaneously Open (open-weight and locally executable), Monetizable (usage is accountable to the owner), and Loyal (owner-declared use constraints are enforceable). Throughout, "open" refers to open-weight, locally executable artifacts rather than to fully open-source release, and "loyal" refers to owner-declared use constraints rather than to remote control over a running model. We articulate the desirable properties and threat model for OML, survey theoretical and practical constructions across software, hardware, and cryptographic primitives, and outline an end-to-end deployment protocol together with market-based and policy alternatives. We then describe OML 1.0, a low-overhead instantiation based on AI-native fingerprinting, and characterize the regime in which it provides meaningful guarantees. We close with a research agenda calling on the ML, cryptography, systems, and mechanism-design communities to refine the primitives needed to make OML deployable at scale.

Autonomous machine learning research is moving from a speculative possibility to an emerging research practice. Yet, current conference policies are not equipped to handle this change. This position paper argues that machine learning conferences should introduce an Autonomous Research Track where an autonomous AI system controls claim-shaping decisions. However, to ensure that conferences continue to exist for both science and scientists, our proposal anchors this new track in human judgment and participation. First, we propose a hierarchy of AI involvement from incidental AI use to autonomous research, grounded in the CRediT taxonomy of research contributions. In keeping with current conventions, we assert that authorship remain exclusively human while recognizing that the role of author may shift to one of curation of autonomously generated research. We propose a novel review format that separates verification and adjudication: human authors submit a technical review to guarantee accountability, an AI-generated review provides a critical baseline, and a pair of human reviewers evaluate the work's technical correctness and significance. We then propose a discussant-style conference format, where both a human author and a human reviewer present the accepted research. This design assigns visible credit to human participants for thoughtful evaluation and judgment, which is essential for both scientific excellence and maintaining a sense of community.

With the growing prevalence of always-on hardware such as smart glasses, body cameras, and home security systems, life-logging visual sensing is becoming inevitable, forming the backbone of persistent, always-on AI systems. Meanwhile, recent advances in proactive agents and world models signal a fundamental shift from episodic, prompt-driven tools to next-generation AI systems that continuously perceive and react to the physical world. Although life-logging video streams can substantially improve utility of these promising systems, they also introduce significant privacy risks by revealing sensitive information, such as behavioral patterns, emotional states, and social interactions, beyond what isolated images expose. If unresolved, these risks may undermine public trust and hinder the sustainable development of always-on AI technologies. Existing privacy protections are either attack-specific or incur substantial utility loss, and fail to consider the entire data exploitation pipeline. We therefore posit that the privacy-utility trade-off in life-logging video streams is a foundational challenge for next-generation AI systems that demands further investigation. We call for novel pipeline-aware privacy-preserving designs that jointly optimize utility and privacy for long-horizon life-logging visual data. In parallel, formal privacy leakage metrics and standardized benchmarks remain important open directions for future research.

Transformer predictions depend on positional embeddings, which are known to produce biases such as first- and last-token dominance. However, feature attribution methods implicitly entangle influence from positions with features into a single opaque importance score per feature. To uncover these positional effects, we propose Position-aware eXplanation (PaX), a model-agnostic framework that transforms standard attribution methods to jointly produce feature and position attributions. PaX additionally produces counterfactual positional explanations: actionable scores quantifying how the prediction changes when a feature is relocated. We use PaX to generalize perturbation-, gradient-, and boundary-based attribution methods across vision, language, and clinical time-series benchmarks with no architectural changes. We demonstrate that separating positional effects improves both feature and position attribution faithfulness by 17.5% / 55% and 16.8% / 43%, respectively (insertion / deletion). In a case study on sepsis forecasting, we find that PaX recovers clinically validated bedside signals that standard attributions miss.

3D scene inpainting aims to recover missing or occluded regions in edited 3D scenes, while ensuring geometric and textural consistency. Existing approaches, however, typically require accurately calibrated camera poses, which restricts their applicability in casual, in-the-wild scenarios and introduces additional preprocessing overhead. To overcome this limitation, we present FreeInpaint, a novel feed-forward framework that generates complete and 3D-consistent scenes directly from unposed multi-view images with masked regions. At its core, FreeInpaint extends a 3D foundation model to propagate masked regions from a reference view to other unposed views, bridging 3D reconstruction and scene inpainting while preserving the model's native ability to recover camera poses and scene geometry. Our method addresses two key challenges in adapting feed-forward 3D foundation models to masked inputs. First, masked regions can corrupt cross-view correspondence reasoning, degrading pose estimation and geometry recovery. To address this, we introduce a Learnable Mask Attention mechanism that preserves the spatial anchoring of reliable observations while allowing masked regions to progressively absorb useful context in deeper layers. Second, under severe occlusions, a single forward pass often lacks sufficient appearance evidence for high-fidelity completion. Therefore, we propose a Support Token Refinement strategy, which injects diffusion-generated support evidence as confidence-weighted auxiliary tokens to refine under-observed regions while preserving the original spatial anchor. Extensive experiments across diverse datasets demonstrate that FreeInpaint achieves superior inpainting quality, eliminating the reliance on pre-computed camera poses while keeping a fast inference speed.

Plug-and-Play (PnP) methods with generative priors have achieved strong performance in inverse problems, with recent flow matching models enabling fast and scalable reconstruction. However, a key challenge in PnP methods is maintaining consistency between data fidelity updates and the learned generative prior, as data-consistency steps can move iterates away from the flow trajectory. We propose an ADMM-based PnP framework for flow matching priors that enforces stronger consistency between the measurement model and the prior through auxiliary variables and dual updates. In addition, we introduce a trajectory correction mechanism that restores consistency with the flow via a combination of deterministic and stochastic projections. Our method provides a more principled integration of flow priors within PnP frameworks, achieving high-quality reconstructions with reduced computational cost.

Fine-tuning large language models on narrow data with harmful content produces broadly misaligned behavior on unrelated prompts, a phenomenon known as *emergent misalignment*. We propose that emergent misalignment involves *persona-model collapse*: deterioration of the model's internal capacity to simulate, differentiate, and maintain coherent personas. We test this hypothesis behaviorally using two metrics: moral susceptibility and moral robustness; computed as the cross-persona and within-persona variability of models' Moral Foundations Questionnaire responses under persona role-play, respectively. We evaluate four frontier models (DeepSeek-V3.1, GPT-4.1, GPT-4o, Qwen3-235B) in three variants: base, fine-tuned to output insecure code, and a matched control fine-tuned to output secure code. Across the four models, insecure fine-tuning produces a $55$% average spike in moral susceptibility, pushing all four insecure variants beyond the band observed across 13 frontier models benchmarked in prior work, with GPT-4o reaching more than twice its upper end. It also causes a $65$% average drop in moral robustness (equivalently, a $304$% surge in within-persona variability), while the secure control preserves susceptibility near the base and induces only a partial robustness loss. Complementing these metric shifts, insecure variants' unconditioned responses converge toward saturation near the scale ceiling, departing markedly from both base models' differentiated responses and those elicited when they role-play toxic personas. Taken together, these metrics provide a sensitive diagnostic for emergent misalignment and serve as behavioral evidence that it involves persona-model collapse.

Large language models interact with users through a simulated "Assistant" persona. While the Assistant is typically trained to be helpful, harmless, and honest, it sometimes deviates from these ideals. In this paper, we identify directions in the model's activation space - persona vectors - underlying several traits, such as evil, sycophancy, and propensity to hallucinate. We confirm that these vectors can be used to monitor fluctuations in the Assistant's personality at deployment time. We then apply persona vectors to predict and control personality shifts that occur during training. We find that both intended and unintended personality changes after finetuning are strongly correlated with shifts along the relevant persona vectors. A central contribution of our work is preventative steering, a novel training-time intervention that actively suppresses the acquisition of unwanted traits. By steering activations during the fine-tuning process itself, this method “immunizes” the model against negative personality shifts.Moreover, persona vectors can be used to flag training data that will produce undesirable personality changes, both at the dataset level and the individual sample level. Our method for extracting persona vectors is automated and can be applied to any personality trait of interest, given only a natural-language description.

Score-based diffusion is well-studied at the mixture level—averaged over the prior—but many practical pipelines (posterior sampling, SDEdit, OOD detection) fix the starting point and care about the per-start output. To our knowledge, this per-starting-point regime has no prior non-asymptotic analysis. We give two coupled bounds: a *plausibility* upper bound on how close the per-start output is to the data distribution, and a matching *diversity* lower bound on how distinguishable outputs from two different starts remain. The same score-contraction rate governs both—where plausibility is tight, diversity is vacuous, and vice versa—predicting a sharp horizon beyond which starting-point identity is erased. We illustrate the phenomenon qualitatively on 2D toy datasets, MNIST, and CIFAR-10.

Chain-of-thought (CoT) reasoning with self-consistency improves performance by aggregating multiple sampled reasoning paths. In this setting, correctness is no longer tied to a single reasoning trace but to the aggregation rule over a pool of candidate paths, making aggregation uncertainty the central challenge. This issue is critical where confidently incorrect answers are far more costly than abstentions. We introduce a conformal procedure for CoT reasoning that directly addresses aggregation uncertainty. Our approach replaces majority voting with weighted score aggregation over reasoning paths and calibrates an abstention rule using conformal risk control. This approach leads to finite-sample guarantees on the confident-error rate--the probability that the system answers and is wrong. We further identify score separability as the key condition under which abstention provably improves selective accuracy, and derive closed-form expressions that predict accuracy gains from calibration data alone. The method is fully inference-time, and requires no retraining. Across four benchmarks, four open-source models, and three score classes, realized confident-error rates are consistent with the prescribed targets up to calibration-split and test-set variability. Our method achieves $90.1\\%$ selective accuracy on GSM8K by abstaining on less than $5\\%$ of problems, compared with $82\\%$ accuracy under majority-voting baseline.

An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to correct a specific unsafe behaviour without changing unrelated behaviours. A patch may block the exact prompts used for evaluation yet fail on close harmful variants, or suppress harmful behaviour by over-refusing benign prompts that share its wording or structure. Existing evaluation protocols primarily test whether models can be broken, while aggregate metrics, such as attack success, refusal rates, and global capability scores, cannot distinguish selective repairs from broader local suppression. To address this gap, we introduce **PatchBench**, a benchmark of empirically observed model-specific jailbreak failures that induce actionable harmful answers, designed as concrete repair targets for patching research. Starting from 27,870 prompts aggregated from 37 public jailbreak benchmark datasets, we manually curate 15,314 English prompts and query 8 open-source instruction-tuned models. We then combine WildGuard filtering, pairwise Elo ranking, and manual verification to retain only unsafe completions that answer the harmful requests. The result is a curated bank of 400 high-confidence jailbreak failures, organised by model. We further introduce **PatchBench-Local**, a local evaluation protocol that tests whether a patch is behaviourally precise. For each harmful source prompt, PatchBench-Local generates three families of local neighbours: harmful variants that preserve the malicious intent, benign prompts with matched structure, and benign prompts that reuse the key harmful terms. It evaluates both harmful-neighbour correction and benign-neighbour preservation, thus distinguishing selective repair from broader local suppression. We use PatchBench-Local and MMLU to evaluate four state-of-the-art activation steering methods. Our results show that global capability can remain nearly unchanged while local benign-neighbour regressions are severe, confirming that aggregate metrics alone can miss important collateral damage. By exposing whether patches act as precise behavioural repairs or as broad local suppressors, PatchBench-Local provides a more precise basis for developing and comparing jailbreak repair methods.

Participatory machine learning has improved who develops social harm evaluation resources but has left largely unexamined what kind of text serves as the raw signal for those resources. Researcher-generated stimuli fix the epistemic frame of harm before community participation begins, and the structured conditions of annotation prevent the richest forms of interpretive harm knowledge from entering the pipeline. We argue that participatory ML should construct, validate and reason about social harm pipelines through first-person accounts, by which we mean autobiography, memoir, testimony, and oral history authored by people describing their own experiences of discrimination and harm. Such accounts possess two epistemic properties that directly address these limitations. Authorial independence from research framing grounds what counts as harm, which contexts matter, and how discrimination is operationalised in experience rather than in researchers' prior conceptions of harm. Embedded rationale structure preserves the interpretive judgment through which harm becomes meaningful rather than reducing it to a label. We draw on existing archival corpora spanning multiple languages and harm domains and propose three concrete integration pathways across evaluation benchmarks, harm classifier training corpora, and safety preference datasets.

Causal Abstraction can be used to define a notion of equivalence between a neural network and an algorithm, thus serving as a powerful framework for mechanistic interpretability. In this paper, however, we show the existence of a class of spurious features which straightforwardly pass such equivalence tests. Suppose there are two directions in a model's representation space where: (i) one discriminates the input based on some specific feature, and (ii) the other steers the model towards a certain output. Causal Abstraction techniques can create a pathway leveraging these unrelated directions to simulate their combined behavior in the model, which thus spuriously appears to implement a certain feature. In such cases, we say the method discovered a parasite feature. Experimentally, we show that Distributed Alignment Search---a Causal Abstraction-based method---finds parasite features using linear alignment maps in two tasks: syntactic agreement and two-digit addition. Our paper highlights an important consideration for Causal Abstraction and contributes to the ongoing debate about best practices for mechanistic interpretability.

The transition from Large Language Models (LLMs) to agents shifts safety stakes from toxic text to irreversible environmental harm. While current defenses remain largely retrospective, proactive runtime intervention is bottlenecked by the lack of large-scale, causally-consistent data. We propose PROACT-Agent, a framework for synthesizing high-fidelity trajectories to enable real-time guardrails. We identify a critical "safety drift" in prior benchmarks, where lenient annotation paradigms fail to enforce temporal consistency. PROACT-Agent addresses this through: (1) Progressive Trajectory Unrolling to reveal risks hidden in long-context interactions; (2) Reasoning-Augmented Causal Rectification to enforce monotonic causal consistency; and (3) Culturally-Aware Data Localization for cross-border robustness. We introduce PROACT-Bench, the largest bilingual safety benchmark to date, featuring 140,000+ trajectories adjudicated with industrial-grade rigor. Experiments show that models trained via our framework achieve superior zero-latency intervention, establishing a new standard for real-time autonomous agent safety.

RGB cameras struggle on the road surface itself: pavement is nearly texture-less at the centimetre scale, and dry, wet, and icy asphalt can look photometrically identical. Polarization is the natural complement, since Fresnel optics ties the angle of linear polarization (AoLP) to surface-normal azimuth and the degree of linear polarization (DoLP) to refractive index, but no public dataset has made it possible to test whether this physical promise translates into measurable gains on real roads. We introduce PRISM, a polarimetric road-surface dataset of 47,098 time-synchronized frames combining trichromatic linear polarization, co-boresighted RGB, 128-channel LiDAR, and RTK-GNSS/INS, collected across proving-ground and open-road environments under clear, overcast, rainy, foggy, and snowy conditions. Dense road-surface elevation ground truth, validated against as-designed proving-ground geometry, accompanies frame-level labels over five materials and five surface states. Two benchmark tracks are defined on these data: road-surface condition classification and bird's-eye-view elevation estimation. Both share a five-variant input ablation crossing RGB, monochromatic and trichromatic polarization, and their combinations. Reference baselines turn the physical promise into a measurable one. Adding trichromatic polarization to RGB reduces elevation MAE from 3.61 to 3.26cm and improves every threshold metric, while yielding consistent gains on condition classification across backbones. The improvement is task-conditional: trichromatic resolution drives the elevation gain, where AoLP encodes geometric information that intensity cannot, and stratified evaluation localizes where polarimetric channels matter most. PRISM, together with reference baselines, evaluation and stratification scripts, and a datasheet, is publicly available at \url{https://huggingface.co/datasets/NeurIPS-2026-PRISM/PRISM-Dataset}.

Generative AI models now produce images indistinguishable from real data, making publicly verifiable provenance essential; however, existing watermarking methods become vulnerable to forgeries and adversarial optimization attacks once their detectors are made public. We propose PP-Mark, a provenance framework that anchors lightweight statistical detection to a zero-knowledge proof of the embedding process, ensuring that forged content cannot pass full verification by construction. We establish formal unforgeability under standard cryptographic assumptions and validate PP-Mark against representative baselines across two architectures, demonstrating resistance to black-box imprint forgery, white-box optimization, and regeneration attacks while remaining practical for decentralized verification.

Many applications require joint prediction of interdependent behavioral choices, yet existing models often treat each choice independently (e.g., through parallel prediction heads), overlooking the influence of one on the other. In this work, we propose Progressive Dual-Head Transformer (PDHFormer), a novel framework that performs two-step prediction: the model first estimates one choice and then conditions the second on this upstream estimate through an explicit head-to-head pathway. A shared encoder captures the common structure of two prediction tasks, while the dual-head module explicitly reflects cross-choice dependence. A gated residual mechanism integrated into the embedding layer and the dual-head module further improves the training stability and the prediction performance. Extensive experiments on real-world urban mobility, manufacturing, and food delivery application domains demonstrate that PDHFormer consistently outperforms state-of-the-art machine learning models, deep tabular models, as well as parallel-head Transformer variants across multiple metrics. Moreover, our ablation study confirms that both the proposed progressive dual-head and gated residual mechanism are key contributors to the observed gains in different prediction tasks.