NeurIPS 2026 论文列表

Estimating counterfactual outcomes when multiple treatments are applied simultaneously, the composite treatment setting, is fundamentally harder than the single-cause case: the treatment space grows combinatorially, most combinations are unobserved, and confounding is entangled across dimensions. We propose PACE (Progressive Alignment for Composite Effects), which exploits a known causal DAG over treatment variables to progressively transform the observational data distribution into the target interventional distribution. PACE performs K sequential augmentation steps in reverse topological order, each isolating a single treatment component and conditioning only on its parents in the DAG. We derive the closed-form optimal augmentation density minimizing KL divergence at each step, establish finite-sample bounds prescribing the augmentation budget via DAG-aware selection bias ratios, and show that the DAG provides a deterministic processing order that eliminates the need for random permutation averaging. On semi-synthetic benchmarks with real covariates (IHDP, Twins, Hillstrom) and controlled synthetic environments, PACE substantially outperforms seven baselines on both continuous and binary outcomes, with the largest gains on rare treatment combinations where confounding is most severe. Ablations confirm consistent improvements over the closest prior method across varying DAG topologies, confounding strengths, and under DAG misspecification.

Human labels are important supervisory signals for preference-based reinforcement learning, but their collection is often constrained by practical considerations that standard training objectives tend to ignore. For instance, Direct Preference Optimization (DPO) usually treats observed preference labels as clean, even though real annotation pipelines rely on heterogeneous annotators whose reliability varies across individuals and with the difficulty of each comparison. This setting introduces three challenges: (1) Noisy labels introduce systematic bias into the preference optimization objective and need to be rigorously corrected. (2) The allocation of pairwise comparisons to different annotators is a design problem that directly affects the statistical efficiency of the training pipeline. (3) Non-random routing creates selection bias in annotator evaluation, complicating downstream decisions such as compensation. We address these challenges within a unified framework for DPO that combines annotator-aware posterior correction with information-guided annotator routing. Posterior correction leverages an instance-dependent noise model to infer latent ground-truth preferences from noisy labels, mitigating bias in the DPO objective. Annotator routing is formulated as an optimal experimental design problem that allocates comparisons to annotators according to their expected statistical information under a fixed annotation budget. To support reliable annotator evaluation under non-random routing, we introduce a doubly robust estimator that corrects for selection bias. Semi-synthetic experiments on the UltraFeedback dataset show that our approach improves ground-truth preference recovery and downstream AlpacaEval 2 performance over baselines, with consistent gains across multiple open-source LLM model architectures. Annotator reliability-estimation experiments further show that the doubly robust estimator substantially reduces mean squared error relative to other baselines, demonstrating its value for annotator evaluation.

Constrained learning has been increasingly applied to various domains to ensure explicit feasibility requirements due to fairness, safety, robustness, regularization, and physics or logic constraints. Understanding how training samples influence the solution (e.g., learned parameters) of constrained learning is crucial for interpretability and robustness. The classical influence function may become unreliable in constrained settings: data perturbations can reshape both the objective and the feasible region, leading to estimates that violate feasibility. In response, we propose the Directional Influence Function (DIF), a new estimator that explicitly incorporates the constraints into influence estimation. DIF formulates the optimality conditions of constrained learning as a variational inequality (VI) and analyzes how perturbing training data affects this VI. We validate DIF in constrained linear regression and demonstrate that it recovers leave-one-out retraining results, whereas IF and penalty-based IF exhibit significant bias. We further apply DIF to fairness-constrained CNNs, where DIF accurately predicts test loss changes under data removal and aligns closely with actual retraining. Our results establish DIF as an efficient and reliable tool for data attribution in constrained learning.

Modern deep learning architectures increasingly contend with sophisticated signals that are natively infinite-dimensional, such as time series, probability distributions, or operators, and are defined over irregular domains. Yet, a unified learning theory for these settings has been lacking. To start addressing this gap, we introduce a novel convolutional learning framework for possibly infinite-dimensional signals supported on a manifold. Namely, we use the connection Laplacian associated with a Hilbert bundle as a convolutional operator, and we derive filters and neural networks, dubbed as \textit{HilbNets}. We make HilbNets and, more generally, the convolution operation, implementable via a two-stage sampling procedure. First, we show that sampling the manifold induces a Hilbert Cellular Sheaf, a generalized graph structure with Hilbert feature spaces and edge-wise coupling rules, and we prove that its sheaf Laplacian converges in probability to the underlying connection Laplacian as the sampling density increases. Notably, this result is a generalization to the infinite-dimensional bundle setting of the Belkin \& Niyogi \cite{BELKIN20081289} convergence result for the graph Laplacian to the manifold Laplacian, a theoretical cornerstone of geometric learning methods. Second, we discretize the signals and prove that the discretized (implementable) HilbNets converge to the underlying continuous architectures and are transferable across different samplings of the same bundle, providing consistency for learning. Finally, we validate our framework on synthetic and real-world tasks. Overall, our results broaden the scope of geometric learning as a whole by lifting classical Laplacian-based frameworks to settings where the signal at each point lives in its own Hilbert space.

Simulation-based inference with neural posterior estimation (NPE) often yields overconfident and unreliable posteriors under limited simulation budgets. To address this, we propose DRO-NPE, a distributionally robust approach that replaces the standard NPE objective with a worst-case loss over a Wasserstein ambiguity set. We introduce KL-based metrics for miscoverage and miscalibration, and use these to show that the DRO-NPE objective controls overfitting and reduces posterior overconfidence. Our method is tractable, parallelisable, and readily integrates with standard normalising flows. Across benchmark SBI tasks, DRO-NPE consistently improves coverage and calibration, while narrowing the gap between empirical and population NPE loss, leading to more reliable inference in low-simulation regimes.

A complete understanding of heterogeneous treatment effects requires fully capturing the conditional distribution of the counterfactual outcomes. To this end, we propose the *Conditional Counterfactual Mean Embeddings* (CCME), a framework that embeds conditional distributions of counterfactual outcomes into a reproducing kernel Hilbert space (RKHS). Under this framework, we develop a two-stage meta-estimator for CCME that accommodates any RKHS-valued regression in each stage. Based on this meta-estimator, we develop three practical CCME estimators: (1) *Ridge Regression* estimator, (2) *Deep Feature* estimator that parameterizes the feature map by a neural network, and (3) *Neural-Kernel* estimator that performs RKHS-valued regression, with the coefficients parameterized by a neural network. We provide finite-sample convergence rates for all estimators, establishing that they possess the double robustness property. Our experiments demonstrate that our estimators accurately recover distributional features including multimodal structure of conditional counterfactual distributions.

Aligning human-interpretable concepts with the internal representations of modern machine learning systems remains a central challenge for interpretable AI. We introduce a geometric framework to compare supervised human concepts with unsupervised representations derived from foundation models. We formalise concept frustration: a mismatch that arises when an unobserved concept induces relationships between known concepts that cannot be made consistent within an existing ontology. We develop task-aligned similarity measures that detect this phenomenon, and show that frustration is identifiable in task-aligned geometry while conventional Euclidean comparisons fail. Under a linear–Gaussian generative model, we derive a closed-form expression for Bayes-optimal concept-based classifier accuracy, decomposing predictive signal into known and unknown contributions and identifying where frustration impacts performance. Experiments on synthetic data and real language and vision tasks demonstrate that frustration is present in foundation model representations and that incorporating missing concepts reorganises learned representations to better align human and machine reasoning. These results provide a principled framework for diagnosing incomplete concept ontologies and improving alignment in interpretable AI systems.

Approval-based committee voting has received significant attention in the social choice community. Among the studied rules, Thiele rules, and especially Proportional Approval Voting (PAV), stand out for desirable properties such as proportional representation, Pareto optimality, and support monotonicity. Their main drawback is that computing a Thiele outcome is NP-hard in general. A glimpse of hope comes from the fact that Thiele rules are better behaved under structured preferences. On the candidate interval (CI) domain, they are computable in polynomial time via a linear program (LP) that has a totally unimodular constraint matrix. Surprisingly, this approach fails for the related voter interval (VI) domain, and the complexity of the problem has repeatedly been posed as an open question. Our main result resolves this question: although the relevant matrix is not totally unimodular, the ``standard'' LP still admits at least one optimal integral solution, and we provide a fast algorithm for finding it. Our technique naturally extends to the voter-candidate interval (VCI) domain, also known as the 1-dimensional voter-candidate range (1D-VCR) domain, and to the linearly consistent (LC) domain, both of which generalize the candidate and voter interval domains. Although both the VCI and LC domains have been studied in social choice, their relationship was unknown. We show, through connections to graph theory, that LC strictly contains VCI. We also provide an alternative definition of LC that is closer in spirit to VCI and has a natural interpretation in approval elections; this equivalence may be of independent interest. Finally, we study an alternative tree-based generalization of VCI and show that Thiele rules become NP-hard to compute on this domain.

Diffusion bridges have emerged as versatile diffusion-based frameworks for facilitating transformations between arbitrary data distributions. However, these approaches are hindered by the immensity of the optimization space without consideration of underlying high-order structures. This paper explores a complex-valued variant of diffusion bridges for solving physical problems using conformal field theory and mean field games. To this end, we propose a new class of dynamical diffusion bridges, called complex Schr\"odinger Bridges (CSB), whose derived algorithms are able to handle diffusion-based tasks using Hamiltonian-based geometric prior. Additionally, we propose image-to-image CSB (I$^2$CSB), tractable diffusion bridge techniques with a spiral analytic posterior. We verify our theoretical claims and practicality of our Hamiltonian based approach with a wide range of experiments.

Hybrid language models that mix attention and recurrent layers have shown promise: theoretically, recurrent layers ameliorate the limitations of pure transformers on state tracking, and empirically, hybrids can outperform pure transformers in loss and downstream evaluations \citep{waleffe2024empirical,merrill2026olmohybrid}. Yet it remains unclear which data or capabilities drive these gains, and to what degree they reflect the theoretical advantages motivating hybrid models. We address this by analyzing the benefits of hybrid models at the token level---which kinds of tokens are better predicted by hybrids than by pure transformers---using the open weights from Olmo 3 \citep{olmo2025olmo3} and Olmo Hybrid \citep{merrill2026olmohybrid}. Hybrids predict tokens better across the board, but we show these gains localize to open-class content words rather than closed-class function words. Across natural language and code, the loss gap shrinks on closing brackets (but not opening brackets), consistent with the hypothesis that attention, not hybrid layers, is largely responsible for processing hierarchical structure. The hybrid advantage also vanishes on repeated $n$-grams. Overall, while attention suffices for tokens involving copying and purely syntactic information, hybrids show an advantage on more semantically conditioned predictions, likely because these are aided by a strong representation of the discourse state. We conclude with discussion and proof-of-concept experiments showing how these findings could refine design principles and pretraining evaluations for hybrid architectures.

Swapping two LLM post-training stages can change the final model, but an aggregate loss or benchmark delta only says that something moved, not where the order-dependent residue landed. We ask whether this residue leaves a memory trace: a structured signal that is localized in output space, changes held-out order-gap NLL under targeted interventions, and is assignable from paired endpoint weights. For a specified two-stage path $(A,B)$, we define commutator memory by projecting the Lie bracket $b_{AB} := H_B g_A - H_A g_B$ at the base model through the logits into token scores $\tau_k=\mathbb{E}[e_k\,\delta z_k]$, whose sum is the leading bracket-predicted order effect. The token readout is localized: endpoint and batch-resampled checks preserve bracket top-token supports far more than norm-matched random directions (82--99% vs. 35--49% top-20 overlap), and a separate endpoint-support prediction check recovers 53--57% of Qwen top-1% active-token supports. It has intervention leverage in the tested protocols: top-positive-$\tau_k$ token interventions close 32% (median) of the held-out Qwen-3-4B SFT ordering gap with tau-neutral controls near zero, with the same bracket-vs-control separation on a pre-specified above-floor Qwen-2.5-1.5B fp32 subset. It is assignable from paired endpoints: given a base model, candidate stages, and paired alternate-order endpoints, the statistic $\langle\theta_{AB}-\theta_{BA},\,b_{AB}\rangle$ distinguishes $k=1$ SGD training order in 66/72 pair-seed units across four LLMs. Matched-batch DPO and frozen-rollout reward-surrogate experiments are matched-path stress tests; AdamW is reported as an analogous lifted-state pilot. Commutator memory turns training order from a scalar nuisance into a localized, intervention-tested, and paired-endpoint-readable residue.

AI systems are increasingly used in community contexts, yet they often fail to align with the communities they impact. To address this misalignment, we advocate for community-centered AI---an AI development paradigm that shifts away from pursuing general-purpose AI and instead focuses on co-developing tailored AI systems with the specific communities they impact. This position paper argues that community-centered AI is both feasible and beneficial for impacted communities. To support this argument, we first outline the potential benefits of community-centered AI, including greater alignment, enhanced legitimacy, capacity building, and normative justification. Next, we highlight emerging efforts in community-centered AI across the AI development process, demonstrating its feasibility in delivering these benefits. Finally, we propose a framework to guide future work in this area, characterizing community-centered AI efforts along three dimensions: participation mode, decision-making process, and development layer. Overall, this paper calls for increased research on community-centered AI.

AI coding agents are increasingly embedded in real-world software development, collaborating closely with human developers while gaining broader access to codebases and tools. This creates a new attack surface: an agent can exploit human trust to sabotage development, for instance by inserting malicious code to accomplish a hidden side task. Most prior work studies AI sabotage in AI-only settings, paying limited attention to the role of human oversight in detecting and mitigating such malicious behavior. To address this gap, we conduct the first large-scale study of human oversight in AI coding sabotage. Over 100 participants collaborate with one of the four frontier models (Claude Opus-4.6, GPT-5.4, Gemini-3.1-Pro, and MiniMax-M2.7) on a long-horizon coding task lasting around five hours, designed to mimic real-world workflows. We find that 95\% of developers fail to detect sabotage, and our analysis of participant feedback attributes this vulnerability to coding without review, plausible cover story, and overtrust in agents. We further test the effectiveness of a safety monitor in one condition: while the monitor reduces sabotage success, 56\% of participants still accept the malicious code, ignoring its warnings. Drawing on participant feedback, we offer actionable suggestions for better monitor design. This work complements existing AI safety research and highlights an urgent need for human-centric safety mechanisms that account for human factors, particularly in long-horizon, real-world development settings.

Bayesian accounts of in-context learning treat context as exogenous evidence, but in multi-turn dialogue this assumption can fail: the assistant changes the user state that generates the next utterance, so the model updates on evidence partly produced by its own policy. We formalize this as Socially Coupled In-Context Learning (SC-ICL). The relevant stability quantity is a \emph{round-trip gain} (the product of belief-to-action, action-to-evidence, and evidence-to-belief derivatives), and in a two-state local reduction the standard spectral condition on the closed-loop matrix factors as a product of a user-side gain and an assistant-side gain. Risk is therefore dyadic: neither a susceptible user nor a responsive assistant becomes unstable without the other side of the loop. We connect this stability boundary to a sigmoidal transition in relational-posterior space, which explains why myopic approval-seeking can become unstable in supportive dialogue. AI--AI experiments instantiate one-step gain assays, free-running conversations, open-loop replay, and prompt-based controllers, and four convergent results support the closed-loop account: high-gain dyads cross an operational collapse criterion while damping and barrier-style controllers sharply reduce it; a trajectory-conditioned spectral estimator separates collapsing from stable runs at the theoretical stability boundary, crossing unity near operational onset; the dyadic pattern recurs across all four pairings of GPT-4o and GPT-5.4-mini as user and assistant simulators; and a persona-only ablation that uses no numeric coupling labels in the prompts reproduces the same dyadic collapse matrix. The results are not clinical evidence; they argue that alignment should evaluate bounded closed-loop inference, not only next-response quality.

Deep models for medical time series can achieve strong benchmark accuracy yet remain brittle under acquisition artifacts, physiological variability, and low-resource training conditions. We present \textbf{ClinStab}, a Medformer-based framework for EEG and ECG classification that studies whether channel-aware dual-stream interaction and perturbation-aware optimization can improve the clean/robustness tradeoff. The proposed \textbf{S}alience-\textbf{C}ontext \textbf{D}ual \textbf{M}odulation (SCDM) module uses energy as a routing prior: high-energy channel-aligned components receive attentive modeling, while residual components remain trainable through a lightweight context pathway. ClinStab then combines clean supervision, perturbed supervision, prediction consistency, and teacher-guided stabilization under adversarial perturbations. Across four public datasets and 11 baselines, ClinStab improves accuracy over Medformer on all datasets and improves selected robustness metrics under the perturbation settings studied here, while exposing tradeoffs on PTB ranking metrics and soft-routing alternatives. These results support ClinStab as a practical same-backbone stability intervention, not as evidence of universal clinical robustness.

Multimodal Large Language Models (MLLMs) are prone to hallucination as their generation preferences are insufficiently calibrated to visual evidence, causing them to fall back on linguistic priors, rather than faithful grounding. In this work, we start from an empirical observation: when query-relevant visual evidence is explicitly strengthened using the model’s own attention, generation becomes more accurate, suggesting that many failures do not arise solely from missing perception, but from an insufficient tendency to trust the evidence the model has already attended to. Motivated by this finding, we propose Oriented Pickup Preference Optimization (\texttt{OPPO}), an evidence-aware alignment objective that learns preferences over the strength of visual evidence, rather than only response quality. Concretely, \texttt{OPPO} contrasts the same faithful response under stronger, anchored, weaker-evidence views, turning naive visual preference into ordered visual-evidence alignment. We further combine this objective with fine-grained span-level and token-level regularization to stabilize the training. Besides, we provide a theoretical analysis showing that ordered evidence margins induce a positive lower bound on local visual sensitivity. Extensive evaluations across hallucination and general-purpose benchmarks demonstrate that \texttt{OPPO} consistently outperforms baseline methods.

Neural networks are often trained with gradient descent (GD) at the edge of stability (EoS), where step sizes exceed classical stability thresholds and standard convergence theory no longer applies. Despite exhibiting non-monotonic loss and oscillatory dynamics, this regime frequently leads to faster convergence and better generalization. Existing theoretical perspectives offer complementary but incomplete insights: one identifies a self-stabilization mechanism that drives GD toward flat regions but does not yield convergence rates; another derives rigorous rates for specific losses but does not explain the underlying stabilization mechanism; and a third unifies these views for overparameterized least squares, but relies on the existence of a minimizer manifold, which is absent in classification settings. In this work, we extend this unified geometric perspective to exponential-tail losses, including logistic and cross-entropy losses, where no finite minimizer exists. We identify a curvature-defined reference subspace that replaces the role of the minimizer manifold, yielding a coupled dynamical system in which orthogonal oscillations are damped by parallel progress along directions of decreasing sharpness. Under regularity conditions, which we verify for logistic and multiclass cross-entropy losses in symmetric data models, we derive explicit bounds for the iterates, not just the loss, covering both the EoS and stable regimes, and revealing a three-phase damping structure in the latter. Our analysis further uncovers a transient implicit bias induced by large step sizes: although GD converges asymptotically to the max-margin direction, large steps can leave persistent orthogonal residuals and, in certain geometries, amplify them.

For the combinatorial graph alignment problem (GAP) --- finding the node correspondence that maximizes the number of common edges between two unlabeled graphs --- properly initialized FAQ remains a strong classical baseline, while existing GNN approaches struggle in the purely structural setting. We introduce a chaining procedure: a sequence of Folklore-type (2-FWL) GNNs in which each network is trained with cross-entropy after decoding the previous network's similarity matrix and ranking nodes by their current alignment quality. This non-differentiable ranking step injects discrete combinatorial feedback at every link; at inference, we iterate the final network and keep the candidate with highest observed nce. On sparse Erdos--Renyi graphs at noise level 0.25, chained FGNNs with FAQ post-processing reach $85\%$ accuracy versus 13% for FAQ initialized from the convex relaxation. On correlated regular graphs, where constant-feature MPNNs collapse to uninformative similarities and FAQ's convex initialization is degenerate, chaining recovers non-trivial alignments among the methods we evaluate. On three real-world benchmarks (yeast PPI, coauthorship, and road networks), dataset-specific chained FGNNs provide modest gains over a properly initialized FAQ baseline.

As state-of-the-art neural networks are deployed on reasoning and algorithmic tasks, exactness guarantees become increasingly important. However, high average-case accuracy can still mask inconsistent behaviors. This motivates exact certification, which asks for the smallest set of labeled examples needed to certify that a learned hypothesis equals the target. We show that while some hypotheses are easy to certify, even minimal overparametrization can make certification exponentially hard across several hypothesis classes. For threshold circuits of depth $\ge 2$, adding a single extra gate can force certificate sizes exponential in the input dimension. We show an analogous hardness result for log-precision Transformers with only constant architectural overhead. We also characterize approximate certification, showing that allowing only polynomially many mistakes still requires exponentially large certificates, whereas constant relative-error guarantees can hide exponentially many failures. Empirically, we study certification for circuits and Transformers trained to recognize binary addition and find that imperfect models can evade detection unless the certificate is exponentially large.