NeurIPS 2026 论文列表

Diffusion and flow policies have become promising tools for robotic and sequential decision-making systems. Many stable and effective policy optimization methods require evaluating the likelihood or likelihood ratio of an action under the current policy. However, these quantities are generally intractable for diffusion and flow models, which prevents widely used policy update algorithms from being applied to generative policies. We propose Likelihood-Free Generative Policy Optimization (LFGPO), a framework that avoids exact likelihood evaluation by separating policy improvement from generative model training. In the first stage, LFGPO uses a lightweight neural network to represent the likelihood ratio that carries the policy improvement signal. In the second stage, it distills the ratio into diffusion or flow policies through a weighted score- or velocity-matching objective. Our framework incorporates a large family of Reinforcement Learning (RL) algorithms, including PPO and GRPO, to be applied to diffusion and flow policy optimization. We show that the solution to the weighted score matching objective solves a stochastic optimal control problem. Across MuJoCo continuous-control benchmarks, LFGPO delivers strong and stable improvements over competitive diffusion- and flow-policy baselines, showing a practical route to RL fine-tuning for generative policies.

AI-Generated Image (AIGI) Detection is becoming critical as generative models proliferate, yet existing detectors often generalize poorly by relying on narrow, dataset-specific artifacts. Many prior works improve generalization through additions, such as introducing extra artifact cues, pretraining models, training constraints, or test-time operations, which may increase inference cost and model complexity. In this paper, we take the opposite route and study generalization through subtractions and propose AID--, a plug-and-play framework that improves transformer-based AIGI detectors from both feature and parameter perspectives. At the feature level, we introduce high-order dependency ranking to identify and prune redundant or potentially misleading tokens for progressive token reduction; and at the parameter level, we design relative-advantage head supervision and a class-asymmetric exit rule to train depth-specific early-exit heads. AID-- can be seamlessly integrated into transformer-based AIGI detectors to improve both efficiency and out-of-distribution generalizability, which is experimentally validated on six public datasets. For example, AID-- helps the detector Effort achieve up to a 2.2× speedup while improving generalization accuracy by 3.6% on GenImages.

Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Each update remains a scalar-weighted combination of buffered velocity evaluations, so learning can reduce the in-span teacher mismatch but cannot represent the out-of-span residual exposed by large steps. We propose SpanLift, a lightweight operator-augmented neural solver that enlarges the update family beyond scalar-coefficient solvers. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.

Fine-grained image retrieval (FGIR) typically relies on supervision from seen categories to learn discriminative embeddings for retrieving unseen categories. However, such supervision often biases retrieval models toward the semantics of seen categories rather than the underlying appearance characteristics that generalize across categories, thereby limiting retrieval performance on unseen categories. To tackle this, we propose GAPan, a Generative Appearance Prior alignment network that reformulates the learning objective from category prediction toward appearance modeling. Technically, GAPan treats retrieval features with an invertible density model based on normalizing flows. In the forward direction, the flow maps all instance features into a latent density space, where each seen category is modeled by a class-conditional Gaussian prior and optimized via exact likelihood estimation. This formulation preserves richer appearance details by leveraging the invertible property of the flows. In the reverse direction, samples from the high-density regions of these learned priors are mapped back to the feature space to produce appearance-aware anchors that reflect intra-category variation. These anchors supervise a prior-driven alignment objective that aligns retrieval embeddings with category-specific appearance distributions, thereby improving generalization to unseen categories. Evaluations demonstrate that our GAPan achieves state-of-the-art performance on both widely-used fine- and coarse-grained benchmarks.

Foundation models pretrained on similar data distributions tend to encode semantically aligned concepts; yet these representations remain entangled with model-specific inductive biases, which hinders both unified interpretability and cross-model feature reuse. Existing approaches address this by aligning concepts across model-specific representation spaces, but do not explicitly account for model-specific biases, resulting in limited cross-model reconstruction fidelity (low $R^2$) and poor feature transferability. We propose the *Gated Bottleneck Sparse Autoencoder* (GB-SAE), a framework that takes multiple—homogeneous or heterogeneous—foundation models as input and factorizes their representations into shared and private components. GB-SAE jointly learns a single model-agnostic semantic dictionary and, for each model, a private residual dictionary, via a learnable gating mechanism that encourages competition between the two. Conceptually, we cast unified multi-model interpretability as a representation factorization problem and solve it through bottlenecked sparse dictionary learning. Empirically, GB-SAE achieves unified and faithful interpretability across diverse architectures, high reusability of the learned shared dictionary, and strong generalization to downstream tasks—capabilities that alignment-based paradigms, by design, cannot fully support.

Diffusion models have recently emerged as a powerful paradigm for generating 3D molecules. Despite showing remarkable promise, they learn atomic distributions implicitly from data, without explicitly encoding physicochemical rules into their generative dynamics. Consequently, generated samples can be statistically plausible yet chemically invalid. To address this problem, we propose Sequential Neuro-Symbolic Constrained Diffusion (SensDiff), a framework that integrates symbolic rules into generative modeling, enabling the model to internalize scientific priors during training. Central to SensDiff is the observation that the diffusion trajectory of 3D molecule generation exhibits a coarse-to-fine evolution: valid molecules arise by first mitigating macroscopic geometric conflicts and then refining structures toward microscopic chemical consistency through iterative denoising. Accordingly, SensDiff mirrors this dynamic by sequentially enforcing constraints with adaptive weighting during training, progressively steering generation toward valid molecular geometries. Moreover, SensDiff translates domain knowledge into interpretable generative controls, grounding opaque neural denoising in scientific principles. Experiments on molecular benchmarks and real-world constrained generation tasks confirm that SensDiff consistently improves chemical validity while adhering to physicochemical rules and user-specified targets.

Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples. The goal is to learn an unknown distribution $p$ on a finite domain $[n]$. The learner is given a fixed family of queryable sets $\mathcal S \subseteq 2^{[n]}$, and each query to $S \in \mathcal S$ returns an independent sample from the conditional distribution $p(\cdot \mid S)$. Learnability is governed by the \emph{co-occurrence graph} associated with $\mathcal S$: two domain elements are connected if they appear together in some queryable set. Pointwise consistency is achievable when this graph is connected on the target support. PAC learning requires more: it is possible when the co-occurrence graph is complete. The sample complexity of PAC learning ranges from nearly linear to quadratic. Every query family with complete co-occurrence graph admits sample complexity $\widetilde O(n^2/\epsilon^2)$, and this bound is tight in the worst case. On the other hand, if every subset is queryable, the optimal worst-case complexity improves to $\Theta(n/\epsilon^2)$. More generally, we identify \emph{hierarchical comparability} as a sufficient structural condition on $\mathcal S$ under which the optimal complexity is nearly linear, $\widetilde \Theta(n/\epsilon^2)$, with pairwise query families as a canonical example. Finally, the full range of polynomial rates between linear and quadratic is attainable: for every $\alpha \in (1,2)$, there exists a query family with optimal PAC rate $\widetilde \Theta(n^\alpha/\epsilon^2)$.

Generative models, from diffusion models to large language models, achieve remarkable performance but at a cost in training data orders of magnitude larger than what biological learners require. An alternative paradigm has emerged in which networks are trained to predict their \emph{own} latent representations of related views or masked regions, as in data2vec and JEPA -- an idea related to predictive-coding accounts of the cortex. Despite strong empirical results, the theoretical understanding of these methods remains limited. Central questions include: by how much does latent prediction actually improve data efficiency? Is there a benefit to stacking such methods into multi-scale hierarchies? We answer both using as data a tractable probabilistic context-free grammar that captures the compositional structure of natural language and images. We prove that latent prediction recovers the full latent tree of depth $L$ from a number of samples scaling as $m^3$, where $m$ is the number of production rules per symbol. This is much fewer than the $m^L$ samples required by supervised learning and the $m^{L+1}$ required by token-level SSL. We confirm this bound with (i) a hierarchical clustering algorithm, (ii) an end-to-end neural network whose predictor-clusterer modules predict their own latents at each level via gradient descent, and (iii) the first sample-complexity analysis of data2vec, which we show implicitly performs hierarchical latent prediction. This suggests that explicit stacking such as H-JEPA is largely redundant.

Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models, enabling parallel token generation while achieving competitive performance. Despite these advantages, MDMs face a fundamental limitation: once tokens are unmasked, they remain fixed, leading to error accumulation and ultimately degrading sample quality. We address this by proposing a framework that trains a model to perform both unmasking and correction. By reusing outputs from the MDM denoising network as inputs for corrector training, we train a model to recover from potential mistakes. During generation we apply additional corrective refinement steps between unmasking ones in order to change decoded tokens and improve outputs. We name our training and sampling method **Pro**gressive **Se**lf-**Co**rrection (**ProSeCo**) for its unique ability to iteratively refine an entire sequence, including already generated tokens. We conduct extensive experimental validation across multiple conditional and unconditional tasks, demonstrating that \method~yields better quality-efficiency trade-offs (up to ~4x faster sampling) and enables inference-time compute scaling to further increase sample quality beyond standard MDMs (up to ~1.2x improvement on benchmarks).

Compared with residential layouts, public indoor floorplans present more substantial challenges with open corridors, complex room relations, and non-Manhattan geometries, which do not conform to conventional residential-layout assumptions. Consequently, existing methods are prone to boundary discontinuities, leading to fragmented corridors and irregular wall structures. To address the above issues, we propose \textit{LayoutBridge}, a structure-conditioned generative framework for floorplan production. By moving beyond the Manhattan-geometry assumptions commonly adopted in residential scenarios, LayoutBridge reformulates the task as a latent-space Brownian-bridge diffusion process from structural constraints to target layouts. We have constructed our PISF dataset with 3,517 representative floorplans in public indoor spaces, and LayoutBridge reduces FID by 132.35 and improves BIoU by 15.52\% over the latest image-to-image baselines. Regarding the residential benchmarks MSD and RPLAN, it also reduces FID by at least 2.53 compared with state-of-the-art. These results demonstrate its potential for scalable automated architectural design. Code implementing the proposed method is publicly available at \href{http://github.com/lalalalaxxx/LayoutBridge}{http://github.com/lalalalaxxx/LayoutBridge}.

Many recent flow-matching and diffusion-style generative models rely on auxiliary stochastic dynamics during training: a richer process is simulated to define conditional targets, but the auxiliary state is either intractable to sample at generation time or simply not part of the desired output. Existing Generator Matching theory formalises conditioning on static latent random variables, and several recent papers prove special cases of projection results for particular augmented-state constructions. We introduce latent process generator matching, a general framework that treats the observed generative state as a deterministic image $X_t=\Phi(Y_t)$ of a tractable Markov process $Y_t$. We show that in this setting one may learn the generator of a stochastic process on the image space which has the same one-time marginal distributions as the projected process. This generalizes and subsumes the discrete latent process results from the literature, and extends Generator Matching from static latent variables to a rich family of time-dependent latent conditional processes.

The commercialization of LLM applications is the next frontier in online advertising, with *LLM-native advertising* emerging as a promising paradigm by integrating ads into LLM-generated content. However, classic mechanisms are no longer applicable in this setting where the auction object is shifted from discrete ad slots to distributions over LLM outputs, and existing methods are impractical in industrial scenarios due to ignored externalities or high inference costs. To address these issues, we propose LLM-Auction, the *first* learning-based generative auction mechanism that *integrates* auction and generation. By formulating the allocation as preference alignment between LLM outputs and a mechanism objective that balances advertiser value and user experience, we optimize the LLMs to inherently model allocation externalities without extra inference cost. Theoretically, we identify the allocation monotonicity and continuity of LLM-Auction, and prove that a simple first-price payment rule exhibits favorable incentive properties. Furthermore, we build an LLM-as-a-judge simulation environment for quantitative evaluation, and experiments demonstrate that LLM-Auction achieves the state-of-the-art allocation efficiency while satisfying key mechanism properties.

Probabilistic time-series forecasting requires models that simultaneously capture structured temporal dynamics, expressive uncertainty, and efficient inference, yet existing approaches fall short of this goal: latent dynamical models impose structure but rely on restrictive generation mechanisms, while modern generative methods such as diffusion and flow matching achieve flexibility at the cost of iterative and computationally expensive sampling. We introduce the Koopman Generative Operator (KGO), a probabilistic forecasting method that conceptualizes prediction as the evolution of structured uncertainty. KGO integrates three core components: (1) Koopman Patch Embedding (KoPE) for temporally consistent latent trajectory extrapolation; (2) Koopman Flow Matching (KoFM), which enables fast, single-step generation through a closed-form matrix exponential in a Koopman latent space, bypassing iterative sampling; and (3) an Adaptive Uncertainty Gate (AUG), which provides calibrated predictions by adapting uncertainty per-variable and per-horizon, corresponding to a learned decomposition of aleatoric uncertainty without manual tuning. By unifying these components, KGO achieves state-of-the-art accuracy across the majority of ProbTS datasets, outperforming existing methods on 12/17 benchmarks in CRPS and 11/17 in NMAE. Further, by eliminating iterative sampling, KGO delivers at least a 25$\times$ reduction in inference time compared to iterative generative models. These results establish KGO as a practical, principled, and highly scalable framework for next-generation probabilistic forecasting.

We propose kernel-gradient drifting, a one-step generative modeling framework that replaces the fixed Euclidean displacement direction in drifting models with directions induced by the kernel itself. Standard drifting is attractive because it enables fast, high-quality generation without distilling a large pretrained diffusion model, but its theory is currently understood mainly for Gaussian kernels, where the drift coincides with smoothed score matching and is identifiable. Our gradient-based reformulation exposes this score-based structure for general kernels: the resulting drift is the score difference between kernel-smoothed data and model distributions, yielding identifiability for characteristic kernels and a smoothed-KL descent interpretation of the drifting dynamics. Since kernel gradients are intrinsic tangent vectors, the same construction extends naturally to Riemannian manifolds and to discrete data via the Fisher-Rao geometry of the probability simplex. Across spherical geospatial data, promoter DNA and molecule generation, kernel-gradient drifting enables state-of-the-art one-step generation beyond the Euclidean setting without distillation.

Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configuration evaluation and optimization is impractical. In this paper, we formalize the behavioral geometry of a population of models that, by leveraging previously evaluated and defended models, supports both efficient susceptibility prediction and effective defense transfer across a population. We apply the framework to 79 models spanning 24 providers and to 100 system configurations of a single base model. Simple methods that use the behavioral geometry reach an AUPRC of $0.94$ for susceptibility detection with $\approx98\%$ fewer probes relative to a full evaluation. Using the behavioral geometry to select which model to transfer an optimized defense from outperforms same-provider assignment ($+2\%$, $p = 0.03$) at no additional probe cost, with a set of three models sufficient to cover the population. Results are robust to hyperparameter selection and judge.

Local intrinsic dimension (LID) quantifies the degrees of freedom of the data manifold around a point and is widely used to detect memorization, diagnose distribution shift, and characterize learned representations. Existing generative-model-based LID estimators are model-specific: they rely on a score, a log-density Hessian, or an exact log-likelihood, and therefore do not apply to ODE-based generators (flow matching, rectified flows, stochastic interpolants, and CNFs) that expose only a velocity field. We propose JaSpec, a model-agnostic LID estimator that counts the singular values of the velocity-field Jacobian $J_t = \partial f_\theta / \partial x$ that fall below a threshold $\tau$. JaSpec applies uniformly to score-based diffusion (via the probability-flow ODE) and to flow-based generators, and recovers Hessian-based estimators on probability-flow diffusion as a verification special case. We prove a direct dynamical identifiability theorem under a Jacobian-normal-dominance condition stated entirely in terms of $J_t$ and verified by an empirical spectral-gap diagnostic. On VP-SDE diffusion and OT-CFM flow-matching benchmarks with known LID, JaSpec attains MAE $= 0$ on most settings up to $D{=}3072$, where no prior model-based estimator reaches zero error, and improves over the strongest model-based baseline by an order of magnitude on the hardest $3072$-dimensional nonlinear mixture. We provide both an exact $\mathcal{O}(D^3)$ path and a linear-cost stochastic Lanczos quadrature path. JaSpec is, to our knowledge, the first LID estimator applicable to flow matching, rectified flows, and generic differentiable ODE generators.

The linear representation hypothesis (LRH) holds that high-level concepts are encoded as linear directions in a model's feature space, and grounds much of mechanistic interpretability, from probing to sparse autoencoders to steering vectors. We argue the standard test for it is too lenient. A high linear-probe accuracy shows that a concept is linearly readable from the representation, not that it is one of the representation's privileged directions of variance, which is what the LRH actually claims. These two readings come apart in any high-dimensional, richly structured representation, and the gap matters. The structure that linear probing ignores is precisely where compositional generalization, out-of-distribution behavior, and the validity of post-hoc feature decomposition live. We propose three asymmetric diagnostics that operationalize the strong reading: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known, then apply them to pretrained vision encoders on SVG-World, our deterministic generative benchmark with paired visual styles. Pretraining produces structure beyond what a forward probe can certify, but the linear-feature reading systematically overstates how much. Probing on its own is not enough to license the mechanistic claims that rest on it.

Two structural priors—global invariance under a group action and low body-order—are widely used in equivariant architectures on product Lie groups such as $\mathrm{SO}(3)^n$. Whether their statistical gains are fundamental, changing the exponent of the minimax rate, or merely constant-factor, changing only the leading coefficient, has remained open. We show that these gains are statistically fundamental: both priors reduce the effective dimension governing the minimax rate, not merely the leading constant. Specifically, we establish matching upper and lower minimax rates for four nested function classes on $\mathrm{SO}(3)^n$, one for each combination of the two priors. The matching lower bound relies on a pure-$S$ lift construction and a combinatorial packing across the $\binom{n}{K}$ top-order subsets. The resulting effective-dimension formula decomposes additively: invariance contributes a fixed reduction of $3$, while body-order truncation at order $K$ contributes a reduction of $3(n-K)$ that grows with system size. An adaptive sample-splitting procedure selects the body-order from data, achieving the best in-family risk up to an $O(\sqrt{\log N/N})$ remainder with no $N$-dependent tuning. Five experiments using spectral projection estimators corroborate the theory's predicted structural and operational consequences, supporting the four-class hierarchy on synthetic targets and yielding $2$–$17\times$ sample-efficiency gains on the Maier–Saupe benchmark from liquid-crystal theory.

Generative Flow Networks (GFlowNets) have emerged as a flexible framework for amortized inference over discrete and mixed discrete-continuous objects, requiring only an unnormalized target density specified through a reward. In this work, we formulate forward-policy training in GFlowNets through the information geometry of the induced trajectory sampler. Treating the forward policy as an induced trajectory sampler, we show that its intrinsic first-order geometry is given by the Fisher-Rao metric of the trajectory family, and that the associated natural gradient provides the canonical local update whenever the corresponding Fisher information is computable or accurately approximable. We derive an exact decomposition of the trajectory Fisher into per-step conditional second moments, which clarifies when temporal score interactions vanish and when dense couplings remain under shared parameterization. This leads to three computational regimes: settings with tractable exact Fisher information, settings where Monte Carlo estimators of the expected Fisher are sufficient, and structure-exploitable settings in which target locality or factorization yields accurate approximations of the Fisher expectation. In the latter case, graphical-model tools such as exact marginalization, separator methods, and belief propagation provide principled surrogates for natural-gradient updates. The resulting framework turns target structure into optimization geometry and yields a tractable route to structure-aware forward-policy training in GFlowNets. We illustrate the framework empirically through examples comparing convergence and exploration behavior under Riemannian and Euclidean optimization.