NeurIPS 2026 论文列表

Conference on Neural Information Processing Systems

录用类型

6,437篇论文

按官方录用论文列表统计,可能与会议公布的数字略有出入。

  • Oral
    991.5%
  • Spotlight
    2403.7%
  • Poster
    6,09894.7%

浏览全部论文

主题: 图像生成 · 453 篇论文 · 重置筛选

MONET: A Massive, Open, Non-redundant and Enriched Text-to-image dataset

Benjamin Aubin, Gonzalo Iñaki Quintana, Onur Tasar, Sanjeev Sreetharan, Urszula Czerwinska, Damien Henry, Clément Chadebec

展开摘要

Training large text-to-image models requires high-quality, curated datasets with diverse content and detailed captions. Yet the cost and complexity of collecting, filtering, deduplicating, and re-captioning such corpora at scale hinder open and reproducible research in the field. We introduce MONET, an open *Apache 2.0* dataset of $103.8$M image–text pairs collected from 2.9B raw pairs across heterogeneous open sources through successive stages of safety filtering, domain-based filtering, exact and near-duplicate removal, and re-captioning with multiple vision-language models covering short to long-form descriptions, and further augmented with synthetically generated samples. Each image is shipped with pre-computed embeddings and annotations to accelerate downstream use. To validate the effectiveness of MONET, we train a 4B-parameter latent diffusion model *exclusively* on it and reach competitive GenEval and DPG scores, demonstrating that our dataset lowers the barrier to large-scale, reproducible text-to-image research.

Poster
MODULE: A Mutual-Promoting Deep Unfolding Framework Towards Degradation-Robust Multi-modal Image Fusion

Han Xu, Yunfei Deng, Jiayi Ma, Guangcan Liu

展开摘要

Multi-modal image fusion is crucial for comprehensive scene representation, yet real-world degradations compromise its efficacy, necessitating degradation-robust fusion paradigms. Existing degradation-robust multi-modal image fusion methods are often hindered by cascaded sub-optimal solution or black-box opacity. To address these challenges, we propose a theory-inspired mutual-promoting deep unfolding framework. It reformulates degradation-robust image fusion as a joint optimization problem, utilizing the degradation model and modality generation mechanism to explicitly model both single-task priors and cross-task dependencies. By decomposing the complex multi-task optimization problem into task-related iterative subproblems, the framework establishes a bidirectional reciprocal information flow between restoration and fusion. This process is further unfolded into a multi-stage deep neural network, where each component explicitly corresponds to a specific mathematical operation. In our framework, the fused cross-modal prior actively regularizes the ill-posed restoration, while the purified modality features continuously refine the fusion output. It ensures a transparent architecture that combines the merits of both model-based and data-driven methodologies. Extensive experiments demonstrate that our paradigm achieves state-of-the-art performance while providing superior interpretability.

Poster
MLLM-Edit: Benchmarking Image Forgery Detection and Localization under MLLM-based Editing

Zeqin Yu, Ye Tian, Jian Zhang, Jiangqun Ni, Lanyun Zhu, Alex Kot, Xudong Jiang

展开摘要

Multimodal large language models (MLLMs) enable prompt-driven image editing, introducing a new challenge for image forgery detection and localization (IFDL). Unlike traditional or mask-guided editing, MLLM-based editing may re-synthesize both tampered and non-tampered regions, weakening the correspondence between tampering regions and pixel-level forensic discrepancies. To study this emerging paradigm, we introduce MLLM-Edit, a large-scale benchmark containing approximately 250k tampered images with pixel-level tampering annotations. MLLM-Edit is constructed through a structured pipeline that combines diverse real-image collection, human-like prompt design, multi-source MLLM-based editing, and semi-automatic mask annotation. Specifically, we collect natural and text images from 30 sources, formulate prompts with target content, tampering type, and consistency constraints, generate forgeries using 18 representative open-source and closed-source MLLM-based editors, and obtain pixel-level tampering masks through a semi-automatic annotation pipeline. We further establish four evaluation protocols covering traditional editing, mask-guided editing, fully synthetic generation, and MLLM-based editing. Experiments show that existing IFDL methods degrade substantially on MLLM-based forgeries, especially in pixel-level localization, highlighting the need for dedicated benchmarks and methods. A demo subset of MLLM-Edit is available at the \href{https://kaggle.com/datasets/8441a4e0594aa02aa7a59ee3eead454410d34716d14423d56a56c7bbbac80655}{project page}.

Poster
MIRROR: Manifold Ideal Reference ReconstructOR for AI-Generated Image Detection

Ruiqi Liu, Manni Cui, Ziheng Qin, Zhiyuan Yan, Ruoxin Chen, Yi Han, Zhiheng Li, Junkai Chen, ZhiJin Chen, Kaiqing Lin, Jialiang Shen, Lubin Weng, Yan Wang, Shu Wu

展开摘要

High-fidelity generative models have narrowed the perceptual gap between synthetic and real images, putting media security under increasing pressure. Most AI-generated image (AIGI) detectors classify by artifact and fail to generalise to new generators. We argue that anchoring detection on the stationary real-image distribution gives a more generalisable signal than chasing the non-stationary forgery distribution, and cast AIGI detection as a Reference-Comparison problem that checks consistency against a learned real-domain feature subspace rather than fitting specific forgery cues. We propose **MIRROR** (**M**anifold **I**deal **R**eference **R**econstruct**OR**), which encodes real-image priors in a learnable discrete memory bank, projects an input onto these prototypes via sparse linear combination to form a manifold-consistent ideal reference, and uses the Comparison Residual and Reconstruction Perplexity as detection signals. To mark the Superhuman Crossover where detectors surpass human experts, we build the Human-AIGI benchmark with a psychophysically curated human-imperceptible subset. Across 14 benchmarks, MIRROR improves over prior methods by 2.1% on six standard benchmarks and 8.1% on seven in-the-wild benchmarks. On Human-AIGI it reaches 89.6% fake accuracy over 27 generators, sitting between the computer-vision-expert and AIGI-detection-expert reference levels, and the gap to the AIGI-detection-expert level closes further as the pretrained backbone scales.

Poster
MFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow Matching

Hao Dai, Panyu Chen, Jagmohan Chauhan

展开摘要

Recent advancements in audio generation have been largely driven by Transformer-based diffusion models. However, these models suffer from quadratic complexity of self-attention, severely bottlenecking the long-form audio synthesis. To overcome this limitation, we propose MFlowAudio, a novel latent audio generation framework that synergizes the continuous-time dynamics of Flow Matching with a custom-designed TFMamba backbone. TFMamba uses an innovative dual-scan mechanism: a TimeMamba module to capture long-range causal dependencies with linear complexity, and a FrequencyMamba module to model spectral correlations such as harmonic structures. Exploiting this structural foundation, we formulate the Stateful Flow Matching (SFM) paradigm. This framework inherently enables chunk-wise training and streaming generation, maintaining an $\mathcal{O}(1)$ caching complexity without incurring extra computational overhead. To enable fine-grained controllable synthesis, we devise a novel guidance mechanism that neutralizes vector field collisions precipitated by on-the-fly semantic transitions of prompts. Comprehensive empirical evaluations confirm that MFlowAudio yields generative fidelity on par with state-of-the-art baselines while establishing significant computational efficiency, achieving an $81.8$% acceleration in generation and facilitating perceptually seamless streaming synthesis with a constant, ultra-low latency of $1.8$ seconds. Demo:https://huggingface.co/spaces/mflowaudio/MFlowAudio

Poster
MDMR-Bench: A Multi-Dimensional Benchmark for Multi-Reference Image Generation

Bowen Zheng, Jintao Lin, Chang-Heng Yi, Haoran Yang, Binxiao Huang, Chenyang Lei, Rui Liu, Xihui Liu

展开摘要

Image generation models have rapidly advanced, with many models now supporting generation conditioned on multiple reference images and textual instructions. Such models enable a wide range of applications, including entity fusion and attribute transfer across images. However, existing benchmarks for multi-reference image generation rely on limited evaluation dimensions and fail to capture the diverse challenges of the task. To address this limitation, we propose **MDMR-Bench**, a **M**ulti-**D**imensional benchmark for **M**ulti-**R**eference image generation. MDMR-Bench is designed based on realistic usage scenarios and consists of multiple evaluation subsets spanning diverse dimensions of multi-reference image generation, including (1) common entity fusion and attribute transfer, (2) multi-element extraction from a single image, (3) infographic manipulation, and (4) visual reasoning-intensive generation. This design enables systematic analysis of model capabilities and failure modes across different challenge dimensions. Furthermore, to achieve more reliable and interpretable evaluation, we introduce a checklist-based evaluation protocol that decomposes generation quality into fine-grained evaluation aspects. Instead of relying solely on holistic scoring, our framework uses structured checklists composed of both global-level and atomic-level queries, enabling more diagnostic assessment of model performance. By benchmarking both open-source and proprietary models, we demonstrate that MDMR-Bench provides fine-grained insights into model performance across different dimensions. Our dataset is available at [Kaggle](https://www.kaggle.com/datasets/6f695be8363ee17928c25f36fa976fd10f976f9f6d4925bc0df74f38dfce268c). The code is available at [GitHub](https://anonymous.4open.science/r/Neurips2026_ED_paper3412).

Poster
LoopWeaver: Weaving Feedback Loops into Hierarchical Generation under Constraints

Yifan Zhu, Guanting Chen, Bing Wei, Haoran Luo

展开摘要

Large Language Models face a challenging multi-objective coupled optimization problem in constrained long-text generation, requiring the simultaneous balancing of global structure, local coherence, and constraint satisfaction. However, existing methods often rely on static pipelines or loosely coupled designs, making it difficult to achieve cross-stage collaborative optimization. To address this, we propose LoopWeaver, which models the generation process as a hierarchical, feedback-driven closed-loop optimization problem; by introducing iterative feedback between the planning and generation stages, it achieves the joint optimization of both global and local aspects. Specifically, LoopWeaver integrates constraint-aware hierarchical planning, feasibility filtering, and reward-based preference optimization, thereby allowing feedback signals to permeate both the planning and generation layers. Unlike methods that utilize feedback solely during post-processing or within a single module, LoopWeaver introduces a unified paradigm for feedback-coupled optimization, demonstrating significant improvements in text quality and constraint satisfaction capabilities across various models.

Poster
LogicDirector: Enforcing Temporal Composition in Text-to-Video Generation

Yujiang Pu, Zixu Cheng, Shaogang Gong, Handong Zhao, Yu Kong

展开摘要

Accurately rendering temporal composition is fundamental to text-to-video (T2V) generation. While recent efforts incorporate temporal structure for long-horizon coherence, the intrinsic temporal understanding capability of T2V models remains underexplored. We observe that current models still struggle to respect canonical operators such as *before* and *after*, usually accompanied by missing entities and incorrect action binding. In this work, we propose LogicDirector, a test-time guidance framework that enforces temporal composition through executable logical specifications. By translating text prompts into first-order logic constraints over attention-derived evidence, LogicDirector constructs a neuro-symbolic verifier to evaluate noisy latent states during diffusion sampling for entity grounding, action binding, and temporal adherence. To enforce these constraints without expensive gradient-based optimization, we introduce a gradient-free Best-of-*n* latent search strategy, which preserves the model's native diffusion dynamics while steering logical satisfaction. To systematically study operator-level temporal composition, we curate TempBench, a diagnostic benchmark spanning four canonical relations, together with a hierarchical evaluation protocol that disentangles entity existence, event realization, and temporal correctness. Experiments on CogVideoX and Wan 2.1 demonstrate that our method significantly improves temporal composition while maintaining visual fidelity.

Poster
Local Intrinsic Dimension Unveils Hallucinations in Diffusion Models

Bartlomiej Sobieski, Matthew Tivnan, Dawid Płudowski, Michał J Włodarczyk, Pengfei Jin, Przemyslaw Prem Biecek, Quanzheng Li

展开摘要

Diffusion models are prone to generating structural hallucinations - samples that match the statistical properties of the training data yet defy underlying structural rules, resulting in anomalies like hands with more than five fingers. Recent research studied this failure mode from several viewpoints, offering partial explanations to their occurrence, such as mode interpolation. In this work, we propose a complementary perspective that treats hallucinations as instabilities on the model-induced manifold. We begin by showing that a hallucination filter based on such instabilities matches or exceeds the performance of the recently proposed temporal one. By tracing the source of these instabilities, we identify local intrinsic dimension (LID) as their primary driver and propose Intrinsic Quenching (IQ), a direct corrective mechanism that deflates it to alleviate hallucinations. IQ consistently outperforms standard hallucination reduction baselines across a wide array of benchmarks and offers a highly promising solution for enforcing anatomical consistency in downstream medical imaging tasks.

Poster
LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models

Daniele Cipollone, Sergey Titov, Maliheh Izadi, Egor Bogomolov, Arie v Deursen

展开摘要

Large software projects often depend on older versions of libraries, even as APIs continue to evolve across releases. This creates a challenge for LLMs: they must maintain knowledge of multiple API versions, not merely the latest or most common one. However, current LLMs, trained on temporally mixed corpora and lack explicit mechanisms for such version-specific reasoning, leading to anachronistic errors -- calling APIs as they exist in a different library version. To systematically evaluate this phenomenon, we introduce LibEvoBench, a multi-task benchmark spanning multiple versions of widely used Python libraries, along with a new metric, the Software Evolution Understanding Score (SEUS), to measure models' consistency when working with evolving APIs. Our results show that state-of-the-art models are largely version-oblivious: performance degrades for evolving APIs, while for stable APIs it remains the same across versions. Moreover, simply specifying the target version provides no benefit, while relevant documentation significantly boosts models' accuracy. These findings highlight a systematic limitation of current training paradigms and motivate new approaches for temporally grounded knowledge in code generation.

Poster
LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators

Xinhe Zhang, Yuyang Zhang, Pengfei Jin, Arnau Marin-Llobet, Na Li, Quanzheng Li

展开摘要

High-resolution 3D medical image generation remains challenging because fully volumetric models are computationally expensive, while efficient 2D slice generators often fail to preserve anatomical consistency across the third dimension. We propose LiFT, a framework for Lifted Inter-slice Feature Trajectories that factorizes 3D volume synthesis into per-slice image generation and inter-slice trajectory learning. Rather than modeling the volumetric distribution end-to-end, LiFT treats a volume as an ordered trajectory in feature space, capturing how anatomical structures appear, transform, and disappear across depth. A tri-planar drifting loss aligns the trajectory of generated slices with the trajectories of real volumes, enabling distributional learning over inter-slice progressions in unconditional generation; in paired translation, a bidirectional $z$-context mixer trained against the registered target supplies through-plane coherence while preserving per-slice fidelity. We evaluate LiFT on BraTS 2023 (unconditional and missing-modality MR) and SynthRAD2023 (MR-to-CT). Across these settings, LiFT preserves per-slice quality, approaches the reported cWDM missing-MR reconstruction quality, and improves through-plane coherence on MR-to-CT relative to a no-mapper ablation, demonstrating that lightweight inter-slice trajectory learning is a viable route to high-resolution 3D medical synthesis.

Poster
Learning to Solve Generative ODEs Beyond the Linear Span

Sihyeon Kim, Seunghun Lee, Vikas Singh, Hyunwoo J Kim

展开摘要

Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Each update remains a scalar-weighted combination of buffered velocity evaluations, so learning can reduce the in-span teacher mismatch but cannot represent the out-of-span residual exposed by large steps. We propose SpanLift, a lightweight operator-augmented neural solver that enlarges the update family beyond scalar-coefficient solvers. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.

Poster
Learning to Complete Extremal Mathematical Structures

Mengzhou Sun

展开摘要

Extremal combinatorics seeks rare mathematical structures that satisfy hard constraints while maximizing a global objective. Most existing neural systems for mathematical discovery train a model to generate complete configurations from scratch. We argue that this framing mismatches the dynamics of effective search: high-quality solutions are more effectively found by preserving a feasible partial core and conditionally regenerating the remaining uncertain objects. We formulate extremal structure discovery as object-level active-core completion: the model is trained to conditionally complete masked objects given a feasible partial core, capturing inter-object relations rather than fitting a distribution over complete configurations. We instantiate this with a full-attention Transformer over object-aligned slots, and propose Inpainting PatternBoost, a variant of PatternBoost. In each iteration, elite constructions from a top pool are partially destroyed by random or structured remasking; the model learns to reconstruct the missing objects, and their completions are reinserted into the pool. We evaluate on the no-three-in-line problem and maximum sum-of-radii circle packing in the unit square, under a single shared abstraction. On no-three-in-line, our framework recovers the optimal $2n$-point configuration for $n$ up to $18$, and reaches $2n-1$ points for $n=19, 20, 25$, outperforming prior neural baselines. On circle packing, where FlowBoost combines neural generation with gradient-based local search, our approach matches or exceeds FlowBoost when paired with the same local search; even without local search, it reaches an average of 88.1% of FlowBoost's reported score across instances.

Poster
Learning Where and What to Restore for Composite Image Restoration

Jiachen Jiang, Tianyu DING, Ke Zhang, Jinxin Zhou, Tianyi Chen, Ilya Zharkov, Zhihui Zhu, Luming Liang

展开摘要

Real-world degraded images often contain multiple co-occurring degradation types, making composite image restoration fundamentally different from the single-degradation setting assumed by most existing all-in-one methods. These methods typically apply uniform spatial computation and single-label task conditioning, limiting both spatial adaptivity and explicit modeling of degradation mixtures. We propose **CART** (**C**omposite-**A**daptive **R**outing and **T**ask Conditioning), a unified framework that jointly decides *where* to spend computation and *what* degradation cues should guide restoration: a spatial mixer routes patches to graded-capacity experts by local restoration difficulty, and a channel mixer routes each image through degradation-specific experts, conditioned on a global task feature with multi-hot classification supervision. Empirically, CART achieves state-of-the-art performance for composite degradation restoration on CDD-11 and delivers the best results to date on the conventional 3-task and 5-task all-in-one benchmarks.

Poster
Learning Cultural Vectors for Cross-Cultural Generation

Sina Malakouti, Deepti Ghadiyaram, Boqing Gong, Adriana Kovashka

展开摘要

Current text-to-image diffusion models remain unreliable for culturally grounded generation, often producing incorrect or stereotyped depictions, particularly for underrepresented cultures. While prior work has largely focused on evaluation and benchmarking, improving cultural generation remains underexplored. In this work, we study cross-cultural image generation through cultural vectors, defined as the difference between fine-tuned and pretrained model parameters. We show that cultural vectors enable fine-grained control over cultural alignment and failure modes through simple scaling at inference time. We further find that deeper U-Net layers encode stronger cultural information, while cultural vectors across countries occupy largely independent directions in parameter space. However, unlike prior work on task arithmetic, naively combining cultural vectors fails to produce meaningful multi-cultural behavior due to strong interference across cultures. Motivated by this limitation, we propose Descriptor-Guided Merging (DGM), a culturally-aware merging approach that estimates cultural vectors for unseen countries using descriptor-based similarities across cultures. Experiments on two benchmarks across 16/10 countries show that cultural vectors improve generation over pretrained models by 18.9%, while DGM improves cross-cultural generalization by 4.2% over naive merging.

Poster
Learn Locally, Recurse Globally: Neural Circuit Synthesis Beyond Training Depth

Emile Richard

展开摘要

Reasoning, planning, and agentic systems demand that neural models extrapolate from the easy training instances to harder, more compositional problem instances they encounter at deployment. Neural networks trained by gradient descent are reliable interpolators \emph{within} their training domain but struggle to extrapolate beyond it: they learn local generalizers even when global solutions are representable in their architecture class. We propose a \emph{learn locally, recurse globally} paradigm-: train a neural model on in-distribution sub-problems and apply it iteratively under a verifiable outer loop that decomposes hard instances into a sequence of in-distribution sub-instances and reassembles the result. We instantiate this paradigm in Boolean circuit synthesis from truth-tables, which we adopt as a faithful, controllable abstraction of logical reasoning: any propositional inference reduces to evaluating a Boolean circuit, and circuit \emph{depth} is a clean proxy for the difficulty of the underlying reasoning task. Empirically, a $2.2$M-parameter encoder-decoder Transformer trained on depth-$\leq 2$ circuits, wrapped in our outer loops, synthesizes circuits at $5\times$ training depth and at input arities for which the base model's encoder is too small for direct inference, with synthesis quality matching a standard EDA tool while producing markedly shallower circuits. We connect the algorithms to classical Boolean decomposition theory (Shannon, Ashenhurst-Curtis, Reed-Muller) and to recent formal results on the AC$^0$ barrier for constant-depth Transformers.

Poster
LayoutBridge: Anisotropic Brownian Bridges for Public Indoor Floorplan Generation

Tongyang Dai, Ruipeng Gao, Feng Liu, Wenqiu Bai, Lingkun Li, Qiang Ni

展开摘要

Compared with residential layouts, public indoor floorplans present more substantial challenges with open corridors, complex room relations, and non-Manhattan geometries, which do not conform to conventional residential-layout assumptions. Consequently, existing methods are prone to boundary discontinuities, leading to fragmented corridors and irregular wall structures. To address the above issues, we propose \textit{LayoutBridge}, a structure-conditioned generative framework for floorplan production. By moving beyond the Manhattan-geometry assumptions commonly adopted in residential scenarios, LayoutBridge reformulates the task as a latent-space Brownian-bridge diffusion process from structural constraints to target layouts. We have constructed our PISF dataset with 3,517 representative floorplans in public indoor spaces, and LayoutBridge reduces FID by 132.35 and improves BIoU by 15.52\% over the latest image-to-image baselines. Regarding the residential benchmarks MSD and RPLAN, it also reduces FID by at least 2.53 compared with state-of-the-art. These results demonstrate its potential for scalable automated architectural design. Code implementing the proposed method is publicly available at \href{http://github.com/lalalalaxxx/LayoutBridge}{http://github.com/lalalalaxxx/LayoutBridge}.

Poster
Layout Before Pixels: Topology-Anchored Transcriptome-to-Histology Generation

Jianwei Zhao, Xin Li, Fan Yang, Qiang Zhai, Ao Luo, Hong Cheng

展开摘要

Transcriptome-conditioned whole-slide image (WSI) synthesis aims to decode the complex mapping from molecular states to histologic phenotypes. Existing generative paradigms predominantly rely on a direct black-box transcriptome-to-image mapping, tasking a single generator with the simultaneous interpretation of transcriptome signals, inference of multicellular spatial organization, and rendering of fine-grained textures. This formulation, however, is structurally under-constrained; the underlying tissue architecture, the critical substrate linking gene expression to morphology, remains latent and only weakly supervised by pixel-level objectives. We present \textsc{TopoScape}, a topology-anchored framework for transcriptome-to-histology generation under a \emph{Layout-Before-Pixels} principle. Instead of synthesizing pixels directly from transcriptomic embeddings, \textsc{TopoScape} first resolves molecular states into explicit, multi-class cellular topologies via an RNA-guided Topological Prior Generator, reinforced by persistence-based topological regularization. These resolved topologies are further reparameterized into a suite of topology-derived controls, including structure-aware initialization, continuous density fields, and distance representations, which guide a hierarchical, frequency-decoupled flow-matching trajectory. By introducing multicellular organization as an explicit structural mediator, \textsc{TopoScape} allows molecular states to deterministically shape both the macroscopic tissue architecture and the microscopic generative process. Across five TCGA benchmarks, \textsc{TopoScape} consistently outperforms state-of-the-art methods in generative fidelity, biological realism, and downstream predictive utility. Code and pretrained models will be released.

Poster
Latent Refinement Decoding: Enhancing Diffusion Language Models by Refining Belief States

Qinglin Zhu, Yizhen Yao, Runcong Zhao, Yanzheng Xiang, Siya Qi, Amrutha Saseendran, Chen Jin, Philip A Teare, Bin Liang, Yulan He, Lin Gui

展开摘要

Autoregressive (AR) models remain the dominant paradigm for natural language generation, but their strictly sequential decoding process leads to high inference latency. Recent diffusion-inspired language models, such as LLaDA and Dream, alleviate this issue through parallel generation, yet they still face two key limitations: information loss, since predictive distributions over non-finalised tokens are discarded at each step, and unstable commitment dynamics, where local token decisions are not sufficiently coordinated at the global level. We propose Latent Refinement Decoding (LRD), a two-stage decoding framework consisting of Latent Refinement and a Predictive Feedback Loop. In the first stage, LRD keeps masked positions as distributional mixtures of predicted tokens and the mask embedding, enabling the model to form more globally consistent beliefs before committing tokens. In the second stage, LRD progressively finalises confident tokens while preserving uncertain ones for further iterative refinement, with KL-divergence dynamics serving as a stable criterion for convergence and early stopping. Experiments show that LRD improves performance on both coding tasks, including HumanEval by 6.3 and MBPP by 2.6, and reasoning tasks, including GSM8K by 2.9 and MATH500 by 3.8, while achieving up to 10.6× overall speedup. Moreover, LRD is compatible with single-token decoding, multi-token threshold commitment, and system-level accelerators; when combined with Fast-dLLM, it reaches up to 18.4× speedup over vanilla decoding while further improving accuracy.

Poster
Latent Process Generator Matching

Lukas Billera, Hedwig Nora Nordlinder, Ben Murrell

展开摘要

Many recent flow-matching and diffusion-style generative models rely on auxiliary stochastic dynamics during training: a richer process is simulated to define conditional targets, but the auxiliary state is either intractable to sample at generation time or simply not part of the desired output. Existing Generator Matching theory formalises conditioning on static latent random variables, and several recent papers prove special cases of projection results for particular augmented-state constructions. We introduce latent process generator matching, a general framework that treats the observed generative state as a deterministic image $X_t=\Phi(Y_t)$ of a tractable Markov process $Y_t$. We show that in this setting one may learn the generator of a stochastic process on the image space which has the same one-time marginal distributions as the projected process. This generalizes and subsumes the discrete latent process results from the literature, and extends Generator Matching from static latent variables to a rich family of time-dependent latent conditional processes.

Poster