NeurIPS 2026 论文列表

Image-to-video generation powers a vast array of downstream applications. However, while the de facto standard, ie, latent diffusion models, typically employ heavily conditioned denoising networks, their decoders often remain unconditional. We observe that this architectural asymmetry leads to significant loss of detail and inconsistency relative to the input image. To address this, we argue that the decoder requires equal conditioning to preserve structural integrity. We focus specifically on image-based generation tasks, where maintaining the structural and textural integrity of the source image is paramount. We introduce RefDecoder, a reference-conditioned video VAE decoder by injecting the reference image directly into the decoding process via reference attention. Specifically, a lightweight image encoder maps the reference frame into the detail-rich high-dimensional tokens, which are co-processed with the denoised video latent tokens at each decoder up-sampling stage. We demonstrate consistent improvements across several distinct decoder backbones (eg, Wan 2.1 and VideoVAE+) achieving consistent improvement on three reconstruction benchmarks. Notably, RefDecoder can be directly swapped into existing video generation systems without additional fine-tuning, and we report across-the-board improvements in subject consistency, background consistency, and overall quality scores on the VBench I2V benchmark. Furthermore, RefDecoder generalizes well to other visual generation tasks such as style transfer.

Image-to-video models often generate videos that remain overly static, compared to text-to-video models. While prior approaches mitigate this issue by weakening or modifying the image-conditioning signal, they often require additional training or sacrifice fidelity to the reference image. In this work, we identify reference-frame dominance as a key mechanism behind motion suppression. We observe that non-reference frames in I2V models allocate excessive self-attention to reference-frame key tokens, causing reference information to be over-propagated across time and suppressing inter-frame dynamics. Based on this finding, we propose DyMoS (Dynamic Motion Slider), a training-free and model-agnostic method that rebalances the attention pathway from generated frames to the reference frame during initial denoising steps. DyMoS leaves both the input image and model weights unchanged and introduces a single scalar parameter for continuous control over motion strength. Experiments across multiple state-of-the-art I2V backbones demonstrate that DyMoS consistently improves motion dynamics while maintaining visual quality and fidelity to the reference image.

Text-aware image super-resolution (TAISR) aims to recover high-resolution images while preserving textual content. Yet some degraded text cannot be restored from local visual evidence alone: humans often read it by reasoning over image context, logical patterns, and prior knowledge. This raises a fundamental question: does TAISR need reasoning? We answer with two new diagnostic tools: ReasonText, a human-curated benchmark with perception-grounded difficulty levels that separate reasoning-required text from locally recoverable text, and GTTCA, a crop-based recognition accuracy metric that isolates restoration quality from detection errors. Evaluating twelve recent SR models, we find a consistent and substantial performance drop on reasoning-required text, revealing a systematic blind spot in current TAISR designs. To address this limitation, we build on two observations: (1) SR models benefit from ground-truth text labels provided as captions, and (2) modern MLLMs can infer degraded text from LR images through reasoning. These observations motivate Reasoning Transfer via Captioning (RTC), a training-free plug-in that uses an external MLLM as a reasoning module and injects its inferred text into SR models through natural-language captions. Surprisingly, with RTC, real-world SR models can even outperform dedicated text-aware SR models, suggesting that reasoning should be treated as a core design axis for future TAISR.

In this paper, we propose **SpectraReward**, a training-free reward function that turns pretrained MLLMs into off-the-shelf reward models for image-generation reinforcement learning. Instead of asking the MLLM to judge a generated image or answer decomposed verification questions, SpectraReward measures how well the original prompt can be recovered from the generated image through a single image-conditioned, teacher-forced forward pass. We use the average image-conditioned prompt log-likelihood as the reward, directly reusing the MLLM's pretrained image-text alignment ability without preference labels, reward-model fine-tuning. We further introduce **Self-SpectraReward,** a special case for unified multimodal models where the policy's own understanding branch serves as the reward model for its generation branch, forming a closed-loop self-improving framework without external reward models or external knowledge. Extensive experiments validate SpectraReward through a broad image-generation RL study covering two diffusion models, three RL algorithms, nine reward MLLM backbones from four MLLM families spanning 4B to 235B parameters, and five out-of-distribution text-to-image benchmarks. Results show that both SpectraReward and Self-SpectraReward significantly and consistently improve generation performance and outperform prior MLLM-derived reward training methods. Further analysis reveals that larger reward MLLMs are not always better, while Self-SpectraReward can match or surpass much larger external reward models, suggesting that reward-policy alignment is a key factor for effective image-generation RL.

Recent pixel-space generative models avoid the reconstruction bottleneck of latent diffusion, but direct denoising over high-dimensional pixel tokens remains computationally demanding. This paper studies a recursive alternative for efficient pixel-space diffusion. We propose RePiD, a hierarchical patch-level diffusion framework that recursively partitions an image and performs the core denoising computation on lower-dimensional pixel groups instead of repeatedly operating on full-resolution images. To preserve spatial coherence across patch-wise processing, RePiD introduces a neighborhood embedding module that conditions each patch on its surrounding regions. Experiments on class-to-image generation and unpaired image-to-image translation show that RePiD provides a favorable quality--efficiency trade-off for direct pixel-space generation. These results suggest that recursive patch-level denoising is a practical design direction for efficient pixel-space generative modeling. Our code is publicly available at \textcolor[HTML]{387FB9}{https://anonymous.4open.science/r/Recursive-Pixel-based-Diffusion-Model-1735}.

In recent years there has been a flurry of activity around using pretrained diffusion models as informed data priors for solving inverse problems, and more generally around steering these models towards certain reward models. Training-free methods like gradient guidance have offered simple, flexible approaches for these tasks, but when the reward is not informative enough, e.g., in inverse problems with highly compressive measurements, these techniques can veer off the data manifold, failing to produce realistic data samples. To address this challenge, we devise a simple algorithm, *ReGuidance*, that leverages prior methods' solutions as strong initializations and substantially enhancing their realism. Given a candidate solution $x$ produced by a given method, we propose inverting the solution by running the unconditional probability flow ODE in reverse starting from $x$, and then using the resulting latent as an initialization for a deterministic steering process. Empirically, we evaluate our algorithm on difficult image restoration tasks including large box inpainting, heavily downscaled superresolution, and high noise deblurring with both linear and nonlinear blurring operations. We find that, using a wide range of baseline methods as initializations, applying our method results in much stronger samples with better realism and measurement consistency. We complement these results with rigorous proofs in commonly studied theoretical settings showing our technique boosts the reward and brings $x$ closer to the data manifold.

Reflections reveal scene content beyond the camera's direct field of view, but faithfully recovering this hidden information is challenging as reflective observations are heterogeneous, incomplete, and physically modulated. Existing methods either rely on explicit reflection modeling with privileged geometric priors or defer to generative synthesis that may hallucinate plausible but unfaithful content. To address these challenges, we propose \textbf{ReFlex}, an observation-constrained framework that casts faithful panorama reconstruction as a controlled inverse problem. We first develop an \emph{Observation-Consistent Estimator} with hierarchical token prediction and signed routing to integrate heterogeneous reflection cues while stabilizing global structure. Subsequently, we propose a \emph{Reliability-Guided Refiner}, which estimates spatial reliability from token predictions and guides a frozen diffusion model to refine uncertain regions without overwriting observation-supported structures. To validate the effectiveness of ReFlex, we further build ReFlex-Bench, a paired benchmark with 110,760 reflective RGB inputs and aligned target panoramas across diverse scenes, reflective geometries, and observation regimes. Experiments show that ReFlex delivers superior reconstruction fidelity without normals, field-of-view annotations, masks, or other privileged information, and obtains the best performance on eight of nine downstream full-view perception protocols.

Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation. A notable limitation of the masked formulation, however, is that once a token has been unmasked it can no longer be revised, leaving dLLMs vulnerable to early sampling mistakes. To address this, a growing body of work has sought to extend masked dLLMs with self-correcting (remasking) capabilities. One appealing subset of these methods does so in a training-free, post-hoc manner based on token confidence, with encouraging early reported results. In this work, we revisit the empirical evaluation of a representative post-hoc remasking method, WINO, and find that under standard decoding settings (shorter block lengths) it brings little-to-no benefit over confidence-based unmasking alone. Extending the evaluation to non-greedy decoding, we find that while confidence-based remasking can mitigate errors introduced by increased stochasticity to some extent, it also exacerbates the diversity collapse previously reported for confidence-based unmasking. Overall, our results show that the benefits of post-hoc confidence-based remasking are highly setting-dependent, underscoring the need for a more comprehensive evaluation framework.

Discrete autoregressive (AR) text-to-image (T2I) models adopt a two-stage paradigm in which a VQ tokenizer maps images to discrete codes and an AR policy models their distribution. Current post-training methods optimize only the AR policy while keeping the VQ decoder frozen. We show that this practice introduces Latent Covariate Shift: as the policy evolves, its token distribution progressively diverges from the ground-truth distribution on which the decoder was trained, such that reward scores improve while decoded image quality degrades. To address this mismatch, we propose **RankE**, the first end-to-end post-training framework for discrete T2I generation. Rather than optimizing the policy against a fixed decoder, RankE co-evolves the policy and decoder through alternating optimization: the policy is refined via group-relative preference optimization, while the decoder is jointly adapted via reward-aware adversarial training. This co-evolution ends the fidelity--alignment trade-off that plagues frozen-decoder approaches: on LlamaGen-XL (775M), standard RL improves CLIP but degrades FID, whereas RankE simultaneously improves both (FID 15.21, CLIP 33.76 on MS-COCO 30K). Consistent joint gains on Janus-Pro (1B) confirm that decoder co-evolution reliably converts reward optimization into pixel-space quality improvements.

Vanilla SVGD is known to have a computational cost of $\mathcal{O}(N^2)$ per iteration. In this work, we introduce Random-Projection Tree Stein Variational Gradient Descent (RP-SVGD) to alleviate this computational challenge. This is achieved by restricting kernel interactions to spatially proximal particles clustered via a random projection tree, further reducing to a cost of $\mathcal{O}(N d(\log (N/C)+C))$, where $C$ is a hyperparameter representing the maximum leaf node capacity of the spanning tree. To establish theoretical validity, we introduce a smoothed RP-Tree kernel for any fixed tree realization and prove that it belongs to the Stein class of the target distribution, thereby ensuring the generation of valid gradient flows. In addition, by considering the effective kernel as the expectation of random tree partitions, we verify that it preserves regularity along with other necessary conditions, which guarantee convergence under the approximate gradient flow framework. Extensive experimental results demonstrate that RP-SVGD tends to have competitive performance and significant speedups across various tasks.

On modern production text-to-image systems, successful policy violations are rare, and previously effective human-written seeds are often patched out. Current automated red-teamers are poorly matched to this regime in two ways: unreliable success measurement and poor exploration. First, we find that judges widely used in prior T2I red-teaming work are unreliable under vague unsafe-content targets: they either miss true violations or reward benign borderline images on hardened APIs. We therefore define strict category-specific success criteria and calibrate strong VLM judges against human labels. Second, we show that broadly used prompt-modification pipelines do not solve the exploration problem: on harder guardrail settings they remain tied to seed prompts, fail to transfer, or cannot bootstrap positive examples. We introduce RISE, which evolves reusable strategies used to generate prompts rather than rewriting them one by one. The best discovered strategies are then reused to generate attacks across new scenarios. On DALL·E 3, Nano Banana 2 (Google) and GPT-Image-2, RISE reaches up to 13\% human-verified ASR; under the same calibrated evaluation, prior methods with reported ASR as high as roughly 30\% fall to near zero.

Capturing screens is common, but photos of emissive displays are often influenced by \emph{flicker-banding} (FB), some alternating bright--dark stripes due to temporal aliasing between a camera's rolling-shutter readout and display's brightness modulation. Unlike moir\'e degradation, FB remains underexplored despite its frequent and severe impact on readability and perceived quality. We formulate FB removal as a dedicated restoration task and introduce Removal of Image Flicker-B}anding via Latent Diffusion Enhancement, RIFLE, a diffusion-based framework designed to remove FB with fine details. We propose Banding-Suppressed High-Frequency Prior (BSHP), which combines gradient-based high-frequency localization with a smooth structural support map to build a compact restoration prior, and injects it into the restoration backbone via multi-stage FiLM modulation. Moreover, Masked Loss (ML) is proposed to concentrate supervision on banded regions without sacrificing global fidelity. To overcome data scarcity, we provide a simulation pipeline synthesizing FB in the luminance domain with stochastic jitter in banding angle, spacing, and width. Feathered boundaries and sensor noise are also applied for a more realistic simulation. For evaluation, we collect a paired real-world FB dataset with pixel-aligned banding-free references captured via long exposure. Across quantitative metrics and visual comparisons on our real-world dataset, RIFLE consistently outperforms recent image reconstruction baselines. To the best of our knowledge, it is the first work to research the simulation and removal of FB. Our dataset and code will be released soon.

RNA molecules often function through multiple experimentally observed conformational states. A central challenge is to capture ensemble diversity without sacrificing structural fidelity: samples may collapse toward averaged representative structures or drift into unsupported conformational regions. We introduce $\texttt{REFLEX}$ (\underline{R}NA \underline{E}nsemble generation calibrated by \underline{FLEX}ibility), a sequence-conditioned generator for RNA backbone-frame ensembles based on a heteroscedastic stochastic bridge on $\mathrm{SE}(3)$. During training, experimental $\mathrm{B}$-factors calibrate residue-wise bridge widths, exposing each observed conformer through local perturbation neighborhoods with residue-specific scales while keeping constrained regions sharper. A learned flexibility condition guides the bridge-field network to recover target conformer-specific frames from these calibrated noisy states. At inference, $\texttt{REFLEX}$ uses only the input sequence, requiring no MSA-subsampling heuristics. On held-out multi-conformer RNA clusters, $\texttt{REFLEX}$ achieves the best precision--recall trade-off for ensemble coverage across evaluation thresholds, while ablations indicate that the learned bridge field recovers conformer-specific basins more reliably.

Latent diffusion models (LDMs) achieve high-fidelity generation by moving diffusion from pixels to a compact latent space, but standard tokenizers are optimized for compression, not robustness under diffusion-time corruption. We formulate latent tokenization as diffusion-aware joint source--channel coding and diagnose standard LDM pipelines as missing an explicit channel-coding step. A diffusion-aware rate--robustness functional admits an explicit spectral characterization in $G^\top G$, yielding the Redundancy-Balance Principle: optimal redundancy is not uniform, but aligns with semantic sensitivity and equalizes marginal integrated-risk reduction across active directions. The resulting optima take weighted tight-frame and water-filling forms; isotropic bottlenecks are minimax-suboptimal under heterogeneous sensitivity, and a group-conditioned extension identifies rare groups as capacity-limiting under a fixed latent budget. We instantiate the theory as RB-LDC (Redundancy-Balanced Latent-Diffusion Coding), a lightweight tokenizer modification that inserts a learned coding layer before diffusion. Six controlled experiments validate the theorem sequence up to $k{=}1024$, with a $53.5\times$ alignment gap, $659\times$ minimax gap, and real-image VAE learnability via Adam recovery of the KKT spectrum with relative error below 0.5 percent. On SD-VAE and VA-VAE at ImageNet-256, RB-LDC outperforms isotropic uniform redundancy in all 20 perturbed metric cells over a 15-SNR grid; VA-VAE block-local obtains $-15.40$ average pFID and $-105.79$ peak pFID at $\log_{10}\rho{=}+1$, while preserving clean rFID parity (gap $-0.0001$). RB-LDC turns latent tokenizer design into a reliability problem for diffusion models.

Despite rapid advances in text-to-image (T2I) generation, current models often struggle with rare, compositional, and culturally grounded concepts where training data is sparse and visual priors are weak or absent. Increasing model capacity or prompt detail alone does not consistently resolve these challenges, particularly in long-tail settings where attribute grounding remains unstable due to attention skew, semantic collapse, and needle-in-a-haystack retrieval dynamics. We introduce RAVEL, a training-free framework that improves rare concept generation, context-driven image editing, and self-correction by integrating graph-based retrieval-augmented generation (RAG) into T2I pipelines. Unlike prior RAG and LLM-enhanced approaches that rely on visual exemplars, static captions, or implicit model knowledge, RAVEL leverages structured knowledge graphs to retrieve targeted compositional, symbolic, and relational context, enabling more precise grounding even in the absence of visual priors. To further refine generation quality, we propose SRD, a self-correction module that iteratively updates prompts using multi-aspect alignment feedback, improving attribute accuracy, narrative coherence, and semantic fidelity. Our framework is model-agnostic and is compatible with leading diffusion and autoregressive models, including Stable Diffusion XL, Flux, DALL-E 3, GLM-Image, and Janus-Pro. We conduct extensive evaluations across three newly proposed benchmarks: MythoBench, Rare-Concept-1K, and NovelBench, and show consistent improvements over SOTA baselines across perceptual, alignment, and LLM-as-a-Judge metrics, while also demonstrating stable and interpretable refinement trajectories for rare concepts across diverse model families. These results position RAVEL as a simple and robust approach for controllable and interpretable T2I generation in long-tail domains. Our code is available at: https://anonymous.4open.science/r/ravel-54E5/

The most widely used RANSAC variants score candidate models by counting inliers or summing truncated likelihoods; every such score requires a user-supplied parameter that is a function of the inlier scale, which must itself be estimated from contaminated data. We remove this dependence by reversing the usual order of inference: for a fixed inlier partition we marginalize $\sigma$ analytically in closed form under a conjugate Inverse-Gamma prior, then optimize over partitions. A single closed-form expression spans the non-informative Jeffreys limit (which requires no validation data to fit the prior) and informative empirical-Bayes priors fit from a small validation set, so the same score adapts across data-rich and data-scarce regimes without any change to the algorithm. To our knowledge this is the first RANSAC score in which the inlier scale is genuinely absent from the score formula. The score admits $O(N \log N)$ computation via sort-and-sweep. On a benchmark of nearly $70\,000$ image pairs spanning different two-view estimation problems and both engineered and learned feature pipelines, the proposed score matches or exceeds the state of the art (RANSAC, MSAC, GaU, MAGSAC++), excelling in robustness to hyperparameter miscalibration, sample efficiency at small validation budgets, and adaptive regularization across data-rich and data-scarce regimes.

Modern image generative models are able to produce photorealistic images. As those images become increasingly indistinguishable from real data and are published online, they are often scraped for subsequent training runs of new generative models. This practice of training on generated data has been shown to degrade model performance and cause model collapse. A possible mitigation lies in embedding radioactive watermarks into generated content. Radioactive watermarks are robust marks that are detectable in outputs of new models trained on watermarked data, enabling provenance tracing of generated content. In this work, we analyze the persistence of image watermarks across multiple training-generation runs. To do so, we introduce a novel statistical testing method RADIUM (RadioActive Decay of Image-Underlaid Marks) for reliable radioactivity detection across various watermarking methods. Using our RADIUM method, we observe disparate radioactivity across watermarking methods for image generative models. Only few watermarks remain detectable in subsequently trained models, while most decay severely, especially when the architecture of models differs. Our analysis highlights the critical need for more radioactive watermarking methods in the vision domain.

Specifying segmentation targets via text descriptions provides flexible semantic conditioning for medical image segmentation. However, existing methods still struggle to robustly translate high-level textual semantics into accurate pixel-level spatial prompts. In this paper, we propose RADAR (Residual Aggregation and Dense Alignment Representations), a parameter-efficient framework for text-guided medical image segmentation. RADAR explicitly decouples and synergizes the medical semantic anchoring capabilities of BiomedCLIP with the prompt-driven mask decoding abilities of MedSAM2, achieving low-cost task adaptation via LoRA. Specifically, RADAR introduces Guided Residual Depth Aggregation, which adaptively aggregates historical layer features under semantic guidance to mitigate textual semantic injection lag and deep feature dilution. Furthermore, we propose Structured Dense Representation Alignment, which leverages a frozen BiomedCLIP to construct crop-level semantic anchors, namely target, boundary, and context, to enhance the local discriminative power of dense features and the reliability of automated prompts. Additionally, Multi-modal Prompt Generation is designed to transform cross-modal semantic responses into composite prompts consisting of text, points, and masks. Experimental results on three public medical image segmentation datasets demonstrate that RADAR achieves competitive performance while introducing merely 0.74M trainable parameters, validating the effectiveness of the proposed approach in parameter-efficient adaptation and the enhancement of cross-layer and local representations. The source code is provided in the supplementary material.

Machine learning methods rely on data. However, gathering suitable data can be challenging due to availability constraints, cost, or the need for domain expertise. Expanding datasets with additional sources is a common response to limited data, yet this practice does not always improve downstream performance and can sometimes lead to a loss of performance, known as negative transfer. We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models. RADAR analyzes the layer-wise evolution of representations by measuring angular alignments and relative changes in distance along layer-to-layer displacement trajectories, and by comparing empirical distributions of within-domain and cross-domain dynamics. We hypothesize that domain transferability is related to the divergence between these trajectory distributions. We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text embedding models and cross-domain image classification with foundation vision models. Across several settings, RADAR provides competitive predictive performance relative to existing transferability metrics on several vision and text benchmarks, with particularly strong results when domain transitions are smooth or cleanly separated. Our ablations further suggest that the effectiveness of transferability estimation depends on the geometry of the model’s internal representation space, with different modalities favoring different topological formulations.