Drift-AR: Single-Step Visual Autoregressive Generation via Anti-Symmetric Drifting

Abstract

Autoregressive (AR)-Di_x001B_usion hybrid paradigms combine AR'sstructured semantic modeling with di_x001B_usion's high-_x001C_delity synthesis, yetsu_x001B_er from a dual speed bottleneck: the sequential AR stage and theiterative multi-step denoising of the di_x001B_usion vision decode stage. Exist-ing methods address each in isolation without a uni_x001C_ed principle design.We observe that the per-position prediction entropy of continuous-spaceAR models naturally encodes spatially varying generation uncertainty,which simultaneously governing draft prediction quality in the AR stageand re_x001D_ecting the corrective e_x001B_ort required by vision decoding stage,which is not fully explored before. Since entropy is inherently tied toboth bottlenecks, it serves as a natural unifying signal for joint accel-eration. In this work, we propose Drift-AR, which leverages entropysignal to accelerate both stages: 1) for AR acceleration, we introduceEntropy-Informed Speculative Decoding that align draft_x0015_target entropydistributions via a causal-normalized entropy loss, resolving the entropymismatch that causes excessive draft rejection; 2) for visual decoder ac-celeration, we reinterpret entropy as the physical variance of the initialstate for an anti-symmetric drifting _x001C_eld_x0016_high-entropy positions acti-vate stronger drift toward the data manifold while low-entropy positionsyield vanishing drift_x0016_enabling single-step (1-NFE) decoding without it-erative denoising or distillation. Moreover, both stages share the sameentropy signal, which is computed once with no extra cost. Experimentson MAR, TransDi_x001B_, and NextStep-1 demonstrate 3.8_x0015_5.5× speedup withgenuine 1-NFE decoding, matching or surpassing original quality. Codeis available at https://github.com/aSleepyTree/Drift-AR.