STAT: Soft Tail-dropping for Adaptive Visual Tokenization

摘要

We present Soft Tail-dropping Adaptive Tokenizer (STAT),a discrete tokenizer that learns adaptive visual representations. STATadjusts the number of tokens allocated to each image according to itsperceptual complexity. Specifically, it encodes an image into discrete to-kens together with token-wise keep probabilities indicating whether eachtoken is necessary for faithful reconstruction or can be safely dropped.Through this learned adaptivity, STAT achieves state-of-the-art recon-struction quality while using fewer tokens on average. When integratedwith vanilla causal autoregressive (AR) modeling, STAT enables a content-aware generative model with adaptive-length sampling. The model achievescompetitive or superior visual generation quality compared with othergenerative model families while exhibiting favorable scaling behavior thathas been elusive in prior vanilla AR visual generation attempts.