PromptRL: Prompt Matters in RL for Flow-Based Image Generation

摘要

Flow matching models (FMs) have revolutionized text-to-image (T2I) generation, with reinforcement learning (RL) serving as a critical post-training strategy. We show that current RL pipelines for FMs suffer from two underappreciated limitations: sample inefficiency from collapsed generation diversity, and prompt overfitting, where models memorize training phrasings and collapse on semantically equivalent paraphrases. We present **PromptRL** (**P**rompt **M**atters in **RL** for Flow-Based Image Generation), a framework that incorporates language models (LMs) as trainable prompt refinement agents directly within the flow-based RL loop, yielding both a learned prompt rewriter and a synergistic training regime that reshapes optimization dynamics. PromptRL attains state-of-the-art scores of **0.97** on GenEval, **0.98** on OCR accuracy, and **24.05** on PickScore, and on instructional editing it improves FLUX.1-Kontext from **1.19** to **1.43** EditReward with only **0.06M** rollouts, surpassing Gemini 2.5 Flash Image (Nano Banana, **1.37**) and matching ReasonEdit-Think (**1.44**), which relied on fine-grained data annotations and a multi-stage pipeline. Our experiments show that PromptRL achieves higher performance ceilings with approximately **50% fewer rollouts** than naive flow-only RL. Our code is available at [https://github.com/G-U-N/UniRL](https://github.com/G-U-N/UniRL).