Learning to Diversify Neural Text Generation via Degenerative Model

摘要

works focus on explicitly defining undesirable behaviors and penalizing them in a training phase. Although these studies have shown promising results, we argue that identifying such negative behaviors of models can be laborious and task-dependent. Instead, we propose a novel approach that does not require explicitly specifying the negative behaviors of generation models. Our approach is based on a fundamental observation (§3): Models are misguided by attributes within training examples that may be harmful to reflecting human diversity. Based on the observation, we propose L F D: Learning from Degeneration, a novel approach to remedy degeneration problems in openended applications. Specifically, we first train a model which is designed to Degenerate by amplifying undesirable patterns in examples (§4.2). We then train the second model to enhance its diversity by leveraging the predictions of the first model (§4.3). Experimental results on two representative open-ended generation tasks demonstrate the effectiveness of our approach. In summary, our contributions include: Neural language models often fail to generate diverse and informative texts, limiting their applicability in real-world problems. While previous approaches have proposed to address these issues by identifying and penalizing undesirable behaviors (e.g., repetition, overuse of frequent words) from language models, we propose an alternative approach based on an observation: models primarily learn attributes within examples that are likely to cause degeneration problems. Based on this observation, we propose a new approach to prevent degeneration problems by training two models. Specifically, we first train a model that is designed to amplify undesirable patterns. We then enhance the diversity of the second model by focusing on patterns that the first model fails to learn. Extensive experiments on two tasks, namely language modeling and dialogue generation, demonstrate the effectiveness of our approach.