What Learned Representations and Influence Functions Can Tell Us About Adversarial Examples

摘要

Work in the NLP space has followed that in image processing. Here, in addition to the goal of Adversarial examples, deliberately crafted usimpacting the model’s prediction, adversarial text ing small perturbations to fool deep neural netexamples need to be syntactically and semantically works, were first studied in image processing sound to the reader. Consequently, adversarial atand more recently in NLP. While approaches to detecting adversarial examples in NLP have tack techniques on text use semantics-preserving largely relied on search over input perturbatextual changes at the character level, word level tions, image processing has seen a range of and phrase level or sentence level (Pruthi et al., techniques that aim to characterise adversarial 2019; Alzantot et al., 2018; Li et al., 2020, for exsubspaces over the learned representations. ample). Table 1 illustrates two examples, showing In this paper, we adapt two such approaches different types of attack formulation in NLP. to NLP, one based on nearest neighbors and In the image domain, defence against adversarinfluence functions and one on Mahalanobis ial attack can be ‘proactive’ or ‘reactive’ (Cohen distances. The former in particular produces a state-of-the-art detector when compared et al., 2020), where proactive defence refers to imagainst several strong baselines; moreover, the proving the model’s robustness (Madry et al., 2018; novel use of influence functions provides inGopinath et al., 2018; Cohen et al., 2019) and reacsight into how the nature of adversarial examtive defence focuses on detecting real adversarial ple subspaces in NLP relate to those in image examples before they are passed to neural networks processing, and also how they differ depending (Feinman et al., 2017; Ma et al., 2018; Lee et al., on the kind of NLP task. 2018; Papernot and McDaniel, 2018). Broadly