AACL 2023 论文列表

A key feature of neural models is that they can produce semantic vector representations of objects (texts, images, speech, etc.) ensuring that similar objects are close to each other in the vector space.While much work has focused on learning representations for other modalities, there are no aligned crossmodal representations for text and knowledge base (KB) elements.One challenge for learning such representations is the lack of parallel data, which we use contrastive training on heuristics-based datasets and data augmentation to overcome, training embedding models on (KB graph, text) pairs.On WEBNLG, a cleaner manually crafted dataset, we show that they learn aligned representations suitable for retrieval.We then fine-tune on annotated data to create EREDAT (Ensembled Representations for Evaluation of DAta-to-Text), a similarity metric between English text and KB graphs.EREDAT outperforms or matches state-of-theart metrics in terms of correlation with human judgments on WEBNLG even though, unlike them, it does not require a reference text to compare against.

Question: Hi! My wife and I are in our late thirties and going to be in Dublin on September 28 and 29th. We are staying in the Grafton Street area. Does anybody have any suggestions for some fairly priced restaurants with great food within walking distance of Grafton Street? Also, What about some good pubs with live local music? (I realize it is a Sun and Mon night and may be slow) Any suggestion would be appreciated! Thanks! Answer ID: 11_R_4392 Answer Name: The Porterhouse Central, 45-47 Nassau Street, Dublin. Answering real-world tourism questions that seek Point-of-Interest (POI) recommendations is challenging, as it requires both spatial and non-spatial reasoning, over a large candidate pool. The traditional method of encoding each pair of question and POI becomes inefficient when the number of candidates increases, making it infeasible for real-world applications. To overcome this, we propose treating the QA task as a dense vector retrieval problem, where we encode questions and POIs separately and retrieve the most relevant POIs for a question by utilizing embedding space similarity. We use pretrained language models (PLMs) to encode textual information, and train a location encoder to capture spatial information of POIs. Experiments on a real-world tourism QA dataset demonstrate that our approach is effective, efficient, and outperforms previous methods across all metrics. Enabled by the dense retrieval architecture, we further build a global evaluation baseline, expanding the search space by 20 times compared to previous work. We also explore several factors that impact on the model’s performance through follow-up experiments. Our code and model are publicly available at https://github. com/haonan-li/LAMB. 1 Figure 1: An example of real-world POI recommendation question from the TourismQA dataset (Contractor et al., 2021b). Colored text represents constraints relevant to recommending POIs. or vague and ambiguous (e.g., within walking distance of Grafton Street) spatial constraints, as well as a fast indexing method that supports large-scale reasoning over both spatial and non-spatial (e.g., fairly priced restaurants) constraints. Recently, there has been increased interest in geospatial QA. Most approaches focus on querying structured knowledge bases, based on translating natural language questions into structured queries, e.g., using SPARQL (Punjani et al., 2018; Li et al., 2021; Hamzei et al., 2022). Separately, Contractor et al. (2021b) introduced the task of answering POIseeking questions using geospatial metadata and reviews that describe POIs. In later work, they proposed a spatial–textual reasoning network that uses distance-aware question embeddings as input and encodes question–POI pairs using attention (Contractor et al., 2021a). However, as their model creates separate question embeddings for each POI, the inference cost increases linearly in the number of POIs, and the model is incompatible with large

Utterance Intent Label/Short phrase Summary Dataset: CLINC150 give me a hand finding my mobile find phone device how to keep my credit score from improve credit score going down i lost my chase bank card and report lost card want it labeled as lost Dataset: SNIPS Give me Steve‘s address in Man- Get Place Details hattan Get me a table at a restaurant Book Restaurant near Emily’s place for tomorrow 9pm What is the cheapest restau- Compare Places rant between Balthazar and Lombardi’s? Prior work in the field of text summarization mostly focuses on generating summaries that are a sentence or two long. In this work, we introduce the task of abstractive short-phrase summarization (PhraseSumm), which aims at capturing the central theme of a document through a generated short phrase. We explore BART & T5-based neural summarization models, and measure their effectiveness for the task using both standard summarization metrics as well as human evaluation. Our work showcases the benefits of pre-training the summarization models using tasks such as phrasal paraphrase alignment and NLI before fine-tuning on the task itself, both of which help the model with abstraction and thereby yield improvements over the baselines. Human evaluation reveals that model generated summaries are often judged better than or equal to reference summaries, demonstrating that ROUGE scores underestimate true performance. Finally, we create and release a dataset for this task to enable further research in the area. 1 Table 1: Examples of utterances & intent labels from CLINC150 & SNIPS datasets. The intent labels are used as short phrase summaries for our task. Most prior work in both these areas focus on summarizing long documents such as news articles (Eyal et al., 2019; Nenkova et al., 2011; Ahuja et al., 2022), novel chapters (Ladhak et al., 2020; Wu et al., 2021), movie scripts (Chen et al., 2022; Gorinski and Lapata, 2015) etc. to generate summaries that are at least a few sentences in length. However, in this work, we introduce PhraseSumm, a new task of abstractive short-phrase summarization that aims at generating abstractive summaries consisting of a short phrase (2-3 words) from documents that are relatively shorter in length (in our case, mostly consisting of 5-12 words in a single sentence). The short phrase should aim to capture the main theme within the document. To the best of our knowledge, our work is unique in its focus on summarization using short phrases. Our task involves summarizing an input document using a short phrase which is often not present within the document, and can not, therefore, be obtained using extractive techniques alone. Thus, approaches based on topic modeling (Card

We propose a Neighbourhood-Aware Differential Privacy (NADP) mechanism considering the neighbourhood of a word in a pretrained static word embedding space to determine the minimal amount of noise required to guarantee a specified privacy level. We first construct a nearest neighbour graph over the words using their embeddings, and factorise it into a set of connected components (i.e. neighbourhoods). We then separately apply different levels of Gaussian noise to the words in each neighbourhood, determined by the set of words in that neighbourhood. Experiments show that our proposed NADP mechanism consistently outperforms multiple previously proposed DP mechanisms such as Laplacian, Gaussian, and Mahalanobis in multiple downstream tasks, while guaranteeing higher levels of privacy. 1 Figure 1: Anonymizing a target word (shown in red) in a dense (left) vs. a sparse (right) neighbourhoods of words (shown in blue). In the sparse neighbourhood, NADP adds a higher level of perturbation noise z to the target word embedding x in order to protect its privacy by disguising it among its neighbours, while in a dense neighbourhood it adds less noise. the vertices of the nearest neighbour graph) computed using their word embeddings. Second, we compute the connected components in the nearest neighbour graph to find the neighbourhoods of words. Third, we apply Gaussian noise to all words in each neighbourhood, such that the variance of the noise is determined by the words in that neighbourhood.

works focus on explicitly defining undesirable behaviors and penalizing them in a training phase. Although these studies have shown promising results, we argue that identifying such negative behaviors of models can be laborious and task-dependent. Instead, we propose a novel approach that does not require explicitly specifying the negative behaviors of generation models. Our approach is based on a fundamental observation (§3): Models are misguided by attributes within training examples that may be harmful to reflecting human diversity. Based on the observation, we propose L F D: Learning from Degeneration, a novel approach to remedy degeneration problems in openended applications. Specifically, we first train a model which is designed to Degenerate by amplifying undesirable patterns in examples (§4.2). We then train the second model to enhance its diversity by leveraging the predictions of the first model (§4.3). Experimental results on two representative open-ended generation tasks demonstrate the effectiveness of our approach. In summary, our contributions include: Neural language models often fail to generate diverse and informative texts, limiting their applicability in real-world problems. While previous approaches have proposed to address these issues by identifying and penalizing undesirable behaviors (e.g., repetition, overuse of frequent words) from language models, we propose an alternative approach based on an observation: models primarily learn attributes within examples that are likely to cause degeneration problems. Based on this observation, we propose a new approach to prevent degeneration problems by training two models. Specifically, we first train a model that is designed to amplify undesirable patterns. We then enhance the diversity of the second model by focusing on patterns that the first model fails to learn. Extensive experiments on two tasks, namely language modeling and dialogue generation, demonstrate the effectiveness of our approach.

Meeting Transcripts Project Manager Query-Focused Meeting Summarization (QFMS) aims to generate a summary of a given meeting transcript conditioned upon a query. The main challenges for QFMS are the long input text length and sparse query-relevant information in the meeting transcript. In this paper, we propose a knowledge-enhanced two-stage framework called Knowledge-Aware Summarizer (KAS) to tackle the challenges. In the first stage, we introduce knowledge-aware scores to improve the query-relevant segment extraction. In the second stage, we incorporate query-relevant knowledge in the summary generation. Experimental results on the QMSum dataset show that our approach achieves state-of-the-art performance. Further analysis proves the competency of our methods in generating relevant and faithful summaries.1 1 Okay. Push-buttons are buttons not the most expensive , but do add extra cost . Industrial Designer Query What did the group think about the reduction of production costs ? Query-Relevant Knowledge Triple buttons add cost Figure 1: An example of query-relevant knowledge triple extracted from meeting transcripts. The knowledge triple can be used in query-relevant segment extraction as well as summary generation. particular query. Therefore, the models need to reduce the noisy information’s impact effectively. In this paper, we focus on the two-stage framework, extracting query-relevant segments from the meeting transcripts and then generating the summary based on the selected content. Compared to the end-to-end approaches (Zhu et al., 2020; Pagnoni et al., 2022) that directly encode the entire meeting transcripts, the two-stage framework is better at keeping computational efficiency and easier to scale up to longer inputs. Specifically, we propose Knowledge-Aware Summarizer (KAS) that incorporates query-relevant knowledge in both stages. In the first stage, we extract knowledge triples from the text segments and introduce knowledge-aware scores to improve segment ranking. In the second stage, the extracted query-relevant knowledge triples are utilized as extra input for the summary generation. We conduct experiments on a QFMS dataset named QMSum (Zhong et al., 2021) and achieve state-of-the-art performance. We further investigate how the different numbers of extracted segments affect the final performance. In addition, we manually evaluate the generation quality regarding fluency, relevance and factual correctness. Our contributions in this work are threefold: (1) Our work demonstrates the effectiveness of leveraging query-relevant knowledge in QFMS. (2) We

Document-level relation extraction (DocRE) aims to extract relations of all entity pairs in a document. A key challenge in DocRE is the cost of annotating such data which requires intensive human effort. Thus, we investigate the case of DocRE in a low-resource setting, and we find that existing models trained on low data overestimate the NA (“no relation”) label, causing limited performance. In this work, we approach the problem from a calibration perspective and propose PRiSM, which learns to adapt logits based on relation semantic information. We evaluate our method on three DocRE datasets and demonstrate that integrating existing models with PRiSM improves performance by as much as 26.38 F1 score, while the calibration error drops as much as 36 times when trained with about 3% of data. The code is publicly available at https: //github.com/brightjade/PRiSM. 1 Figure 1: An overview of our proposed method. Top represents the original DocRE framework. PRiSM (bottom) leverages relation descriptions to compute scores for each relation triple. These scores are then used to reweight the prediction logits. triples. Under the low-resource setting, chances to observe infrequent relations become much harder. Second, DocRE models predict the NA (“no relation”) label if an entity pair does not express any relation. In DocRED, about 97% of all entity pairs have the NA label. With limited data, there is a much less signal for ground-truth (GT) labels during training, resulting in models overpredicting the NA label instead. High confidence in common relations and the NA label and low confidence in rare relations suggest that models may be miscalibrated. We hypothesize that lowering the former and raising the latter would improve the overall RE performance. At a high level, we wish to penalize logits of frequent labels (including NA) and supplement logits of infrequent labels such that models are able to predict them without seeing them much during training. To implement such behavior, we leverage relation semantic information, which has proved to be effective in low-resource sentence-level RE (Yang et al., 2020; Dong et al., 2021; Zhang and Lu, 2022). In this work, we propose the Pair-Relation Similarity Module (PRiSM) that learns to adapt logits by exploiting semantic information from label descriptions, as depicted in Figure 1. Specifically, we compute a similarity function for each entity pair embedding, constructed from two entities of interest, with relation embeddings, built from corresponding label descriptions. PRiSM then learns re-

5G is the 5th generation cellular network protocol. It is the state-of-the-art global wireless standard that enables an advanced kind of network designed to connect virtually everyone and everything with increased speed and reduced latency. Therefore, its development, analysis, and security are critical. However, all approaches to the 5G protocol development and security analysis, e.g., property extraction, protocol summarization, and semantic analysis of the protocol specifications and implementations are completely manual. To reduce such manual effort, in this paper, we curate SPEC5G the first-ever public 5G dataset for NLP research. The dataset contains 3,547,586 sentences with 134M words, from 13094 cellular network specifications and 13 online websites. By leveraging large-scale pre-trained language models that have achieved state-of-the-art results on NLP tasks, we use this dataset for security-related text classification and summarization. Security-related text classification can be used to extract relevant security-related properties for protocol testing. On the other hand, summarization can help developers and practitioners understand the high level of the protocol, which is itself a daunting task. Our results show the value of our 5G-centric dataset in 5G protocol analysis automation. We believe that SPEC5G will enable a new research direction into automatic analyses for the 5G cellular network protocol and numerous related downstream tasks. Our data and code are publicly available.

The Secretary of Homeland Security was in attendance. Named Entity Recognition (NER) aims to extract and classify entity mentions in the text into pre-defined types (e.g., organization or person name). Recently, many works have been proposed to shape the NER as a machine reading comprehension problem (also termed MRC-based NER), in which entity recognition is achieved by answering the formulated questions related to pre-defined entity types through MRC, based on the contexts. However, these works ignore the label dependencies among entity types, which are critical for precisely recognizing named entities. In this paper, we propose to incorporate the label dependencies among entity types into a multi-task learning framework for better MRC-based NER. We decompose MRC-based NER into multiple tasks and use a self-attention module to capture label dependencies. Comprehensive experiments on both nested NER and flat NER datasets are conducted to validate the effectiveness of the proposed Multi-NER. Experimental results show that Multi-NER can achieve better performance on all datasets. ORG PER Figure 1: An example of nested NER. and Manning, 2009). Figure 1 shows an example of nested NER: Homeland Security can be recognized as O RGANIZATION , as well as P ERSON. To mitigate this issue, many works resort to formulating NER as a Machine Reading Comprehension (MRC) question answering task (termed MRC-based NER) (Wang et al., 2020; Li et al., 2020; Wang et al., 2022). For example, to recognize the O RGANIZATION, a natural-language question “Which O RGANIZATION is mentioned in the text?” is formulated. Then the goal of NER is transformed to answer the formulated questions through machine reading comprehension, given the contexts. MRC-based NER provides a unified solution for both flat and nested NER tasks since each entity type has its corresponding entity span positions as the answer, and these output answers are independent of each other. Despite much progress having been made in MRC-based NER, existing approaches tend to ignore the label dependencies among entity types, which are critical for precise NER. Label dependencies indicate that different entities in the text have some relations with each other. For example, in a sentence “Ousted WeWork founder Adam Neumann lists his Manhattan penthouse for $37.5 million”, once Adam Neumann is recognized as P ERSON, it is expected to help with the recognition

Question answering on tabular data (a.k.a TableQA), which aims at generating answers to questions grounded on a provided table, has gained significant attention recently. Prior work primarily produces concise factual responses through information extraction from individual or limited table cells, lacking the ability to reason across diverse table cells. Yet, the realm of free-form TableQA, which demands intricate strategies for selecting relevant table cells and the sophisticated integration and inference of discrete data fragments, remains mostly unexplored. To this end, this paper proposes a generalized three-stage approach: Table-to- Graph conversion and cell localizing, external knowledge retrieval, and the fusion of table and text (called TAG-QA), to address the challenge of inferring long free-form answers in generative TableQA. In particular, TAG-QA (1) locates relevant table cells using a graph neural network to gather intersecting cells between relevant rows and columns, (2) leverages external knowledge from Wikipedia, and (3) generates answers by integrating both tabular data and natural linguistic information. Experiments showcase the superior capabilities of TAG-QA in generating sentences that are both faithful and coherent, particularly when compared to several state-of-the-art baselines. Notably, TAG-QA surpasses the robust pipeline-based baseline TAPAS by 17% and 14% in terms of BLEU-4 and PARENT F-score, respectively. Furthermore, TAG-QA outperforms the end-to-end model T5 by 16% and 12% on BLEU-4 and PARENT F-score, respectively.