Treepiece: Faster Semantic Parsing via Tree Tokenization

Abstract

\emph{Autoregressive} (AR) encoder-decoder neural networks have proved successful in many NLP problems, including \emph{Semantic Parsing} -- a task that translates natural language to machine-readable \emph{parse trees}. However, the sequential prediction process of AR models can be slow. To accelerate AR for semantic parsing, we introduce a new technique called \emph{TreePiece} that tokenizes a parse tree into subtrees and generates one subtree per decoding step. On TOPv2 benchmark, TreePiece shows $4.6$ times faster decoding speed than standard AR, and comparable speed but significantly higher accuracy compared to \emph{Non-Autoregressive} (NAR).