Neural Shuffle-Exchange Networks - Sequence Processing in O(n log n) Time

摘要

A key requirement in sequence to sequence processing is the modeling of long range dependencies. To this end, a vast majority of the state-of-the-art models use attention mechanism which is of O(n^2) complexity that leads to slow execution for long sequences.