Benchmarking Ultra-Low-Power μNPUs
Authors: Josh Millar, Yushan Huang, Sarab S. Sethi, Hamed Haddadi, Anil Madhavapeddy
Abstract
Efficient on-device neural network (NN) inference offers predictable latency, improved privacy and reliability, and lower operating costs for vendors than cloud-based inference. This has sparked recent development of microcontroller-scale NN accelerators, also known as neural processing units (μNPUs), designed specifically for ultra-low-power applications.