Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Abstract

We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the overparametrized regime where the layer width m is larger than the input dimension d.