Stochastic gradient descent with discontinuity across a manifold

Abstract

Stochastic gradient descent for a loss function discontinuous across lower dimensional manifolds is analyzed by studying its differential equation limit.