Stochastic gradient descent with discontinuity across a manifold

摘要

Stochastic gradient descent for a loss function discontinuous across lower dimensional manifolds is analyzed by studying its differential equation limit.