New Lower Bounds for Non-Convex Stochastic Optimization through Divergence Decomposition

Abstract

We study fundamental limits of firstorder stochastic optimization in a range of nonconvex settings, including L-smooth functions satisfying QuasarConvexity (QC), Quadratic Growth (QG), and Restricted Secant Inequalities (RSI). While the convergence properties of standard algorithms are well understood in deterministic regimes, significantly fewer results address the stochastic case, where only unbiased and noisy gradients are available. We establish new lower bounds on the number of noisy gradient queries to minimize these classes of functions, also showing that they are tight (up to a logarithmic factor) in all the relevant quantities characterizing each class. Our approach reformulates the optimization task as a function identification problem, leveraging divergence decomposition arguments to construct a challenging subclass that leads to sharp lower bounds. Furthermore, we present a specialized algorithm in the onedimensional setting that achieves faster rates, suggesting that certain dimensional thresholds are intrinsic to the complexity of nonconvex stochastic optimization.