Stochastic Approximation with Dynamic Distributions

Abstract

We consider first the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine learning and signal processing literature, under the names of concept drift, stochastic tracking, and performative prediction. In this setting, we provide novel non-asymptotic convergence guarantees for the proximal stochastic gradient method with iterate averaging, focusing on bounds valid both in expectation and with high probability. The efficiency estimates we obtain clearly decouple the contributions of optimization error, gradient noise, and time drift; notably, we identify a low drift-to-noise regime in which the tracking efficiency benefits significantly from a step decay schedule. Next, we analyze a stochastic forward-backward method (SFB) for decision-dependent stochastic approximation problems, wherein the data distribution used by the algorithm evolves along the iterate sequence. The primary examples of such problems appear in performative prediction and its multiplayer extensions. We show that under mild assumptions, the deviation between the averaged SFB iterate and the solution is asymptotically normal, with a covariance that clearly decouples the effects of the gradient noise and the distributional dynamics. Moreover, building on the work of Hájek and Le Cam, we show that the asymptotic performance of SFB with averaging is locally minimax optimal.