Geometry and algorithms for signal recovery: from convex duality to non-convex formulations

Abstract

Structured signal recovery is a central task in a variety of scientific applications, and naturally leads to non-linear and non-convex optimization problems that present many interesting mathematical and algorithmic challenges. In this work, we showcase results for three approaches to solving convex and non-convex optimization problems motivated by structured signal recovery. First, we generalize and strengthen the theory of gauge duality in convex optimization. This includes developing a perturbation framework for gauge duality, establishing optimality conditions, and generalizing the gauge results to all nonnegative and convex functions. Second, we investigate a generalization of subgradient methods, originally designed for convex functions, to functions that are only weakly convex but enjoy the geometric advantage of sharpness. We see that subgradient methods on this class of functions converge at a local linear rate. Finally, we extend our work to the stochastic setting, presenting results for stochastic model-based minimization of functions with high-order growth. These results relax the traditional requirement of a global Lipschitz constant and allow for higher-order growth in the function to be minimized, using the tools of Legendre functions and Bregman divergences. Throughout and wherever possible, we emphasize applications arising in the context of signal recovery, and provide numerical illustrations of our results.