Structure-Exploiting and Accelerated Methods for Convex Optimization

Abstract

Convex optimization has matured into a reliable technology, with efficient algorithms and software available for most problem classes. Despite this maturity, developing solvers that are faster on small problems and capable of scaling to larger ones remains important. Latency-sensitive applications such as real-time trajectory optimization, real-time control, and portfolio backtesting demand fast solves on smaller problems, and applications in supply chain optimization, machine learning, and signal processing require solvers that scale to very large problems. This dissertation presents three main contributions towards these goals, two practical and one theoretical. First, we present QOCOGEN, a custom solver generator for quadratic-objective second-order cone programs (SOCPs). QOCOGEN autocodes a primal-dual interior point method that exploits the sparsity structure of the problem at hand, achieving nearly a 4x speedup over QOCO, a generic implementation of the same algorithm. Second, we develop a GPU-accelerated backend for QOCO to solve large quadratic-objective SOCPs, achieving nearly a 70x speedup over QOCO’s CPU backend on some large problems. Both QOCO (with its CPU and GPU backends) and QOCOGEN are open-source and can be called from modeling languages such as CVXPY and CVXPYgen. Third, we derive Fast Douglas–Rachford splitting (FDR), an accelerated operator-splitting method for minimizing the sum of a convex and a strongly convex function. FDR achieves an accelerated O(1/N^2) convergence rate for the squared distance to the optimal solution, with a leading constant that is a factor of four better than the fastest previously known methods. We also construct a complexity lower bound establishing the optimality of both FDR's O(1/N^2) convergence rate and the leading-order constant.