A Finite Approximation Framework for Infinite Dimensional Functional Problems

Abstract

It is often of interest to non-parametrically estimate regression functions. Penalized regression (PR) is one effective, well-studied solution to this problem. Unfortunately, in many cases, finding exact solutions to PR problems is computationally intractable. In this manuscript, we propose a \textit{mesh-based approximate solution}, or MBS, for those scenarios. MBS transforms the complicated functional minimization of PR, to a finite parameter, discrete convex minimization allowing us to leverage the tools of modern convex optimization. We show applications of MBS for both univariate and multivariate regression with a number of explicit examples (including isotonic regression and partially linear additive models), and explore how the number of parameters must increase with our sample-size in order for MBS to maintain the rate-optimality of PR. We also give an efficient algorithm to minimize the MBS objective while effectively leveraging the sparsity inherent in MBS.