Topics on Least Squares Estimation

Abstract

We revisit and make progress on some old but challenging problems concerning least squares estimation. Two major problems are addressed: (i) least squares estimation with heavy-tailed errors, and (ii) least squares estimation in non-Donsker classes. For (i), we study this problem both from a worst-case perspective, and a more refined envelope perspective. For (ii), we perform two case studies in the context of (a) estimation involving sets and (b) estimation of multivariate isotonic functions. Understanding these particular aspects of least squares estimation problems requires several new tools in the empirical process theory, including a sharp multiplier inequality controlling the size of the multiplier empirical process, and matching upper and lower bounds for empirical processes indexed by non-Donsker classes.