Model-Based Penalized Regression

Abstract

This thesis contains three chapters that consider penalized regression from a model-based perspective, interpreting penalties as assumed prior distributions for unknown regression coefficients. In the first chapter, we show that treating a lasso penalty as a prior can facilitate the choice of tuning parameters when standard methods for choosing the tuning parameters are not available, and when it is necessary to choose multiple tuning parameters simultaneously. In the second chapter, we consider a possible drawback of treating penalties as models, specifically possible misspecification. We introduce an easy-to-compute moment-based misspecification test for the Laplace prior, argue that the risk of misspecification calls for consideration of a larger class of penalties and corresponding prior distributions, and define easy-to-compute moment-based unknown prior parameters that yield improved estimation of the unknown regression coefficients in simulations. In the third chapter, we introduce structured shrinkage priors for dependent regression coefficients which generalize popular independent shrinkage priors. These can be useful in various applied settings where many regression coefficients are not only expected to be nearly or exactly equal to zero, but also structured.