Leveraging Decision Theory to Address Statistical Challenges and Regression Under Additive Nonignorable Missingness

Abstract

This dissertation is composed of three mutually exclusive projects, presented in individual chapters. The first chapter summarizes a novel method for regression estimation under nonignorable missingness. The latter two chapters use decision theory to address two ubiquitous statistical challenges: post hoc test assessment and shrinkage. Chapter 1: Regression Estimation Under Additive Nonignorable Missingness (advised by Mauricio Sadinle, PhD) Existing methods to estimate associations between outcomes subject to missingness and covariates often depend on the assumption that missingness is ignorable. Chapter 1 lays out a novel approach to estimation that allows for nonignorable missingness. Assumptions are limited to the missingness mechanism falling in a broadly-defined class and availability of estimates of marginal means of the outcome from auxiliary data sources or expert opinion. Due to Sadinle and Reiter 2019, such assumptions ensure identifiability of parameters indexing the regression function, which are estimable using the generalized method of moments in all but pathological scenarios. Chapter 2: Decision-Theoretic Approach to Post Hoc Significance Test Assessment (advised by Kenneth Rice, PhD)Binary decisions about parameters of interest---comparing a p-value to a pre-specified field-standardized threshold in a significance test---are ubiquitous in statistical analysis. These binary decisions are appealingly simple, and in some areas, reflect the binary nature of how knowledge will be applied (e.g. allowing a medication to be sold). However, the reliability of such tests is often poorly understood, and the inability of p-values to provide information about reliability is widely overlooked. Moreover, existing measures designed to accompany significance tests to assess their reliability are prone to paradoxical results, misinterpretation, dependence on further arbitrary thresholds, and/or have failed to gain popularity. As an alternative, we consider Bayesian decision-theoretic analogs to one- and two-sided significance tests, which may approximate frequentist Type I error rates and inference when desired (Rice et al., 2020). Loss and risk arise naturally as measures of test reliability from the motivation of the test itself; in Chapter 2, we study feasibility, accuracy, precision, and usefulness of estimates of these metrics. Chapter 3: Decision-Theoretic Justification for Winner's Curse Reduction (advised by Kenneth Rice, PhD) Default estimators often fail to retain their ideal properties (e.g. unbiasedness, admissibility, etc.) when used in non-regular settings. A classic example is the Winner's Curse bias, or upward bias in estimates when reports are limited to significant results. Improvements to default estimators, such as Zhong and Prentice (2008)'s amelioration of the Winner's Curse bias, often distill to shrinkage, or the movement of a complex decision to a simple or parsimonious one. Motivating estimators such as these using a single family of loss functions in a decision-theoretic framework allows for (1) intuitive understanding of the estimator, (2) incorporation of prior information, and (3) a natural framework to compare seemingly dissimilar estimators. We introduce such a family of loss functions in Chapter 3. Our particular focus is on decision-theoretic justification for Zhong and Prentice (2008)'s estimators, their ability to reduce the Winner's Curse bias compared to other estimators in this newly-defined class, and how prior skepticism affects bias reduction.