Informative Variance Priors for Bayesian Multilevel Models

Abstract

This dissertation is focused on the use of informative variance component priors for small- and large-sample Bayesian multilevel modeling. Specifically, I explored the transformative relationships among chi-square, gamma, and inverse-gamma distributions to develop tailored informative priors that may be useful in modeling data with small L2 sample sizes. First, I used Monte Carlo simulation to compare model parameter estimation with maximum likelihood and Bayesian MCMC uninformative, weak informative, and strong informative priors across design conditions including different numbers of clusters (J = 10, 30, 100), cluster sizes (M = 5, 30), intraclass correlations (ICCs; .01, .20, and .40), and levels of explained variance at L1 and L2 (R-squared = .0, .2, .4). The simulation results indicated that use of strongly informative variance component priors (i.e., inverse-gamma with hyperparameters set to have a mode = true variance) was better than Uniform (flat/uninformative) variance priors, and was also better than weak informative priors (i.e., inverse-gamma with shape and scale hyperparameters set to .01 and .01, respectively) when dealing with variables measured on typical education scales (e.g., standard deviations of 100). In addition to simulations, I also demonstrated the use of variance priors to analyze the TIMSS 2019 Grade 8 U.S. Science achievement data, with both the full sample (N = 273 schools) as well as a randomly selected small-sample subset (n = 30 schools). Those results were consistent with the simulation findings but did in fact show that the use of strongly informative priors was best: it captured the full sample’s variance but with the advantage of a smaller credible interval width. Taken together, these findings suggest that, in real-world settings, the use of an informative variance prior is advantageous because it will more precisely capture the true variability in L2 units (the context in which we measure individuals),particularly when the number of L2 units is small.