Problems with Ignoring Clustering in Confirmatory Factor Analysis: A Monte Carlo Simulation Study

Abstract

The purpose of the current study is to highlight and extend previous research on the consequences of ignoring cluster dependencies in confirmatory factor analysis (CFA) models, also known as measurement models. Although applied researchers are now well aware that they should avoid violating the independence assumption in univariate analyses (e.g., using multilevel models or unilevel regression with cluster-robust standard errors), some may not be aware that the independence assumption applies to multivariate analyses as well. Assuming the same 2- factor, tau-equivalent model at Level 1 (e.g., students) and Level 2 (e.g., schools), a relatively high within-cluster factor reliability of .90, and a modest intraclass correlation of .20, we specifically studied scenarios in which the Level 2 item-factor relations were either the same or weaker than Level 2 item-factor relations. Our Monte Carlo simulation results show that, when a unilevel factor model is fitted to clustered data, factor loading standard errors will be substantially biased when the Level 1 and Level 2 reliabilities differ, particularly for data structures with large sized clusters. Moreover, irrespective of sample size, a lower Level 2 reliability relative to Level 1 will lead to underestimates of factor reliabilities when clustering is ignored, irrespective of sample size.