Regularization Approaches to Detect Differential Item Functioning: Multiple Covariates, Polytomous Response, and Multidimensional Traits

Abstract

In the field of educational measurement, ensuring that assessment instruments are free from bias is crucial. Differential Item Functioning (DIF) indicates a potential bias in test items, implying that different covariate groups, despite having the same latent trait level, have differing probabilities of responding correctly to an item. Such biases could lead to unfair advantages or disadvantages for certain groups. While several methods exist for DIF detection, there is a growing need for more efficient and robust techniques, especially in situations with multiple covariates and multidimensional latent traits. To address this gap, I propose utilizing regularization methods to detect DIF associated with multiple covariates and multidimensional polytomous response data. The proposed algorithm concurrently estimates DIF effects across numerous covariates, encompassing both continuous and categorical variables. An additional advantage of our approach is the elimination of the necessity for anchor items, thus simplifying the DIF detection process. Furthermore, I introduce two distinct methods to model the impact of covariates on the covariance matrix of the multidimensional latent trait. The first method employs Cholesky decomposition of the covariance matrix, while the second method utilizes covariance regression, which can effectively handle high-dimensional latent traits and a large number of covariates. By incorporating these procedures into our regularization-based DIF detection framework, the proposed method can accurately and effectively recover the impact of these covariates. To assess the performance of the proposed methods, I conduct three simulation studies. The first study focuses on examining uniform DIF associated with three covariates in a two-dimensional graded response model (GRM). The second study explores non-uniform DIF associated with the same set of covariates within the framework of the two-dimensional GRM. Lastly, the third simulation study investigates the efficacy of the group Lasso algorithm under the condition of uniform DIF associated with three covariates using the two-dimensional two-parameter logistic (2D2PL) model. Following the simulation studies, I conduct a real data analysis using the Patient-Reported Outcomes Measurement Information System (PROMIS) dataset. In conclusion, this dissertation presents an emerging approach to addressing the challenges of DIF detection within the complex settings of multiple covariates, polytomous responses, and multidimensional latent traits. By incorporating advanced regularization methods and innovative impact modeling techniques, I aim to enhance the efficacy and robustness of identifying potential biases in educational assessments. The encouraging results from the simulation studies and real data analysis suggest that the proposed methods could significantly contribute to improved fairness and precision in educational measurement.