Culturally Responsive Measurement: Machine Learning and Artificial Intelligence Approaches to DIF Detection, Parameter Estimation, and Scoring

Abstract

The validity of educational and psychological assessments rests on the assumption of measurement equivalence across examinee groups. Differential Item Functioning (DIF) analysis serves as a primary mechanism for evaluating this assumption by identifying items that potentially function differently across grouping categories after controlling for ability. However, traditional marginal DIF approaches model grouping categories independently, implicitly assuming that identity dimensions operate additively. Intersectionality theory directly challenges this separability assumption, arguing that an individual's identity is composed of multiple intersecting dimensions whose combined influence cannot be expressed as the sum of their parts. An argument also supported by Culturally Responsive Measurement frameworks. Intersectional DIF extends marginal DIF to address this limitation, modeling group membership as the interaction of multiple grouping characteristics rather than a single dimension. Operationalizing intersectional DIF analysis therefore requires modeling interactions among many grouping characteristics simultaneously. But, doing so creates high-dimensional subgroup structures with many small groups with complex dependency patterns among pairwise groups. Existing psychometric methods are not well equipped for these high-dimensional, sparse subgroup structures. When marginal DIF procedures are extended to intersectional settings, the multiple pairwise comparisons and small subgroup sizes often lead to inflated Type I error rates. Regularization-based approaches improve Type I error control under these conditions, but typically at the cost of reduced Power to detect DIF. Additionally, identifying DIF is only the first step. Valid latent ability estimation requires translating intersectional DIF findings into coherent measurement invariance constraints across groups. Conventional clustering and invariance-structuring techniques, however, treat comparisons independently and do not enforce the transitivity assumptions that group-wise invariance imposes. This leads to logically impossible or false constraint structures and poorly estimated ability estimates. This dissertation develops and validates an integrated framework designed specifically to solve these challenges. The first contribution, InterDIFNet, reframes intersectional DIF detection as a supervised multilabel classification task. Rather than evaluating pairwise comparisons independently, the model processes a high-dimensional feature representation derived from group-specific item parameter estimates and learns nonlinear patterns predictive of DIF across all items and subgroup comparisons simultaneously. By exploiting dependency structure across comparisons, InterDIFNet improves stability under sparse subgroup conditions. A data-driven threshold optimization procedure selects classification boundaries that balances Type I Error control with the Power to detect DIF. Simulation studies demonstrate that InterDIFNet achieves over 90% detection power in ten-group scenarios while maintaining substantially improved Type I error control relative to current Intersectional DIF detection methods. The second contribution, Transitive DIF Clustering (TDC), translates the probabilistic, pairwise DIF outputs from InterDIFNet into psychometrically coherent measurement invariance structures. By representing intersectional groups as nodes in a graph, TDC enforces transitivity to ensure logically consistent clustering of groups for item parameter estimation. The resulting partial invariance models support accurate latent ability estimation while avoiding over- or under-constrained structures. Across simulation conditions, TDC recovers true invariance patterns with markedly lower error rates than conventional clustering approaches. An empirical application to a college-level computer science assessment further illustrates the framework's interpretability, revealing complex mixed DIF patterns across intersectional groups that prior analyses had not identified. Together, InterDIFNet and TDC establish a coherent end-to-end workflow for intersectional measurement fairness, integrating probabilistic DIF detection, structurally consistent invariance modeling, and valid ability estimation. By combining intersectionality-informed theory with modern machine learning and graph-theoretic methods, this dissertation demonstrates that rigorous and scalable intersectional DIF analysis is both computationally feasible and psychometrically principled.