Handling Item Clustering using 2PL IRT Modeling in an SEM Framework : A Demonstration with PISA 2012 Computerized Math Problems

Abstract

This paper extends earlier work by Costa et al. (2021) on a method for incorporating item-varying response times into binary latent trait modeling using U.S. item-level data from the Programme for International Student Achievement (PISA) 2012 dataset. Specifically, we demonstrate a 2-step factor analytic approach that incorporates three item clusters as nuisance “method” factors with 10 math problem solving items. Step 1 involves estimating separate latent speed and latent trait factor models that constrain the latent variable scales to unit normal (to estimate measurement model item parameters), and step 2 involves estimating a joint model with constraints placed on factor loadings using step 1 item parameter estimates (to estimate the latent variances and structural parameters). Despite missing data issues, we show that factor reliabilities improved in step 1 for both latent speed and latent trait variables, and that the latent trait reliability improved further when latent speed is taken into account in step 2. Our results suggest that: 1) the factor model 2-step method is a viable alternative to high-dimensional item response theory (IRT) model parameterization (which is the case when items are clustered with common prompts), and 2) consistent with Costa et al. (2021), inclusion of process data in modeling item responses may improve measurement reliability.