Using Universal Screening Measures to Accurately Predict Middle School Students’ Reading Skill Status on a High-stakes State Test: An Investigation of the Psychometric Properties of Vocabulary, Comprehension, and Fluency Measures

Abstract

Multiple studies have evaluated universal screening measures used in response to intervention (RtI) frameworks to identify early elementary school students at-risk of academic failure (e.g. Catts et al., 2015; Compton et al., 2006; Johnson et al., 2009; Schatschneider et al., 2004). Less research has been conducted with middle school students (Baker et al., 2015; Espine et al., 2010; Decker et al., 2014; Allison & Johnson, 2011). The purpose of this study was: first, to extend the growing body of research to middle school students and evaluate the effectiveness of individual universal screening tools specifically for students in sixth grade. Second, to determine if a combination of screening measures might increase the classification accuracy rate of these tools. While we expected prior year high-stakes assessment to serve as a strong predictor of future performance, we also hoped to explore alternate means of identification when prior year assessment data is not available. It was predicted that a combination of screeners would reduce the number of false negative results and therein improve consequential validity. This study utilized correlational, logistic regression, predicted probabilities and ROC curve analysis toward this end. Findings of this study provide similar psychometric results reported from prior research (Denton et al., 2011; Jenkins et al., 2007; Kim et al., 2015). Specifically, this study indicated that proximal measures of reading comprehension, such as prior year MSP and the Gates-MacGinitie (Vocab + Comp), outperform distal measures, like ORF and TOSRE, for students in sixth grade. In addition, when prior year assessment or lengthy measures like the Gates-MacGinitie (Vocab + Comp) are not available, specific combinations of reading measures can improve classification accuracy.