Lord's Paradox and Targeted Interventions: The Case of Special Education

Abstract

Lord (1967) describes a hypothetical “paradox” in which two statisticians, analyzing the same dataset using different but defensible methods, come to very different conclusions about the effects of an intervention on student outcomes. I use graphical methods—including a new graphical framework called Single World Object Oriented Plates (SWOOPs)—and detailed, longitudinal data about all public school students in Washington State to investigate a real-life example of Lord’s Paradox that arises in evaluating the impact of special education services on student performance. I then introduce an instrumental variables (IV) approach that exploits a threshold in the state’s special education funding laws that caps per-pupil special education funding at 12.7% of a district’s students, and use SWOOPs to argue that the assumptions that justify this approach are more plausible than the assumptions that justify the methods in the existing literature. I find that students in districts that pass this threshold are far less likely to be placed in special education, all else equal, and use a district’s position relative to this funding threshold as an instrumental variable to estimate the local average treatment effect of special education services on student test performance.