Investigating Distractors with DIF in a College-Level Introductory Computing Assessment

Abstract

Assessing learning at scale often relies on multiple-choice exams, which consist of a stem followed by a correct option and several distractors. While careful design of all parts of a question is essential to ensure assessment validity, each component, including distractors, can introduce bias and unfairly impact specific groups of students. This study extends Differential Item Functioning (DIF) analysis to investigate bias in distractors within an introductory computing assessment for undergraduate students. Using Differential Distractor Functioning (DDF) analysis on responses from 259 students to a computer science introductory level assessment, we identified problematic distractors that exhibited differential performance patterns for male and female students. Our finding provides insights into potential biases within assessments, advancing efforts to create more equitable measures of learning for all students.