Are Students Sexist when Rating Each Other? Bias in Peer Ratings and a Generalization of the Rubric-Based Estimator

Abstract

Education research is often conducted using data generated from single response instruments such as exams or assignments. However, more advanced courses often use instruments that are graded by a rubric. Smith and Wooten (2023) introduced a statistical method to assess rubricrow difficulty and proxies of student ability from data generated through rubrics. In this paper, we expand the estimator in Smith and Wooten (2023) to allow for arbitrary parameters in the model; this will allow researchers to use the estimator to answer a much larger set of research questions. We use this expanded estimator to investigate if students exhibit a bias towards their female classmates when rating each other. This is an important question as students often rate each other as part of group projects or early drafts of written assignments. We find evidence that both genders exhibit a bias when rating female students. This is consistent with the Student Evaluations of Teaching literature that finds a negative bias toward female faculty members.

Keywords: peer grading, peer ratings, bias, measuring knowledge, rubric assessment, A22, A23, C63, assessment