70 Million Canvas Grading Records Reveal Widespread Sequential Bias and System-Induced Surname Initial Disparity

Abstract

The widespread adoption of learning management systems (LMS) in educational institutions has yielded substantial efficiency gains but has also introduced new risks of unequal treatment among students. Using over 70 million grading records from Canvas, we document a widespread sequential grading bias: submissions graded beyond the 60th position receive grades approximately 0.15 standard deviations lower—about 2.5 points on a 100-point scale—than those graded first. This pattern appears in human grading but is absent for automatic grading, is robust across grading orders, model specifications, and sample restrictions, and extends beyond numerical scores to qualitative feedback, which becomes shorter, more negative, and less polite later in the sequence.

We investigate the behavioral mechanisms underlying this bias. Analyses of grading time and breaks provide suggestive evidence of decision fatigue: grades decline with cumulative grading time and partially recover after short breaks. We further use large language model to construct a bias-free benchmark for submission quality. Human graders assign progressively lower scores relative to the AI benchmark for subjective—but not objective—quizzes, indicating that sequential bias primarily reflects drifts in subjective evaluation standards rather than improved accuracy through learning-by-doing. These conclusions are reinforced by heterogeneity analyses across assignment types, grader roles, and course subjects.

Finally, we show that default alphabetical grading order mechanically translates sequential bias into systematic surname disparities. A randomized, message-based intervention implemented in Fall 2025 significantly attenuates the sequential bias. We derive actionable implications for educators, including facilitated grading breaks, standardized rubrics, and bias-aware AI grading assistants.

Keywords: EdTech, education operations, platform design, behavioral bias, service operations management, people-centric operations