Do Students Know Best? Choice, Classroom Time, and Academic Performance

Abstract

We compare student academic performance in traditional twice-a-week and compressed once-a-week lecture formats in introductory microeconomics between one semester in which students were randomly assigned into the formats and another semester when students were allowed to choose their format. In each semester we offered the same course with the sections taught at the same times in the same classrooms by the same professors using the same book, software and lecture slides. Our study design is modeled after a doubly randomized preference trial (DRPT), which provides insights regarding external validity beyond what is possible from a single randomized study. Our goal is to assess whether having a choice modifies the treatment effect of format. Students in the compressed format of the randomized arm of the study scored -0.19 standard deviations less on the combined midterm and final (p<.01) and -0.14 standard deviation less in choice arm (p<.01). There was little evidence of selection bias in choice of format. Future analyses of online learning formats employing randomization should consider DRPT designs to enhance the generalizability of results.