Selection Bias and the Evaluation of Distance Classes: Comparing Traditional and On-Line Economics Course Outcomes
30 Pages Posted: 15 May 2003
Date Written: March 2003
This research examines whether MBA students at a regional, comprehensive university taking completely on-line courses learn as much as students taking identical classes in the traditional, face-to-face format. The courses were an introductory statistics class and a managerial economics class, one of each taught in the traditional format and one of each taught completely on-line. We examine average test scores and value-added (average test scores minus a pre-test score given at the beginning of each class). A simple comparison of test scores indicates that learning outcomes are similar in the two formats. However, controlling for other factors that determine outcomes, regression analysis shows that outcomes in the on-line environment are inferior to the traditional format with respect to the amount of material learned. Given that the choice to take a course in the on-line environment may be related to outcomes, we also use a two stage least squares technique and a switching regression model with endogenous switching to control for potential sample selection bias. The two stage least squares analysis yields findings similar to ordinary least squares estimation. The switching regression analysis shows that students in the on-line class would have learned as much as the students in the traditional class had they chosen that format. However, if students in the traditional class had taken the class in the on-line format they would have not performed as well as the students who actually chose the on-line learning environment.
JEL Classification: A2
Suggested Citation: Suggested Citation