The Unvarnished Report of Conducting an RCT of a Statewide STEM Program in the Process of Scaling Up

22 Pages Posted: 9 Nov 2014

See all articles by Denis Newman

Denis Newman

Empirical Education Inc.

Andrew Jaciw

Empirical Education

Date Written: October 31, 2014

Abstract

We were responsible for a large scale randomized control trial to evaluate the effectiveness of the Alabama Math Science and Technology Initiative (AMSTI). The official report released by IES after considerable technical review, editing, and formatting, is what we call the varnished version. This chapter delves into a number of the nuts and bolts issues and lessons-learned in conducting this study and provides an unvarnished version of the story. Our study has provided a strong example of taking advantage of a planned scale-up of a program by an education agency to conduct an RCT. We provide five lessons we learned from processes that arose behind the scenes and highlight the importance of exploratory findings. These include how the research questions evolved, how the approach to randomization can help in working with a local agency, reasons for difficulty in maintaining a control group embargo when working in the context of a locally-funded scale-up, issues in avoiding bias in the impact model selection, and what the local agency should make of the results of an RCT.

Keywords: AMSTI, RCT, Alabama Math Science and Technology Initiative, research questions, impact model selection, IES, education agency, locally-funded, scale-up, control group, randomization, exploratory findings

Suggested Citation

Newman, Denis and Jaciw, Andrew, The Unvarnished Report of Conducting an RCT of a Statewide STEM Program in the Process of Scaling Up (October 31, 2014). Available at SSRN: https://ssrn.com/abstract=2519677 or http://dx.doi.org/10.2139/ssrn.2519677

Denis Newman (Contact Author)

Empirical Education Inc. ( email )

425 Sherman Ave, #210
Palo Alto, CA 94306
United States

HOME PAGE: http://empiricaleducation.com/

Andrew Jaciw

Empirical Education ( email )

2955 Campus Dr
110
San Mateo, CA 94403
United States

HOME PAGE: http://https://empiricaleducation.com/

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
65
Abstract Views
1,314
Rank
730,641
PlumX Metrics