Abstract

https://ssrn.com/abstract=2850763
 


 



Using Preferred Applicant Random Assignment (PARA) to Reduce Randomization Bias in Randomized Trials of Discretionary Programs


Robert B. Olsen


Rob Olsen LLC

Stephen H. Bell


Abt Associates

Austin Nichols


Abt Associates; The Urban Institute

September 7, 2016


Abstract:     
Heckman and Smith (1995) argued that “randomization bias” can occur in randomized trials when “…random assignment causes the type of persons participating in a program to differ from the type that would participate in the program as it normally operates.” This paper focuses on a form of randomization bias called “applicant inclusion bias”, which can occur in evaluations of discretionary programs that normally choose which eligible applicants to serve. If this selection process is replaced by random assignment, the types of individuals served by the program — and thus its average impact on program participants — could be affected. To estimate the impact of discretionary programs for the individuals that they normally serve, we propose an experimental design called Preferred Applicant Random Assignment (PARA). Prior to random assignment, program staff would identify their “preferred applicants”, those that would have chosen to serve. All eligible applicants are randomly assigned, but the probability of assignment to the program is set higher for preferred applicants than for the remaining applicants. This paper demonstrates the feasibility of the method, the cost in terms in increased sample size requirements, and the benefit in terms of improved generalizability to the population actually served by the program.

Number of Pages in PDF File: 23

Keywords: Impact Evaluation, Random Assignment, External Validity

JEL Classification: C93


Open PDF in Browser Download This Paper

Date posted: October 12, 2016  

Suggested Citation

Olsen, Robert B. and Bell, Stephen H. and Nichols, Austin, Using Preferred Applicant Random Assignment (PARA) to Reduce Randomization Bias in Randomized Trials of Discretionary Programs (September 7, 2016). Available at SSRN: https://ssrn.com/abstract=2850763 or http://dx.doi.org/10.2139/ssrn.2850763

Contact Information

Robert B. Olsen (Contact Author)
Rob Olsen LLC ( email )
Washington, DC
United States
Stephen H. Bell
Abt Associates ( email )
4550 Montgomery Ave #800N
Bethesda, MD 20814
United States
Austin Nichols
Abt Associates ( email )
4550 Montgomery Ave #800N
Bethesda, MD 20814
United States
The Urban Institute
Feedback to SSRN


Paper statistics
Abstract Views: 91
Downloads: 19