Table of Contents

The State Capacity Ceiling on Tax Rates: Evidence from Randomized Tax Abatements in the DRC

Augustin Bergeron, Harvard University
Gabriel Tourek, Massachusetts Institute of Technology (MIT)
Jonathan Weigel, Harvard University

Understanding Performance in Test Taking: The Role of Question Difficulty Order

Lina Anaya, University of Arkansas - Department of Education Reform
Nagore Iriberri, University of the Basque Country
Pedro Rey Biel, University College London
Gema Zamarro, University of Arkansas - Department of Education Reform, Center for Economic and Social Research (CESR)

Are High-Interest Loans Predatory? Theory and Evidence from Payday Lending

Hunt Allcott, New York University (NYU)
Joshua J. Kim, Facebook Inc
Dmitry Taubinsky, University of California, Berkeley - Department of Economics
Jonathan Zinman, Dartmouth College, Innovations for Poverty Action, Jameel Poverty Action Lab, National Bureau of Economic Research (NBER)

Virtual Watercoolers: A Field Experiment on Virtual Synchronous Interactions and Performance of Organizational Newcomers

Iavor Bojinov, Harvard University - Technology & Operations Management Unit
Prithwiraj Choudhury, Harvard University - Business School (HBS)
Jacqueline N. Lane, Harvard Business School

Does Computer-Aided Instruction Improve Children’s Cognitive and Noncognitive Skills?

Hirotake Ito, Keio University - Graduate School of Media and Governance
Keiko Kasai, School of International Development, University of East Anglia
Hiromu Nishiuchi, Yokohama City University - Graduate School of International Management
Makiko Nakamuro, Keio University

How to Increase Survey Response and Manage the Postal and Face-to-Face Field Work of Participant Recruitment during the COVID-19 Pandemic: A Note with Preliminary Evidence from Immigrant-Origin and Native Voters in a German Metropolis in Spring 2021

Achim Goerres, University of Duisburg-Essen - Institute of Political Science
Jonas Elis, University of Duisburg-Essen
Sabrina Mayer, University of Duisburg-Essen - Institute of Political Science
Dennis Spies, Heinrich Heine University Duesseldorf

Near-Optimal Experimental Design for Networks: Independent Block Randomization

Ozan Candogan, University of Chicago - Booth School of Business
Chen Chen, University of Chicago - Booth School of Business
Rad Niazadeh, University of Chicago - Booth School of Business


RANDOMIZED SOCIAL EXPERIMENTS eJOURNAL

"The State Capacity Ceiling on Tax Rates: Evidence from Randomized Tax Abatements in the DRC" Fee Download
CEPR Discussion Paper No. DP16116

AUGUSTIN BERGERON, Harvard University
Email:
GABRIEL TOUREK, Massachusetts Institute of Technology (MIT)
Email:
JONATHAN WEIGEL, Harvard University
Email:

How can developing countries increase the tax revenue they collect? In collaboration with the Provincial Government of Kasaï-Central, we study a policy experiment in the D.R. Congo that randomly assigned 38,028 property owners to different property tax liabilities. We find that status quo tax rates are above the revenue-maximizing tax rate (RMTR). Reducing the property tax rate by approximately 34% would maximize government revenue, by increasing tax compliance. We then investigate how responses to tax rates interact with enforcement. We exploit two sources of variation in enforcement — randomized enforcement letters and random assignment of tax collectors — and show that the RMTR increases with enforcement. Replacing tax collectors in the bottom quartile of enforcement capacity by average collectors would raise the RMTR by 42%. Tax rates and enforcement are thus complementary levers. While a naive government that sequentially implements the RMTR and increases enforcement would raise revenue by 61%, a sophisticated government that prospectively implements the post-enforcement RMTR would instead raise revenue by 77%. These findings provide experimental evidence that low government enforcement capacity sets a binding ceiling on the revenue-maximizing tax rate in some developing countries, and thereby demonstrates the value of increasing tax rates in tandem with tax enforcement to expand fiscal capacity.

"Understanding Performance in Test Taking: The Role of Question Difficulty Order" Fee Download
CEPR Discussion Paper No. DP16099

LINA ANAYA, University of Arkansas - Department of Education Reform
Email:
NAGORE IRIBERRI, University of the Basque Country
Email:
PEDRO REY BIEL, University College London
GEMA ZAMARRO, University of Arkansas - Department of Education Reform, Center for Economic and Social Research (CESR)
Email:

Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty and we find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers. Consistent results are found exploiting the random variation of difficulty across test booklets in the Programme for International Student Assessment (PISA), a triannual international test, for the years of 2009, 2012, and 2015, providing additional external validity. We conclude that the order of the difficulty of the questions in tests should be considered carefully, in particular when comparing performance between test-takers who have faced different order of questions.

"Are High-Interest Loans Predatory? Theory and Evidence from Payday Lending" Fee Download
NBER Working Paper No. w28799

HUNT ALLCOTT, New York University (NYU)
Email:
JOSHUA J. KIM, Facebook Inc
Email:
DMITRY TAUBINSKY, University of California, Berkeley - Department of Economics
Email:
JONATHAN ZINMAN, Dartmouth College, Innovations for Poverty Action, Jameel Poverty Action Lab, National Bureau of Economic Research (NBER)
Email:

It is often argued that people might take on too much high-cost debt because they are present focused and/or overoptimistic about how soon they will repay. We measure borrowers' present focus and overoptimism using an experiment with a large payday lender. Although the most inexperienced quartile of borrowers underestimate their likelihood of future borrowing, the more experienced three quartiles predict correctly on average. This finding contrasts sharply with priors we elicited from 103 payday lending and behavioral economics experts, who believed that the average borrower would be highly overoptimistic about getting out of debt. Borrowers are willing to pay a significant premium for an experimental incentive to avoid future borrowing, which we show implies that they perceive themselves to be time inconsistent. We use borrowers' predicted behavior and valuation of the experimental incentive to estimate a model of present focus and naivete. We then use the model to study common payday lending regulations. In our model, banning payday loans reduces welfare relative to existing regulation, while limits on repeat borrowing might increase welfare by inducing faster repayment that is more consistent with long-run preferences.

Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

"Virtual Watercoolers: A Field Experiment on Virtual Synchronous Interactions and Performance of Organizational Newcomers" Free Download
Harvard Business School Technology & Operations Mgt. Unit Working Paper No. 21-125

IAVOR BOJINOV, Harvard University - Technology & Operations Management Unit
Email:
PRITHWIRAJ CHOUDHURY, Harvard University - Business School (HBS)
Email:
JACQUELINE N. LANE, Harvard Business School
Email:

Do virtual, yet informal and synchronous, interactions affect individual performance outcomes of organizational newcomers? We report results from a randomized field experiment conducted at a large global organization that estimates the performance effects of “virtual water coolers” for remote interns participating in the firm’s flagship summer internship program. Findings indicate that interns who had randomized opportunities to interact synchronously and informally with senior managers were significantly more likely to receive offers for full-time employment, achieved higher weekly performance ratings, and had more positive attitudes toward their remote internships. Further, we observed stronger results when the interns and senior managers were demographically similar. Secondary results also hint at a possible abductive explanation of the performance effects: virtual watercoolers between interns and senior managers may have facilitated knowledge and advice sharing. This study demonstrates that hosting brief virtual water cooler sessions with senior managers might have job and career benefits for organizational newcomers working in remote workplaces, an insight with immediate managerial relevance.

"Does Computer-Aided Instruction Improve Children’s Cognitive and Noncognitive Skills?" Free Download
Asian Development Review 38:1

HIROTAKE ITO, Keio University - Graduate School of Media and Governance
Email:
KEIKO KASAI, School of International Development, University of East Anglia
Email:
HIROMU NISHIUCHI, Yokohama City University - Graduate School of International Management
Email:
MAKIKO NAKAMURO, Keio University

This paper examines the causal effects of computer-aided instruction (CAI) on children’s cognitive and noncognitive skills. We ran a clustered randomized controlled trial at five elementary schools with more than 1,600 students near Phnom Penh, Cambodia. After 3 months of intervention, we find that the average treatment effects on cognitive skills are positive and statistically significant, while hours of study were unchanged both at home and in the classroom. This indicates that CAI is successful in improving students’ learning productivity per hour. Furthermore, we find that CAI raises students’ subjective expectation to attend college in the future.

"How to Increase Survey Response and Manage the Postal and Face-to-Face Field Work of Participant Recruitment during the COVID-19 Pandemic: A Note with Preliminary Evidence from Immigrant-Origin and Native Voters in a German Metropolis in Spring 2021" Free Download

ACHIM GOERRES, University of Duisburg-Essen - Institute of Political Science
Email:
JONAS ELIS, University of Duisburg-Essen
Email:
SABRINA MAYER, University of Duisburg-Essen - Institute of Political Science
Email:
DENNIS SPIES, Heinrich Heine University Duesseldorf
Email:

This note presents preliminary evidence from a postal and face-to-face recruitment field for a telephone survey during the COVID-19 pandemic. Drawn from the city register of Duisburg, a metropolis of 500,000 inhabitants, voters eligible for the September 2021 Bundestag elections are the target population. They are stratified into four groups by onomastic classification: native Germans, Germans of Turkish Descent, Germans with a (Post-) Soviet background and Germans of any other immigrant origin. This note presents ten measures of optimising the recruitment of individuals from these four groups under the constraints of an all-encompassing high-incidence pandemic with distance measures and curfews in place, with closed shops and schools and reduced other care facilities. In addition, the report puts forward statistical analyses of some measures applied in an experimental design and estimates the overall and per-person-recruited costs.

Overall, the pandemic forced us to make drastic changes to research design and field organisation and to invest much more time and money than planned. Especially overburdened postal deliveries and restrictive policies for public places made the endeavour most difficult. So far, the ongoing fieldwork has produced a response rate of 8.2 % (only individuals willing to participate in telephone interview) with 11.0 % as a likely expected outcome. One individual recruited to take part in the telephone panel survey costs an estimated 53-71 €.

Early-bird incentives produced a positive response (meaning the willingness to partake in the survey) of 5.5 % compared to 4.0 % among those without early-bird incentives. The most effective measure to boost overall positive response among initial non-responders was in-person canvassing with unconditional incentive (21.3 %) instead of postal reminders (9.9 %). Canvassing a smaller group of people in person proved to be as costly as reminding a larger group of people in writing: we estimated costs of 60-61 € per additionally recruited survey participants through either channel.

"Near-Optimal Experimental Design for Networks: Independent Block Randomization" Free Download
Chicago Booth Research Paper No. 21-17

OZAN CANDOGAN, University of Chicago - Booth School of Business
Email:
CHEN CHEN, University of Chicago - Booth School of Business
Email:
RAD NIAZADEH, University of Chicago - Booth School of Business
Email:

Motivated by the prevalence of experimentation in online platforms and social networks, we consider the problem of designing randomized experiments to assess the effectiveness of a new market intervention for a network of users. An experiment assigns each user to either the treatment or the control group. The outcome of each user depends on her assignment as well as the assignments of her neighbors. Given the experiment, the unbiased Horvitz-Thompson estimator is used to estimate the total market effect of the treatment. The decision maker chooses randomized assignments of users to treatment and control, in order to minimize the worst-case variance of this estimator. We focus on networks that can be partitioned into communities, where the users in the same community are densely connected, and users from different communities are only loosely connected. In such settings, it is almost without loss to assign all users in the same community to the same variant (treatment or control). The problem of designing the optimal randomized assignments of communities can be formulated as a linear program with exponential number of decision variables and constraints in the number of communities -- and hence is in general computationally intractable.

We develop a family of practical experiments that we refer to as \emph{independent block randomization (IBR)} experiments. Such an experiment partitions communities into blocks so that each block contains communities of similar sizes. It then treats half of the communities in each block (chosen uniformly at random) and does so independently across blocks. The optimal community partition can be obtained in a tractable way using dynamic programming. We show that these policies are asymptotically optimal when the number of communities grows large and no community size dominates the rest. In the special case where community sizes take values in a finite set and the number of communities of each size is a fixed proportion of the total number of communities, the loss is only a constant that is independent of the number of communities. Beyond the asymptotic regime, we show that the IBR experiment is a 7/3-approximation for any problem instance. We also examine the performance of the IBR experiments on data-driven numerical examples, including an example based on a Facebook subnetwork.

^top

About this eJournal

The randomized social experiment is a distinctive modern form of social research activity. This area includes content focused on papers, articles, and research reports concerned with field tests of human behavior in which random assignment has been used for evaluation purposes. Social experiments began in the United States in the late 1960s, and have since been conducted on all the inhabited continents. Random assignment involves neither choice nor discretion. Whereas human subjects may or may not have the right to choose to participate in the experiment, they do not have the right to decide which group within the experiment they will join. Similarly, persons administering the policy intervention may restrict eligibility for participation in the experiment, but once a person is admitted, program staff cannot determine the group in which that subject is enrolled, except by using randomization. If implemented properly, the results of social experiments generally are internally valid, that is, they provide unbiased impact estimates for targeted people subject to different treatments at the particular time and place they were administered. The question of their external validity to larger populations, other places, or time per