Design of Randomized Experiments in Networks

Dylan Walker

Boston University

Lev Muchnik

Hebrew University of Jerusalem - Jerusalem School of Business Administration; New York University (NYU) - Department of Information, Operations, and Management Sciences

August 6, 2014

As our day-to-day activities become increasingly embedded in online and digitally enabled environments, the availability of massive scale yet highly granular data on individuals and social interaction enables new avenues of scientific discovery. The promise of big data seems immense – not just for its scale and scope, but perhaps more importantly because highly detailed individual-level data at scale suggests tailored policies that resist reversion to the mean in domains ranging from medicine and public health to politics, web search, business, e-commerce, and product design. Yet, the promise of big data has recently come under fire for its inability to separate correlation from causation – to derive actionable insights and yield effective policies. This criticism unveils the perhaps lesser known but burgeoning movement of big experiments that is rapidly gaining traction within both academic research and industry practice. The gold standard of causal inference through experimentation is well established in both the public and private sectors. Yet, the realization that our world is highly connected and that behavioral and economic outcomes at the individual and population level depend upon this connectivity challenges the principles of experimental design that lie at the very heart of the scientific process. Traditional experimental designs that randomly assign populations to control and treatment groups to measure the comparative outcome of a treatment do not account for the networked environment in which we live – the natural connections between subjects in these populations. When the impact of treatment can propagate along these connections, the traditional notions of experimental design break down. However, the natural connectivity of our world does not only present a challenge to the conventional paradigm of experimental design, but also reveals opportunities to leverage connectivity through the creation of novel treatments that incorporate both experimental subjects and the connections between them. In this work, we consider several aspects of networked randomized trial design from the perspective of the experimental setting, the process being studied, and the impact of connectivity. We further address emerging methods to analyze and draw statistical inferences from networked randomized trials. Finally, we present a series of novel networked treatment designs and discuss their potential policy implications.

Number of Pages in PDF File: 18

Keywords: randomized trial design, networks, networked experiments, online field experiments, digital experimentation

working papers series

Download This Paper

Date posted: August 7, 2014  

Suggested Citation

Walker, Dylan and Muchnik, Lev, Design of Randomized Experiments in Networks (August 6, 2014). Available at SSRN: http://ssrn.com/abstract=2477076

Contact Information

Dylan Walker (Contact Author)
Boston University ( email )
595 Commonwealth Avenue
Boston, MA 02215
United States
HOME PAGE: http://www.dylantwalker.com
Lev Muchnik
Hebrew University of Jerusalem - Jerusalem School of Business Administration ( email )
Mount Scopus
Jerusalem, 91905
New York University (NYU) - Department of Information, Operations, and Management Sciences ( email )
44 West Fourth Street
New York, NY 10012
United States
Feedback to SSRN

Paper statistics
Abstract Views: 210
Downloads: 72
Download Rank: 196,269

© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.  FAQ   Terms of Use   Privacy Policy   Copyright   Contact Us
This page was processed by apollo2 in 0.328 seconds