Design of Randomized Experiments in Networks
Hebrew University of Jerusalem - Jerusalem School of Business Administration; New York University (NYU) - Department of Information, Operations, and Management Sciences
August 6, 2014
As our day-to-day activities become increasingly embedded in online and digitally enabled environments, the availability of massive scale yet highly granular data on individuals and social interaction enables new avenues of scientific discovery. The promise of big data seems immense – not just for its scale and scope, but perhaps more importantly because highly detailed individual-level data at scale suggests tailored policies that resist reversion to the mean in domains ranging from medicine and public health to politics, web search, business, e-commerce, and product design. Yet, the promise of big data has recently come under fire for its inability to separate correlation from causation – to derive actionable insights and yield effective policies. This criticism unveils the perhaps lesser known but burgeoning movement of big experiments that is rapidly gaining traction within both academic research and industry practice. The gold standard of causal inference through experimentation is well established in both the public and private sectors. Yet, the realization that our world is highly connected and that behavioral and economic outcomes at the individual and population level depend upon this connectivity challenges the principles of experimental design that lie at the very heart of the scientific process. Traditional experimental designs that randomly assign populations to control and treatment groups to measure the comparative outcome of a treatment do not account for the networked environment in which we live – the natural connections between subjects in these populations. When the impact of treatment can propagate along these connections, the traditional notions of experimental design break down. However, the natural connectivity of our world does not only present a challenge to the conventional paradigm of experimental design, but also reveals opportunities to leverage connectivity through the creation of novel treatments that incorporate both experimental subjects and the connections between them. In this work, we consider several aspects of networked randomized trial design from the perspective of the experimental setting, the process being studied, and the impact of connectivity. We further address emerging methods to analyze and draw statistical inferences from networked randomized trials. Finally, we present a series of novel networked treatment designs and discuss their potential policy implications.
Number of Pages in PDF File: 18
Keywords: randomized trial design, networks, networked experiments, online field experiments, digital experimentationworking papers series
Date posted: August 7, 2014
© 2014 Social Science Electronic Publishing, Inc. All Rights Reserved.
This page was processed by apollo1 in 0.657 seconds