Digital experimentation and startup performance: Evidence from A/B testing

75 Pages Posted: 23 Aug 2019 Last revised: 22 Sep 2020

See all articles by Rembrand Koning

Rembrand Koning

Harvard University - Business School (HBS)

Sharique Hasan

Duke University - Fuqua School of Business

Aaron Chatterji

Duke University - Fuqua School of Business

Multiple version iconThere are 2 versions of this paper

Date Written: August 20, 2019

Abstract

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence on how digital experimentation affects a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. Despite its prominence in the business press, relatively few firms have adopted A/B testing tools. However, among those that do, we find increased performance on several critical dimensions, including page views, code changes, and new product features. These results are robust to our use of instrumental variables and synthetic control models, where we leverage Google’s launch of a new A/B testing tool. Our results also show that the value of A/B testing increases over time and is positively related to tail outcomes, with adopt- ing firms both scaling and failing faster. Our results inform the emerging literature on entrepreneurial strategy and illustrate the growing importance of digitization and data-driven decision-making.

Keywords: Entrepreneurial Strategy, Startups, A/B Testing, Data-Driven Decision-Making, Experimentation

Suggested Citation

Koning, Rembrand and Hasan, Sharique and Chatterji, Aaron, Digital experimentation and startup performance: Evidence from A/B testing (August 20, 2019). Available at SSRN: https://ssrn.com/abstract=3440291 or http://dx.doi.org/10.2139/ssrn.3440291

Rembrand Koning (Contact Author)

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Boston, MA 02163
United States

Sharique Hasan

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Aaron Chatterji

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
228
Abstract Views
1,549
rank
146,479
PlumX Metrics