Experimentation and Startup Performance: Evidence from A/B Testing

46 Pages Posted: 23 Aug 2019

See all articles by Rembrand Koning

Rembrand Koning

Harvard University - Business School (HBS)

Sharique Hasan

Duke University - Fuqua School of Business

Aaron Chatterji

Duke University - Fuqua School of Business

Multiple version iconThere are 2 versions of this paper

Date Written: August 20, 2019

Abstract

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

Keywords: Entrepreneurial Strategy, Startups, A/B Testing, Data-Driven Decision-Making, Experimentation

Suggested Citation

Koning, Rembrand and Hasan, Sharique and Chatterji, Aaron, Experimentation and Startup Performance: Evidence from A/B Testing (August 20, 2019). Available at SSRN: https://ssrn.com/abstract=3440291 or http://dx.doi.org/10.2139/ssrn.3440291

Rembrand Koning (Contact Author)

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Boston, MA 02163
United States

Sharique Hasan

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Aaron Chatterji

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Register to save articles to
your library

Register

Paper statistics

Downloads
128
Abstract Views
861
rank
221,262
PlumX Metrics