Experimentation and Startup Performance: Evidence from A/B Testing

47 Pages Posted: 18 Sep 2019

See all articles by Rembrand Koning

Rembrand Koning

Harvard University - Business School (HBS)

Aaron Chatterji

Duke University - Fuqua School of Business

Multiple version iconThere are 2 versions of this paper

Date Written: September 2019


Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing business ideas. This paper provides the first evidence of how digital experimentation affects the performance of a large sample of high-technology startups using data that tracks their growth, technology use, and product launches. We find that, despite its prominence in the business press, relatively few firms have adopted A/B testing. However, among those that do, we find increased performance on several critical dimensions, including page views and new product features. Furthermore, A/B testing is positively related to tail outcomes, with younger ventures failing faster and older firms being more likely to scale. Firms with experienced managers also derive more benefits from A/B testing. Our results inform the emerging literature on entrepreneurial strategy and how digitization and data-driven decision-making are shaping strategy.

Institutional subscribers to the NBER working paper series, and residents of developing countries may download this paper without additional charge at www.nber.org.

Suggested Citation

Koning, Rembrand and Chatterji, Aaron, Experimentation and Startup Performance: Evidence from A/B Testing (September 2019). NBER Working Paper No. w26278. Available at SSRN: https://ssrn.com/abstract=3454379

Rembrand Koning (Contact Author)

Harvard University - Business School (HBS) ( email )

Soldiers Field Road
Boston, MA 02163
United States

Aaron Chatterji

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Abstract Views
PlumX Metrics