Informative or Actionable? Implementation Constraints And Business Experimentation

Abstract

In contrast to scientists, entrepreneurs do not experiment merely to understand phenomena—they do so to make better decisions. However, prior work has tended to assume that actors are unconstrained in their ability to respond to the information generated from experimentation. Building on a growing literature taking a pragmatic view of the entrepreneurial process, this paper argues that implementation constraints associated with particular entrepreneurial strategies significantly shape an actor's choices about when and how to experiment. The key insight of our formal model is that implementation constraints shift the value of experimentation, influencing whether entrepreneurs validate more or less promising ideas through experimentation. Implementation constraints can lead to surprising consequences, such as leading some actors to experiment when, absent implementation constraints, they would not do so. Other actors optimally eschew experimentation even if an inexpensive, informative experiment is available.  The model also illuminates how implementation constraints impact whether to run cheaper, less informative experiments or costlier, more informative experiments. Ultimately, introducing the possibility of implementation constraints suggests that some actors who, from the perspective of prior work, look like they are engaging in sub-optimal experimentation may not be overconfident or lack training. Rather, such actors could be selecting the optimal experimental strategy after considering implementation constraints.

Keywords: Experimentation, Strategy, Entrepreneurship, New Experimental Strategy, Implementation Constraints, Pragmatism, Scientific Approach, Adjustment Costs

JEL Classification: L20, L26, L29, D21, D23