Why Economists Should Conduct Field Experiments and 14 Tips for Pulling One Off

Abstract

The experimental approach has been a cornerstone of the scientific method for centuries. In one classic 1882 example, Louis Pasteur designated half of a group of 50 sheep as controls and vaccinated the other half. All animals then received a lethal dose of anthrax. Two days after inoculation, every one of the 25 control sheep were dead whereas the 25 vaccinated sheep were alive and well!

However, many economists have long been pessimistic that an experimental approach could offer such vivid illustrations of cause and effect in their fifi eld. For example, Samuelson and Nordhaus (1985)wrote in their introductory economics textbook a quarter-century ago:

"The economic world is extremely complicated. There are millions of people and firms, thousands of prices and industries. One possible way of figuring out economic laws in such a setting is by controlled experiments. A controlled experiment takes place when everything else but the item under investigation is held constant. Thus a scientist trying to determine whether saccharine causes cancer in rats will hold “other things equal” and only vary the amount of saccharine. Same air, same light, same type of rat."

Economists have no such luxury when testing economic laws. They cannot perform the controlled experiments of chemists or biologists because they cannot easily control other important factors. Like astronomers or meteorologists, they generally must be content largely to observe. In my own travels, I have often found similar skepticism. However, such skepticism has become more muted in recent decades as experimental methods have found a foothold in economics. In fact, complications that are diffificult to understand or control represent a key reason to conduct a field experiment rather than a reason for skepticism. This is because field experiments use randomization as an instrumental variable, balancing the unobserved variables across the treated and control states.

In this introduction to the symposium, I fifi rst offer an overview of the spectrum of experimental methods in economics, from laboratory experiments to the field experiments that are the subject of this symposium. I then offer some thoughts about the potential gains from doing economic research using field experiments and my own mental checklist of 14 steps to improve the chances of carrying out an economics field experiment successfully.

Keywords: field experiments

JEL Classification: JEL, D9