Policy Pilots: Experimentation in the Administration of the Law
59 Pages Posted: 28 Dec 2018 Last revised: 31 Dec 2018
Date Written: December 12, 2018
Policy problem solvers and academics generally express their recommendations in terms of new or revised laws or policies they’d like to see. But just as products aren’t developed overnight, it’s unrealistic to expect policymakers to introduce new policy or policy changes without first testing them. This essay proposes that policy problem solvers frame their suggestions in terms of experiments to try, not just policies to enact. With a focus on federal agencies, this essay provides a framework for proposing experiments that specify: 1) questions that matter, 2) treatments that can be tested with existing authority and resources, 3) theory of change, 4) testing protocol, and 5) evidence, and, with the outputs from these steps, 6) reflecting and revising accordingly (“MATTER”). An appropriate testing protocol comprises power, randomization or other experimental strategy, proper inference, method, ethics, and resources (“PRIMER”). The United States Patent and Trademark Office (USPTO) is an ideal agency for experimentation due to its strong foundation of open data and commitment to innovation, learning, and improvement. To demonstrate the framework, I propose four experiments related to enhancing patent quality that the agency could try. The proposed experiments are: applicant-deferral of section 101 subject matter rejections, examiner cited non-patent literature as a quality metric, team or time examination on demand, and using artificial intelligence or other technologies to carry out error correction and improve applicant readiness. The decision to experiment must ultimately be made by agency leaders. These ideas are intended to provide starting points, rather than definitive recommendations, for agency action.
Keywords: patents, intellectual property, experimentation, randomization, RCT, causal inference
JEL Classification: K20, L51, O31, O34
Suggested Citation: Suggested Citation