Measuring and Assessing Public Health Emergency Preparedness: A Methodological Primer

Abstract

The COVID-19 pandemic has drawn new attention to the importance of public health emergency preparedness, and to the importance of developing tools for measuring preparedness. While there have been considerable advances in PHEP measurement during the past two decades, it is fair to say the field still remains in its infancy and continues to face a number of important challenges, including the fact that public health emergencies are singular, that their complexity makes it difficult to identify counterfactuals, and that their multi-jurisdictional, multi-sectoral nature means that measurement must often touch upon the activities of a wide and diffuse set of institutions and actors.

With this as background, the paper provides a primer on the current state of the art in PHEP measurement. We argue that the criteria for measure development forwarded by the National Quality Forum for clinical quality of care – validity, reliability, feasibility, utility – are appropriate for PHEP measurement, but that applying them to PHEP requires drawing upon a wider range of specific approaches and techniques than is typically the case in other areas of health. As such, the paper draws upon literatures in industrial engineering, case study methodology and others, in addition to statistics, epidemiology and other more common approaches in public health and health services research.

The paper argues that developers of PHEP measures should begin with the fundamental question of “why measure,” and discussed accountability, quality improvement, and resource allocation/mobilization as the most common uses. Answers to the “why measure” question, in turn, help frame answers to the remaining questions: what to measure, how to measure (i.e., collect data to populate measures), and assessing how well measures work (i.e., their validity, reliability, feasibility, and utility).

We contend that there are no single “gold standard” approaches for answering these questions. Thus, the paper seeks to describe a range of approaches and identify some of circumstances in which each might be more feasible or appropriate. We may have omitted some approaches. Moreover, with the increased attention to public health preparedness in the wake of the Covid pandemic, other approaches likely will be developed in the coming years. We hope this paper can provide a foundation for those working in the field, and that it can be updated in the future to reflect new developments.