The Broken Promise of Obra '87: The Failure to Validate the Survey Protocol

Abstract

The Nursing Home Reform Act, OBRA '87, mandated that the protocol used to assess nursing facility compliance with care standards be “developed, tested, and validated by not later than January 1, 1990,” the date on which the federal government was required to implement the new assessment protocols and procedures. The requirement wrote into law one of the three key recommendations of the landmark study, “Improving the Quality of Care In Nursing Homes,” published by the National Academy of Sciences Institute of Medicine (IOM) in 1986.

This article, beginning in the 1970s, reviews decades of studies of the validity and accuracy of nursing home regulation. Using primarily studies and reports prepared by, or at the behest of, the federal government, this article demonstrates that every study of the federal government's nursing home assessment system has concluded that nursing home quality assessment is inconsistent, inaccurate, and unreliable. Yet, government regulators, industry officials, resident advocates, courts, and public policy researchers persist in relying on survey data — and encourage others to do so as well — to pursue enforcement actions, to draw conclusions about the relative and absolute quality of nursing facilities, to make conclusions about the state of nursing facility care generally, and to recommend public health policy initiatives regarding nursing facility care delivery.

A series of jurisdictional, procedural, economic, and practical barriers insulate the survey protocol from meaningful challenge and judicial oversight; OBRA ’87’s commands to test and validate the survey protocol and to measure and reduce inconsistency are thereby rendered largely meaningless. The inability to challenge the validity and accuracy of the survey protocol also means that government regulators are largely unaccountable for failing to correct survey systems flaws.

There is little incentive for regulators to assure that surveys produce valid, reliable, consistent, and accurate findings of both compliance and noncompliance. To the contrary, by continually focusing on under-citation, regulators advance the politically and professionally desirable goals of enhancing their image and that of the enforcement agencies as tough on noncompliance and committed to ferreting out problems. Thus, declines in deficiency rates and severity are dismissed — based on speculation and without empirical evidence — as attributable to unknown factors and not indicative of care improvements.

Relying on deeply flawed data — and encouraging others to do so — without acknowledging the flaws is a betrayal of the public trust. Most importantly, it deceives those whose health, safety, and welfare depends upon the credibility and effectiveness of the nursing facility survey process. It also sends a message to the regulated, and to the first level regulators, that the survey process is “effective,” whether or not it is accurate, because it serves the political interests of regulators and others who rely on the data despite knowledge that the survey is not an accurate or consistent performance measurement tool. Finally, after almost thirty years, it has institutionalized a survey system that is arbitrary, inconsistent, ineffective, and does not promote quality.

Keywords: nursing home, nursing facility, regulation, survey, enforcement, accuracy, consistency