Is Data for Health–Funded Research Reliable and Useful? Self-Reporting Bias Red Herring in Birth Registration Completeness Evaluation

Abstract

Over the past two decades birth registration has improved immensely. Children across the world are now 40 percent more likely to be registered than they were in 2000; birth registration completeness has risen from 50% to 70% (now 95 million children under one year old registered, versus 40 million still excluded). That we even know this with a good degree of certainty is because in the 1990s UNICEF (and USAID) made the historic decision to end the invisibility of children, and of numbers. By the mid 1990s already conducted household surveys were leveraged by inclusion of a question on the registration status of children. Surveys now account for data collection from almost 6 in 10 countries, and they are especially the poorer (LIC and LMIC) countries. Together with other data sources the surveys now cover 97.5 percent of the world’s population (ex China). UNICEF publishes the data online and annually in its flagship publication The State of the World’s Children. The latest data, because of an inevitable time lag between collection and publication, tell the status by late 2017. These improvements are monumental. Arguably, there is at this junction more enhanced awareness, advocacy, investment, dedication, effective governance, and results. Without data, birth registration and legal identity may not have made the cut for the Sustainable Development Agenda.

Surprisingly, two scholarly articles originating in the vital statistics community have been published that, in sequence, in their titles ask whether birth registration data (“from UNICEF”) are useful, and reliable. In this paper we have visited the four corners of these two papers, and have found them wanting to an extent that has surprised (and worried) us. Worried because of the stature of the authors and the journals in which they published their articles. In previous years we have taken the length and breadth of numberwork done by UNICEF, the United Nations Statistics Division and the World Bank (ID4D), and must have been a pain in some hindquarters, but we call balls and strikes.

The authors of the two articles bring two tape measures to their mission, both administrative measures and benchmarks, and compare survey data for birth registration completeness with their two administrative benchmarks, which are tabulae rasae – unlike the survey rates. And we are transparent about the weights we use. The authors find that survey data exaggerate birth registration rates, and proffer that that is so because of self-reporting bias among the respondents of surveys. It all turns out to be a storm in a glass of water, a red herring, and perhaps also some jalousie de métier. The numberwork of the authors, we fear, gets added to other scalps we have already taken. What is serious about that is that we expect scholars in the vital statistics community to do a proper job, and this it is not. The cause of birth registration and legal identity can also do without shoddy scholarly work, thank you very much. As we write in this paper, India should have stopped the authors in their tracks before they took to publishing their work. India was the elephant in the room, and the authors ignored the giant at their own peril.

Keywords: civil registration, birth registration, vital statistics, completeness estimation, UNICEF, Data for Health, self-reporting bias