Selection Bias and COVID-19 Vaccine Effectiveness Against Death: Evidence from Linked Mortality and Vaccination Records
17 Pages Posted: 2 Nov 2022
Date Written: October 17, 2022
Background. While vaccines against COVID-19 have saved millions of lives, it is important to understand the remaining risk to the vaccinated and the incremental benefit of additional vaccine doses. Comparisons between more and less vaccinated groups can be misleading due to selection bias, because these groups can differ in underlying health and thus COVID-19 risk. We study by how much COVID-19 increased mortality from natural causes, controlling for underlying health.
Methods. We conduct a retrospective analysis of all deaths in Milwaukee County, Wisconsin, linked to vaccination records, and compare the percentage increase in deaths from natural causes due to COVID-19 between unvaccinated persons and those receiving 1, 2, or 3 vaccine doses, using an outcome measure that controls for non-COVID mortality and thus for population health, over April 1, 2021-March 31, 2022. We report how vaccination affects Relative Mortality Risk (RMR, defined as COVID-19 death as a fraction of other natural deaths for vaccinated persons, relative to this fraction for the unvaccinated) by age group and time period.
Findings. RMR was higher (vaccine effectiveness was lower) than in studies that did not address selection. RMR for two-dose vaccine recipients was 15.5% during April-June 2021, 19.0% during July–September, 2021, 22.9% during October-December, 2021 and 36.0% during January-March, 2022, corresponding to Alpha, early Delta, later Delta, and Omicron-dominant periods. A booster dose reduced RMR to 8-9%. RMR was higher for ages 60+. Selection effects were large; unvaccinated persons had over twice the risk of non-COVID natural death than the vaccinated.
Interpretation. Studies of vaccine effectiveness against mortality that do not control for underlying health will overstate effectiveness. Using a measure that controls for population health, fully vaccinated older individuals have substantial RMR, but boosters provide important protection.
Funding. Research support came from NIH, National Center for Advancing Translational Sciences, Award UL1TR001436.
Competing Interest Statement: The authors have no competing interests. The project was approved by the Medical College of Wisconsin Human Research Review Board.
The appendix of this paper is available at http://ssrn.com/abstract=4250492
Funding Information: Research support came from NIH, National Center for Advancing Translational Sciences, Award UL1TR001436.
Conflict of Interests: The authors have no competing interests.
Ethical Approval: The project was approved by the Medical College of Wisconsin Human Research Review Board.
Keywords: COVID-19; COVID-19 mortality; cause of death; COVID Excess Mortality Percentage; vaccine effectiveness; vaccine efficacy; selection bias
Suggested Citation: Suggested Citation