Download This Paper
Preprints with The Lancet is part of SSRN´s First Look, a place where journals identify content of interest prior to publication. Authors have opted in at submission to The Lancet family of journals to post their preprints on Preprints with The Lancet. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early stage research papers that have not been peer-reviewed. The findings should not be used for clinical or public health decision making and should not be presented to a lay audience without highlighting that they are preliminary and have not been peer-reviewed. For more information on this collaboration, see the comments published in The Lancet about the trial period, and our decision to make this a permanent offering, or visit The Lancet´s FAQ page, and for any feedback please contact preprints@lancet.com.
Development and Validation of an Early Warning Model for Hospitalized COVID-19 Patients: A MultiCenter Retrospective Cohort Study
29 Pages Posted: 10 Feb 2022
More...Abstract
Background: Timely identification of deteriorating COVID-19 patients is needed to guide changes in clinical management and admission to intensive care units (ICUs). There is significant concern that widely used early warning scores (EWSs) underestimate illness severity in COVID19 patients and therefore, we developed an early warning model specifically for COVID-19 patients.
Methods: We collected electronic medical record data to extract predictors and used these to fit a random forest model. To simulate the situation in which the model would have been developed after the first COVID-19 ‘wave’ in the Netherlands and implemented during the second wave, we performed a temporal validation by splitting all included patients into groups admitted before and after August 1, 2020. Furthermore, we propose a method for dynamic model updating to retain the model’s predictive performance over time. We evaluated model discrimination and calibration, performed a decision curve analysis, and quantified the importance of predictors using SHapley Additive exPlanations values.
Findings: We included 3,514 COVID-19 patient admissions from six Dutch hospitals between February 2020 until May 2021, and included a total of 18 predictors for model fitting. The model showed a higher discriminative performance in terms of partial area under the receiver operating characteristic curve (0∙82 [0∙80 to 0∙84]) compared to the National Early Warning Score (0∙72 [0∙69 to 0∙74]) and the Modified Early Warning Score (0∙67 [0∙65 to 0∙69]), a greater net benefit over a range of clinically relevant model thresholds, and good calibration (intercept=0∙03 [0∙09 to 0∙14], slope=0∙79 [0∙73 to 0∙86]).
Interpretation: This study shows the potential benefit of moving from early warning models for the general inpatient population to models for specific patient groups. The COVID-19 specific early warning model and is available online at https://github.com/jimmsmit/COVID19_EWS and we encourage others to further validate this model independently.
Funding Information: No external funding.
Declaration of Interests: The authors declare that they have no conflict of interest.
Ethics Approval Statement: Ethical approval was given by the local research ethics committee.
Keywords: Early warning, COVID-19, machine learning, medical prediction model, artificial intelligence
Suggested Citation: Suggested Citation