Download This Paper
Preprints with The Lancet is a collaboration between The Lancet Group of journals and SSRN to facilitate the open sharing of preprints for early engagement, community comment, and collaboration. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early-stage research papers that have not been peer-reviewed. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. The findings should not be used for clinical or public health decision-making or presented without highlighting these facts. For more information, please see the FAQs.
A Validated and Explainable Deep Learning Model Instantly Predicts Survival from Consultation Reports
22 Pages Posted: 7 Apr 2023
More...Abstract
Background: Prognosis estimation is an important component to personalize the care strategy in oncology. Current predictive models often use predefined, curated, and limited clinical and biological variables, often with limited completeness, comprehensiveness and sometimes, accuracy.
Methods: We designed a solution that automatically and adaptively predicts the prognosis of patients with any kind of cancer (type and stage) based on textual electronic consultation reports, i.e. the basic working material for oncologists. To this end, we adapted and fine-tuned the French Transformer CamemBERT (a deep learning model adapted to free-text) to estimate overall survival with a confidence level. We benchmarked three different approaches and selected the optimal model for validation in a large independent dataset. We compared the results with the routinely used Performance Status (PS) as a control.
Findings: Altogether, we used 2.3M medical documents (corresponding to 36,123 patients for whom we had the date of death) to train, validate and test our approaches. The best survival prediction performances were obtained with the K-memBERT-T2 design that can take into account the medical history. K-memBERT-T2 reached a Pearson correlation of 0.655 between predicted and true continuous survival on the test cohort, and 0.621 on a large external cohort of 143k documents (17,633 additional patients) (p-values:<10-5). The 3-month binary survival predictions achieved an AUC of 0.852 on the test cohort and 0.875 on the external dataset. The model related to survival duration better than the PS, independently of its mention in texts. K-memBERT-T2 provides a possible human control on interpretation, extracting words that governed the prediction.
Interpretation: K-memBERT-T2 is a non-invasive, time and resource sparing method achieving accurate prediction performances for overall survival on large independent datasets, paving the way for an easy implementation in French-speaking centers. We have released the code at https://github.com/DITEP/KmemBERT and the web interface for clickable usage of K-memBERT.
Funding: Funding by GR and CLB for human resources.
Declaration of Interest: LV reports personal fees from Adaptherapy, is CEO of RESOLVED, has received non-personal fees from Pierre-Fabre and Servier, and a grant from Bristol-Myers Squibb, all outside the submitted work. As part of the Drug Development Department (DITEP) of Gustave Roussy and of the Phase 1 unit of Centre Léon Bérard, as medical doctor, LV report being: Principal/sub-Investigator of Clinical Trials for Abbvie, Adaptimmune, Aduro Biotech, Agios Pharmaceuticals, Amgen, Argen-X Bvba, Arno Therapeutics, Astex Pharmaceuticals, Astra Zeneca Ab, Aveo, Basilea Pharmaceutica International Ltd, Bayer Healthcare Ag, Bbb Technologies Bv, Beigene, Blueprint Medicines, Boehringer Ingelheim, Boston Pharmaceuticals, Bristol Myers Squibb, Ca, Celgene Corporation, Chugai Pharmaceutical Co, Clovis Oncology, Cullinan- Apollo, Daiichi Sankyo, Debiopharm, Eisai, Eisai Limited, Eli Lilly, Exelixis, Faron Pharmaceuticals Ltd, Forma Tharapeutics, Gamamabs, Genentech, Glaxosmithkline, H3 Biomedicine, Hoffmann La Roche Ag, Imcheck Therapeutics, Innate Pharma, Institut De Recherche Pierre Fabre, Iris Servier, Janssen Cilag, Janssen Research Foundation, Kura Oncology, Kyowa Kirin Pharm. Dev, Lilly France, Loxo Oncology, Lytix Biopharma As, Medimmune, Menarini Ricerche, Merck Sharp & Dohme Chibret, Merrimack Pharmaceuticals, Merus, Millennium Pharmaceuticals, Molecular Partners Ag, Nanobiotix, Nektar Therapeutics, Novartis Pharma, Octimet Oncology Nv, Oncoethix, Oncopeptides, Orion Pharma, Ose Pharma, Pfizer, Pharma Mar, Pierre Fabre, Medicament, Roche, Sanofi Aventis, Seattle Genetics, Sotio A.S, Syros Pharmaceuticals, Taiho Pharma, Tesaro, Xencor. Research Grants from Astrazeneca, BMS, Boehringer Ingelheim, Janssen Cilag, Merck, Novartis, Onxeo, Pfizer, Roche, Sanofi. Non-financial support (drug supplied) from Astrazeneca, Bayer, BMS, Boringher Ingelheim, Medimmune, Merck, NH TherAGuiX, Onxeo, Pfizer. ED reports grants and personal fees from Roche Genentech, grants from Boehringer, grants from Astrazeneca, grants and personal fees from Merck Serono, grants from BMS, and grants from MSD Roche. The other authors declare no potential conflicts of interest.
Ethical Approval: The study complies with the European GDPR regulation 2016/679, the French law, Good Clinical Practice Guidelines of the International Conference on Harmonization and was approved by internal ethics and scientific commissions (notification 2021-66). We registered the project by the MR004 declaration (V3.2 23/08/2021) to the French Health Data Hub and Unicancer at both sites, GR and CLB. We confirmed that all patients gave their consent or were not opposed to the use of their data. The data collected could only be used for the aim of this study, and patient identification was secured.
Keywords: Deep learning, electronic health records, free-text, natural language processing, transformers, prognosis, neoplasms
Suggested Citation: Suggested Citation