A Proposed S.C.O.R.E. Evaluation Framework for Large Language Models – Safety, Consensus & Context, Objectivity, Reproducibility and Explainability

Abstract

A comprehensive qualitative evaluation framework for large language models (LLM) in healthcare expands beyond just accuracy and traditional quantitative metrics is needed. We propose 5 key aspects for evaluation of LLMs: Safety, Consensus & Context, Objectivity, Reproducibility and Explainability (S.C.O.R.E.). We suggest that S.C.O.R.E. may form the basis for an evaluation framework for future LLM-based models that are safe, reliable, trustworthy, and ethical for healthcare and clinical applications.

Keywords: large language models, healthcare, evaluation, generative artificial intelligence