Download This Paper
Preprints with The Lancet is a collaboration between The Lancet Group of journals and SSRN to facilitate the open sharing of preprints for early engagement, community comment, and collaboration. Preprints available here are not Lancet publications or necessarily under review with a Lancet journal. These preprints are early-stage research papers that have not been peer-reviewed. The usual SSRN checks and a Lancet-specific check for appropriateness and transparency have been applied. The findings should not be used for clinical or public health decision-making or presented without highlighting these facts. For more information, please see the FAQs.
The Agreement Between Virtual Patients and Unannounced Standardized Patients in Evaluating Primary Healthcare Quality: A Multicentred, Cross-Sectional Study in Seven Provinces of China
28 Pages Posted: 30 May 2022
More...Abstract
Background: The unannounced standardized patient (USP) is the gold standard for primary healthcare (PHC) quality assessment but has many restrictions associated with high human and resource costs. Virtual patient (VP) is a valid, low-cost software for simulating clinical scenarios, and is widely used in medical education. Whether it can be used to assess the quality of PHC remains unknown. This study aimed to examine the agreement of VP and USP in the PHC quality assessment, and to identify factors influencing the VP-USP agreement.
Methods: Eleven matched USP-VP cases were developed based on clinical guidelines and were implemented on a convenient sample of urban PHCs in the capital city of the seven-study provinces. 720 USP visits were conducted, whereby on-duty primary healthcare providers who met inclusion criteria were randomly selected by the USPs and further received a VP assessment of the same case condition at least a week later. The VP-USP agreement was measured by concordance correlation coefficient (CCC) for continuity scores and weighted kappa for diagnosis. Multiple linear regression was applied to identify factors influencing VP-USP agreement.
Findings: Only 146 VP scores were matched with original USP scores. The CCC of medical history was 0·37 (95%CI 0·24, 0·49), physical examination was 0·27 (95%CI 0·12, 0·42), laboratory and imaging test was -0·03 (95%CI -0·20, 0·14), and treatment was 0·22 (95%CI 0·07, 0·37). The weighted kappa of diagnosis was 0·32 (95%CI 0·13, 0·52). The multiple linear regression model indicated that VP tests were significantly influenced by the different case conditions and the cities studied.
Interpretation: The agreement was low between VP and USP in PHC quality assessment. It may reflect the know-do gap, while VP test results were also influenced by different case conditions, VP’s interactive design and usability. Further modifications for VP’s usability should be user-centred, paying attention to balance usability enhancement and hints avoidance.
Funding: The project has received support from China Medical Board 18-300, National Natural Science Foundation of China 71974211, Swiss Agency for Development and Cooperation 81067392, and National Key R&D Program of China 2021ZD0113401.
Declaration of Interest: We declare no competing interests.
Ethical Approval: Ethical approval was obtained from the Ethics Committee of Sun Yat-sen University (no. 2017-007)
Suggested Citation: Suggested Citation