Staff Experience after Investment in Digital Technologies: a Difference-In-Differences Study in the English NHS

Abstract

Background: In 2021/22, the NHS launched the Digital Productivity Programme, spending £12.217 million to speed up the use of three digital technologies, including artificial intelligence (AI), real-time location services (RTLS) and virtual reality (VR). The programme aims to ease staff shortages, reduce patient waiting times, improve quality of care, and lower the cost of care giving. >However, we do not yet understand how the use of these digital technologies might affect NHS staff. This research will study how digital technologies affect NHS staff satisfaction.

Methods: A two-way fixed effects difference-in-differences regression analysis was conducted using organisation-level data from the NHS Staff Survey and Digital Productivity Programme. By comparing staff satisfaction before (2018-21) and after new digital technologies (2022-23) were introduced into the workplace, we investigated whether staff satisfaction changed after the Digital Productivity Programme. 103 Acute and Acute & Community Trusts were included in our analyses. Outcomes were staff experience scores measured by the NHS Staff Survey on morale and engagement, encompassing sub-scores for thinking about leaving, work pressure, stressors, motivation, involvement, and advocacy. Higher scores are always favourable. Event-study analyses were additionally conducted to check pre-treatment parallel-trends assumptions and post-treatment effects variation over time.

Findings: Investment in digital technologies was weakly and negatively associated with "Thinking about leaving" the NHS (-0.06, 95% CI: [-0.14, 0.01]), significant at the 10% level. No significant changes were observed in other job satisfaction outcomes. The impact of technology on staff experience varied by technology type: AI had no impact on all staff experience outcomes; RTLS was associated with significant declines in Thinking about leaving (-0.13, 95% CI: [-0.25, 0.00], Stressors (-0.06, 95% CI: [-0.11, -0.00]), Engagement (-0.10, 95% CI: [-0.20, -0.01]), Motivation (-0.07, 95% CI: [-0.12, -0.01]) and Involvement (-0.09, 95% CI: [-0.15, -0.03]); and VR was excluded from the analysis because it violated the parallel-trends assumption, which is essential for this study design.

Interpretation: Digital technology investments under the Digital Productivity Programme were associated with some negative effects on some NHS staff job satisfaction outcomes. The type of digital technology significantly influenced staff experience, highlighting the need for preparing the NHS staff to accept and use such technologies. In short, staff’s confidence in digital technologies must be restored during digital transformation.

Keywords: Digital technology, job satisfaction, artificial intelligence, health policy