Evaluating the Impact of Cybersecurity Capacity Building

Abstract

This research is undertaken by the University of Oxford Global Cyber Security Capacity Centre (GCSCC) and Royal Holloway University London (RHUL), in support of the Global Forum on Cyber Expertise (GFCE) Strategy and Assessments Working Group.

This paper presents key findings from interviews with cyber capacity building (CCB) experts, which consisted of questions regarding better understanding of the impact of CCB programmes and projects, and recommendations for data collection and analysis. It also draws from a panel discussion and workshop at the Global Conference on Cyber Capacity Building (GC3B)i in Accra/Ghana in November 2023. The aim of these sessions was to collect feedback on initial findings, identify challenges in impact evaluation, to enhance understanding of CCB impact, and build consensus around areas of resource investment for enhancing the collection and evaluation of data, in order to aid the development of metrics and evidence-based programming.

The interviews and conference activities identified key principles that need to be adhered to in order to maximise impact, and impact evaluation. The process identified challenges around availability, quality, and access to data, and confirmed a disaggregation of data type by virtue of access, sensitivity and relevance. Additionally, it identified that certain data types, which could be beneficial for the purpose of impact evaluation, are often not collected. To demonstrate this, the paper uses a logic model of a CCB intervention, proposing five levels of evaluation for the development of indicators and use of associated data sets.

The process also examined key drivers, identifying that CCB should, ideally, be seen to contribute to higher level socio-economic and human development goals. However, currently, CCB activity neither focuses on nor is measured systematically against such objectives.

In order to better understand and measure outcomes and impacts, this paper suggests emphasising benefits of integrating impact evaluation into CCB programming and building the evidence base for investment, and areas for further research.

The paper forms part of a research project “Cyber Capacity Building Impact Evaluation: Bringing Solutions to Life”1 which aims to elevate understanding, and increase integration, of impact evaluation within CCB programmes, and is a baseline for further research activity. Further validation work is undertaken at various CCB events with goal to present solutions at the second GC3B in Geneva in 2025.

Keywords: cybersecurity capacity building impact evaluation