Hitting a Moving Target: Challenges and Lessons Around Evaluating System Change in Mission-Oriented Innovation

Abstract

In recognition of the complexity of systemic problems like climate change, biodiversity loss, and social inequality, there has been a push towards more challenge-directed “transformative innovation policy” (TIP). This push places new demands on the design and functions of monitoring, evaluation, and learning (MEL) processes for navigating system change. One trend within TIP is the emergence of missions, which act as vehicles for defining and coordinating action to address particular systemic challenges. At their outset, missions often have no clear criteria for either process or success. Their evaluation and governance must therefore accommodate complexity and uncertainty, as well as the inherent distribution of accountability across many actors. This paper discusses how the design and functions of transformative MEL can be expanded to more effectively navigate system transformation. From the literature and emerging practice, we summarise a set of design considerations for a more system-oriented, responsive, and formative mode of MEL. We also highlight a set of functions this type of MEL might perform, from accountability tracking to system-level learning and boundary processes like engagement and trust building. As an illustrative case study, we detail our function-led approach to developing transformative MEL for the “Towards Net Zero” (TNZ) Mission at the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia. Our case study highlights that beyond simply building a set of MEL tasks and tools, designing with these functions in mind can help wire up an integrated learning system for tackling complex challenges.

Keywords: Transformative Innovation Policy, Mission-Oriented Innovation Systems, Monitoring, Evaluation and Learning, System Complexity, Sustainability Transitions