How to Keep it Adequate: A Validation Protocol for Agent-Based Simulation

Abstract

Agent-based models are used in a huge diversity of contexts, which complicates the establishment of a shared understanding of model validity and adequate methods for model construction, inference and validation. Starting from the tenet that model validity can only be judged with respect to a well-defined purpose and context, we conceptualise validation as systematically substantiating the premises on which conclusions from simulation analysis for a specific context are built. We revisit the premises of empirical and structural validation and argue that validation should not be understood as an isolated step in the modelling process. Rather, sound conclusions from simulation analysis require context-adequate choices at all steps of simulation analysis. To facilitate communication, we develop a protocol of guiding questions to analyse the modelling context, choose appropriate methods at each step, document the premises involved in a specific simulation analysis, and demonstrate the adequacy of the model for its context.

Keywords: Model validity, model inference, Calibration, generalisation, regime shift, identifiability