Agent-Based Modeling as an Evaluation Tool to Understand the Mechanisms of a Financial Incentives Scheme for Maternal and Child Health in Tanzania

Abstract

Agent-based models (ABMs) offer a robust mechanism for modeling dynamic health systems and their responses to reforms, capturing vital feedback loops between agents, and incorporating agent heterogeneities. We constructed an ABM to investigate the effects of a supply-side payment-for-performance (P4P) scheme for childbirth care in Tanzania, specifically focusing on its impact on demand-side behaviors. Three classes of agents were included in the model: women of reproductive age, healthcare providers (facilities), and a district manager. For women, we incorporated a key decision-behavior with respect to the location of the birth: opting for the nearest facility or home. On the providers' end, responses to bonus incentives were modeled, considering aspects such as staff kindness and the levying of out-of-pocket informal charges. The model demonstrated that supply-side improvements could occur due to (i) changes in provider behavior driven by financial incentives, (ii) alterations in facility characteristics resulting from received incentive payments, and (iii) district manager facilitation of resource and strategy sharing. In particular, the model captured the potential limits of improvement on the supply side as demand increases, representing the added demand pressure on the system. The agent's decision about delivery site is influenced by (i) her previous experience with home and facility delivery, (ii) experiences shared by peers, and (iii) advice from traditional birth attendants. Agent characteristics were derived from impact evaluation data, a multilevel mixed-effect logistic backward stepwise regression analysis, and unmeasured influences captured through literature and stakeholder input, all contributing to the model's authenticity. The model, developed in AnyLogic, estimates that the current implementation of P4P, including bonus payment delays, led to a 21.5% increase (+15.4 percentage points) in facility-based deliveries compared to a counterfactual without P4P. Furthermore, avoiding payment delays observed during implementation could result in a further increase of 4.7% (+4.1 percentage points) in facility-based deliveries.  The model explored variations in facility responses to P4P, finding that initial facility performance indicators, along with the size of the population of the catchment and the capacity ratios of the facility, are key factors that enabled facilities with lower initial performance and smaller catchment areas to perform better. Programmatic steps to avoid payment delays (and the associated increases in 'out-of-pocket' informal charges during delays) should be prioritized.  Through the model, we have demonstrated how program evaluation data can inform the development of an ABM, which can elucidate the pathways to impact and program bottlenecks by virtually reconstructing agents and observing emergent system-level behaviors. Our framework has generalizable methodological steps for others seeking to use ABM to better understand how health system strengthening programs such as P4P affect the behavior of providers and patients.

Keywords: Agent-Based Modeling, Maternal and Child Health, Pay-For-Performance