Optimizing Policymakers' Loss Functions in Crisis Prediction: Before, Within or After?

Abstract

Early-warning models most commonly optimize signaling thresholds on crisis probabilities. The expost threshold optimization is based upon a loss function accounting for preferences between forecast errors, but comes with two crucial drawbacks: unstable thresholds in recursive estimations and an in-sample overfit at the expense of out-of-sample performance.

We propose two alternatives for threshold setting:

(i) including preferences in the estimation itself and

(ii) setting thresholds ex-ante according to preferences only.

Given probabilistic model output, it is intuitive that a decision rule is independent of the data or model specification, as thresholds on probabilities represent a willingness to issue a false alarm vis-à-vis missing a crisis. We provide simulated and real-world evidence that this simplification results in stable thresholds and improves out-of-sample performance. Our solution is not restricted to binary-choice models, but directly transferable to the signaling approach and all probabilistic early-warning models.

Keywords: early-warning models, loss functions, threshold setting, predictive performance

JEL Classification: C35, C53, G01