The Perils of Counterfactual Analysis with Integrated Processes

41 Pages Posted: 6 Jan 2017

See all articles by Carlos Carvalho

Carlos Carvalho

Pontifical Catholic University of Rio de Janeiro (PUC-Rio) - Department of Economics

Ricardo Masini

Getulio Vargas Foundation (FGV), Sao Paulo School of Economics

Marcelo C. Medeiros

Pontifical Catholic University of Rio de Janeiro (PUC-Rio) - Department of Economics

Date Written: December 27, 2016

Abstract

Recently, there has been a growing interest in developing econometric tools to conduct counterfactual analysis with aggregate data when a “treated” unit suffers an intervention, such as a policy change, and there is no obvious control group. Usually, the proposed methods are based on the construction of an artificial counterfactual from a pool of “untreated” peers, organized in a panel data structure. In this paper, we investigate the consequences of applying such methodologies when the data are formed by integrated process of order 1. We find that without a cointegration relation (spurious case) the intervention estimator diverges resulting in the rejection of the hypothesis of no intervention effect regardless of its existence. Whereas, for the case when at least one cointegration relation exists, we have a √ T-consistent estimator for the intervention effect albeit with a non-standard distribution. However, even in this case, the test of no intervention effect is extremely oversized if nonstationarity is ignored. When a drift is present in the data generating processes, the estimator for both cases (cointegrated and spurious) either diverges or is not well defined asymptotically. As a final recommendation we suggest to work in first-differences to avoid spurious results.

Keywords: Counterfactual Analysis, Comparative Studies, Panel Data, ArCo, Synthetic Control, Policy Evaluation, Intervention, Cointegration, Factor Models, Spurious Regression, Nonstationarity

JEL Classification: C22, C23, C32, C33

Suggested Citation

Carvalho, Carlos and Masini, Ricardo and Cunha Medeiros, Marcelo, The Perils of Counterfactual Analysis with Integrated Processes (December 27, 2016). Available at SSRN: https://ssrn.com/abstract=2894065 or http://dx.doi.org/10.2139/ssrn.2894065

Carlos Carvalho

Pontifical Catholic University of Rio de Janeiro (PUC-Rio) - Department of Economics ( email )

Rua Marques de Sao Vicente, 225/206F
Rio de Janeiro, RJ 22453
Brazil

HOME PAGE: http://https://sites.google.com/site/cvianac2/carloscarvalho

Ricardo Masini

Getulio Vargas Foundation (FGV), Sao Paulo School of Economics ( email )

Rua Itapeva 474 s.1202
São Paulo, São Paulo 01332-000
Brazil

Marcelo Cunha Medeiros (Contact Author)

Pontifical Catholic University of Rio de Janeiro (PUC-Rio) - Department of Economics ( email )

Rua Marques de Sao Vicente, 225/206F
Rio de Janeiro, RJ 22453
Brazil
+55 21 3114-1078 (Phone)

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
173
Abstract Views
1,015
rank
182,038
PlumX Metrics