Strategies of Validation: Assessing the Varieties of Democracy Corruption Data

45 Pages Posted: 4 Feb 2016 Last revised: 24 May 2016

See all articles by Kelly M. McMann

Kelly M. McMann

Case Western Reserve University - Department of Political Science; Göteborg University - V-Dem Institute

Daniel Pemstein

North Dakota State University; Göteborg University - V-Dem Institute

Brigitte Seim

University of North Carolina (UNC) at Chapel Hill

Jan Teorell

Lund University - Department of Political Science

Staffan I. Lindberg

Göteborg University - Varieties of Democracy Institute; Göteborg University - Department of Political Science

Date Written: February 1, 2016

Abstract

Social scientists face the challenge of determining whether their data are valid, yet they lack practical guidance about how to do so. Existing publications on data validation provide mostly abstract information for creating one’s own dataset or establishing that an existing one is adequate. Further, they tend to pit validation techniques against each other, rather than explain how to combine multiple approaches. By contrast, this paper provides a practical guide to data validation in which tools are used in a complementary fashion to identify the strengths and weaknesses of a dataset and thus reveal how it can most effectively be used. We advocate for three approaches, each incorporating multiple tools: 1) assessing content validity through an examination of the resonance, domain, differentiation, fecundity, and consistency of the measure; 2) evaluating data generation validity through an investigation of dataset management structure, data sources, coding procedures, aggregation methods, and geographic and temporal coverage; and 3) assessing convergent validity using case studies and empirical comparisons among coders and among measures. We apply our method to corruption measures from a new dataset, Varieties of Democracy. We show that the data are generally valid and we emphasize that a particular strength of the dataset is its capacity for analysis across countries and over time. These corruption measures represent a significant contribution to the field because, although research questions have focused on geographic differences and temporal trends, other corruption datasets have not been designed for this type of analysis.

Suggested Citation

McMann, Kelly M. and Pemstein, Daniel and Seim, Brigitte and Teorell, Jan and Lindberg, Staffan I., Strategies of Validation: Assessing the Varieties of Democracy Corruption Data (February 1, 2016). V-Dem Working Paper 2016:23. Available at SSRN: https://ssrn.com/abstract=2727595 or http://dx.doi.org/10.2139/ssrn.2727595

Kelly M. McMann (Contact Author)

Case Western Reserve University - Department of Political Science ( email )

11201 Euclid Avenue
Cleveland, OH 44106
United States

Göteborg University - V-Dem Institute ( email )

United States

Daniel Pemstein

North Dakota State University ( email )

Fargo, ND 58105
United States

Göteborg University - V-Dem Institute ( email )

United States

Brigitte Seim

University of North Carolina (UNC) at Chapel Hill ( email )

102 Ridge Road
Chapel Hill, NC 27514
United States

Jan Teorell

Lund University - Department of Political Science ( email )

Department of Political Science
Box 52
Lund, SE 40530
Sweden

Staffan I. Lindberg

Göteborg University - Varieties of Democracy Institute ( email )

Sprängkullsgatan 19
Gothenburg, Gothenburg 405 30
Sweden

HOME PAGE: http://www.pol.gu.se/varianter-pa-demokrati--v-dem-/

Göteborg University - Department of Political Science ( email )

Box 711
Gothenburg, S-405 30
Sweden

HOME PAGE: http://www.pol.gu.se

Register to save articles to
your library

Register

Paper statistics

Downloads
102
Abstract Views
591
rank
261,852
PlumX Metrics