Mapping (A)Ideology: A Taxonomy of European Parties Using Generative LLMs as Zero-Shot Learners

Abstract

We perform the first mapping of the ideological positions of European parties using generative Artificial Intelligence (AI) as a "zero-shot" learner. We ask OpenAI 's Generative Pre-trained Transformer (GPT-3.5) to identify the more "right-wing" option across all possible duplets of European parties at a given point in time, solely based on their names and country of origin, and combine this information via a Bradley-Terry decomposition to create an ideological ranking. Cross-validating our LLM-generated assessment with widely-used expert-, manifesto-and pollbased scaling methods reveals that Large Language Models (LLMs) closely map those obtained through the first, i.e., CHES. Also, left-right scores produced by GPT-3.5 accurately capture changes in parties' ideological positions over time. Given the high cost of scaling parties via trained coders, and the scarcity of expert data before the 1990s, finding that generative AI produces estimates of comparable quality to CHES supports its usage in political science on the grounds of replicability, agility, and affordability.

Keywords: Ideology Scores, Computational Methods, Expert Opinion, Matched Pairs, Text-As-Data