AI Epistemic Risks: Emerging Mechanisms & Evidence

Abstract

Advances in artificial intelligence pose risks to humanity's collective capacity to form accurate beliefs, reason well, and maintain a healthy information environment—risks we term epistemic risks. These risks arise from AI's integration into the infrastructure through which individuals and societies think, form beliefs, and make sense of the world together. 

We highlight three primary mechanisms by which AI can cause systemic epistemic decline: persuasion and manipulation, cognitive offloading, and feedback loops. First, AI can persuade and manipulate. It can be misused for economic or political manipulation, inciting crime or radicalization, and escalating conflict. It can also create unintentional harms, e.g. sycophancy and mental health risks. Second, AI's unique features enable cognitive offloading in a qualitatively different way from previous technologies, risking cognitive decline. Third, human-AI and AI-AI feedback loops can narrow the epistemic space which humans and AI systems draw from. This already drives homogenization and may potentially lead to fragmentation.

We record the emerging empirical evidence for each mechanism and analyze how they likely amplify one another, creating compounding systemic risk. This evidence suggests that AI could be an unprecedented lever for improving epistemics, but this will not happen by default. Finally, we outline cross-cutting solutions which span AI system design, human-AI interaction, institutional reform, and wider incentive structures. Our paper ends with urgency: humanity's ability to navigate AI and non-AI risks alike may depend on how epistemically resilient our institutions are. It may be critical to act before that capacity is eroded.

Keywords: AI, LLM, epistemic, risks, cognitive, neuroscience, reasoning, decision-making, learning, education, feedback loops, echo chambers, human-AI, AI-AI, interaction, HCI, homogenization, algorithmic monoculture, fragmentation, policy, AI safety, AI ethics, metascience, alignment, value alignment, pluralistic alignment, socioaffective, RLHF, reinforcement learning, psychology, persuasion, manipulation, political, influence, misuse, misalignment, incentives, institutions, knowledge commons, data commons, information environment