How Experience Moderates the Impact of Generative AI Ideas on the Research Process

Abstract

At the heart of scientific discovery are expert researchers who identify research ideas worthy of inquiry. While generative artificial intelligence (AI) technologies—large language models, in particular—have been found to outperform humans in some tasks, their impact on assisting with the generation of research ideas, one of the most fundamental tasks in science, remains underexplored. We investigate how the use of generative AI affects researcher perceptions of their research proposals and their attitudes toward integrating generative AI ideas into their research process. In a randomized online experiment with 310 scientists across research disciplines, we study how generative AI ideas affect researchers’ self-evaluation of their proposals, research agenda, and attitudes. We do not find any average effect on their assessment of the proposals’ novelty or feasibility. However, research experience is an important moderator: experience negatively moderates the effect of generative AI on perceived novelty and impact of the proposal, and on their own research agendas. Further analyses suggest that less experienced researchers tended to express acceptance of generative AI, arising primarily from views that new lines of thinking were triggered, but also from the validation of existing ideas. More experienced researchers tended to express aversion, primarily due to discounting outside ideas, as well asl hesitation towards technology, and a perceived challenge to one’s identity. Our findings contribute to the innovation literature by offering initial insights into generative AI’s role in the research idea generation process, and to the growing literature on generative AI’s role in complementing human tasks.

Keywords: generative artificial intelligence (AI), scientific discovery, research idea generation, technology acceptance, technology aversion, large language models (LLMs)

JEL Classification: D89, O39