On the Usefulness of Using Current LLMs for Experimental Auction Valuation

Abstract

Large language models (LLMs) are rapidly finding their way into all parts of human life. For academics the question arises how to practically and ethically make use of the technology. One aspect that researchers are starting to investigate is how LLMs can assist with data collection. We assess the usefulness of three LLMs, ChatGPT, Claude and Gemini, for experimental auction valuation. We compare the bids these LLMs generate with human bids from two different auction experiments conducted in the U.S. and Germany. When providing the LLMs with experiment instructions and participants' demographic data, results show that bids generated by the different LLMs are not comparable to human bids. However, when providing 10 or 20% of the original bids as a reference, the bids generated by the LLMs closely match human bids, though results differ depending on the LLM used. Overall, our empirical tests show that LLMs can be a promising complement to human subjects but not a replacement.

Keywords: Artificial intelligence, large language models, human subject research

JEL Classification: C91, D44, C63