Making GenAI Smarter: Evidence From A Portfolio Allocation Experiment

Abstract

Retrieval-augmented generation (RAG) has emerged as a promising way to improve task-specific performance in generative artificial intelligence (GenAI) applications such as large language models (LLMs). In this study, we evaluate the performance implications of providing various types of domain-specific information to LLMs in a simple portfolio allocation task. We compare the recommendations of seven state-of-the-art LLMs in various experimental conditions against a benchmark of professional financial advisors. Our main result is that the provision of domain-specific information does not unambiguously improve the quality of recommendations. In particular, we find that LLM recommendations underperform recommendations by human financial advisors in the baseline condition. However, providing firm-specific information improves historical performance in LLM portfolios and closes the gap to human advisors. Performance improvements are achieved through higher exposure to market risk and not through an increase in mean-variance efficiency within the risky portfolio share. Notably, risk-averse investors are recommended substantially riskier portfolios when firm-specific information is provided. Finally, we document that quantitative firm-specific information affects recommendations more than qualitative firm-specific information and that providing generic finance theory does not affect recommendations.

Keywords: Generative artificial intelligence, large language models, domain-specific information, retrieval-augmented generation, portfolio management, portfolio allocation

JEL Classification: G00, G11, G40