Implementing Financial Regulations Using Large Language Models

Abstract

To strengthen financial stability, regulators worldwide have expanded re- quirements, impacting the profitability of financial institutions due to stricter capital requirements and increasing the costs for financial institutions’ regu- latory implementation projects and ongoing monitoring. These projects are complex, requiring expert teams, new software for reporting and compliance monitoring, and supporting databases. Recent advancements in generative artificial intelligence (GenAI), particularly large language models (LLMs), are expected to reduce the burden on institutions in these complex implemen- tations, particularly in software development. This paper introduces a novel dataset along with over 6,000 test cases derived from the Basel III standard approach for credit risk, enabling LLM-driven benchmarking for text-to-code capabilities and regulatory text interpretation. We evaluate text-to-code gen- eration capabilities across various state-of-the-art LLMs, temperature set- tings and in-context learning techniques. Advanced prompt engineering is used to enhance code generation. The generated codes are evaluated on test cases, using an unbiased pass@k proxy ––a measure for assessing code gen- eration performance–– achieving pass@1 rates of up to 75.38% and pass@10 rates of up to 91.67%. Additionally, we propose workflows that combine LLMs together with human expertise to enhance regulatory implementation. These results, combined with the workflows we propose, demonstrate that the proposed methods can significantly support the software development for regulatory implementation.

Keywords: Financial Regulation, Compliance, Basel III, Large Language Models, Machine Learning, Code Generation, Generative AI, Artificial Intelligence