Solving Mathematical Problems Using Large Language Models: A Survey

Abstract

Large Language Models (LLMs) exhibit impressive performance across various Natural Language Processing (NLP) tasks due to their robust contextual understanding, content generation and few/zero-shot learning abilities. However, LLMs still show significant limitations while handling with mathematical problems that require complex reasoning skills and interpretable solving processes. Consequently, a series of research efforts have been made in solving mathematical problems using Large Language Models (SMP-LLM). This survey provides a comprehensive review of such endeavors. First, we introduce a two-layer classification system for SMP-LLM: at the primary layer, we categorized extant researches into four classes of solution methods, including: fine-tuning, prompt engineering, collaboration with symbolic solvers, and collaboration with evaluators/validators. At the second layer, we classified mathematical problems into four categories: math word problem, geometry problem, theorem proving, and combinatorial optimization problem. This classification system demonstrated the correlation between solution methods and the categories of mathematical problems. Second, we analyzed typical research works under of each method, and summarized their strengths and weaknesses. Third, elucidated current mainstream datasets for solving mathematical problems and analyzed how these datasets promote SMP-LLM research from different perspectives. Finally, summarized the challenges that SMP-LLM are facing and highlighted four research directions: geometric analysis, comprehension, and generation of mathematical expressions, indirect reasoning and benchmarks for evaluating mathematical ability. We hope that this survey can provide useful references for researchers interested in SMP-LLM.

Keywords: mathematical problems, symbolic solvers, Knowledge Graph, Large Language Models, survey