Antony Papadimitriou


2024

pdf bib
The State of the Art of Large Language Models on Chartered Financial Analyst Exams
Mahmoud Mahfouz | Ethan Callanan | Mathieu Sibue | Antony Papadimitriou | Zhiqiang Ma | Xiaomo Liu | Xiaodan Zhu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track

The Chartered Financial Analyst (CFA) program is one of the most widely recognized financial certifications globally. In this work, we test a variety of state-of-the-art large language models (LLMs) on mock CFA exams to provide an overview of their financial analysis capabilities using the same evaluation standards applied for human professionals. We benchmark five leading proprietary models and eight open-source models on all three levels of the CFA through challenging multiple-choice and essay questions. We find that flagship proprietary models perform relatively well and can solidly pass levels I and II exams, but fail at level III due to essay questions. Open-source models generally fall short of estimated passing scores, but still show strong performance considering their size, cost, and availability advantages. We also find that using textbook data helps bridge the gap between open-source and proprietary models to a certain extent, despite reduced gains in CFA levels II and III. By understanding the current financial analysis abilities of LLMs, we aim to guide practitioners on which models are best suited for enhancing automation in the financial industry.

pdf bib
Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams
Ethan Callanan | Amarachi Mbakwe | Antony Papadimitriou | Yulong Pei | Mathieu Sibue | Xiaodan Zhu | Zhiqiang Ma | Xiaomo Liu | Sameena Shah
Proceedings of the Eighth Financial Technology and Natural Language Processing and the 1st Agent AI for Scenario Planning