FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports

Chenying Li, Wenbo Ye, Yilun Zhao


Abstract
Answering questions over financial reports containing both tabular and textual data (hybrid data) is challenging as it requires models to select information from financial reports and perform complex quantitative analyses. Although current models have demonstrated a solid capability to solve simple questions, they struggle with complex questions that require a multiple-step numerical reasoning process. This paper proposes a new framework named FinMath, which improves the model’s numerical reasoning capacity by injecting a tree-structured neural model to perform multi-step numerical reasoning. Specifically, FinMath extracts supporting evidence from the financial reports given the question in the first phase. In the second phase, a tree-structured neural model is applied to generate a tree expression in a top-down recursive way. Experiments on the TAT-QA dataset show that our proposed approach improves the previous best result by 8.5% absolute for Exact Match (EM) score (50.1% to 58.6%) and 6.1% absolute for numeracy-focused F1 score (58.0% to 64.1%).
Anthology ID:
2022.lrec-1.661
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
6147–6152
Language:
URL:
https://aclanthology.org/2022.lrec-1.661
DOI:
Bibkey:
Cite (ACL):
Chenying Li, Wenbo Ye, and Yilun Zhao. 2022. FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 6147–6152, Marseille, France. European Language Resources Association.
Cite (Informal):
FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports (Li et al., LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.661.pdf
Data
MATHTAT-QA