FinRAD: Financial Readability Assessment Dataset - 13,000+ Definitions of Financial Terms for Measuring Readability

Sohom Ghosh, Shovon Sengupta, Sudip Naskar, Sunny Kumar Singh


Abstract
In today’s world, the advancement and spread of the Internet and digitalization have resulted in most information being openly accessible. This holds true for financial services as well. Investors make data driven decisions by analysing publicly available information like annual reports of listed companies, details regarding asset allocation of mutual funds, etc. Many a time these financial documents contain unknown financial terms. In such cases, it becomes important to look at their definitions. However, not all definitions are equally readable. Readability largely depends on the structure, complexity and constituent terms that make up a definition. This brings in the need for automatically evaluating the readability of definitions of financial terms. This paper presents a dataset, FinRAD consisting of financial terms, their definitions and embeddings. In addition to standard readability scores (like “Flesch Reading Index (FRI)”, “Automated Readability Index (ARI)”, “SMOG Index Score (SIS)”,“Dale-Chall formula (DCF)”, etc.), it also contains the readability scores (AR) assigned based on sources from which the terms have been collected. We manually inspect a sample from it to ensure the quality of the assignment. Subsequently, we prove that the rule-based standard readability scores (like “Flesch Reading Index (FRI)”, “Automated Readability Index (ARI)”, “SMOG Index Score (SIS)”,“Dale-Chall formula (DCF)”, etc.) do not correlate well with the manually assigned binary readability scores of definitions of financial terms. Finally, we present a few neural baselines using transformer based architecture to automatically classify these definitions as readable or not. Pre-trained FinBERT model fine-tuned on FinRAD corpus performs the best (AU-ROC = 0.9927, F1 = 0.9610). This corpus can be downloaded from https://github.com/sohomghosh/FinRAD_Financial_Readability_Assessment_Dataset.
Anthology ID:
2022.fnp-1.1
Volume:
Proceedings of the 4th Financial Narrative Processing Workshop @LREC2022
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Mahmoud El-Haj, Paul Rayson, Nadhem Zmandar
Venue:
FNP
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1–9
Language:
URL:
https://aclanthology.org/2022.fnp-1.1
DOI:
Bibkey:
Cite (ACL):
Sohom Ghosh, Shovon Sengupta, Sudip Naskar, and Sunny Kumar Singh. 2022. FinRAD: Financial Readability Assessment Dataset - 13,000+ Definitions of Financial Terms for Measuring Readability. In Proceedings of the 4th Financial Narrative Processing Workshop @LREC2022, pages 1–9, Marseille, France. European Language Resources Association.
Cite (Informal):
FinRAD: Financial Readability Assessment Dataset - 13,000+ Definitions of Financial Terms for Measuring Readability (Ghosh et al., FNP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.fnp-1.1.pdf
Code
 sohomghosh/finrad_financial_readability_assessment_dataset