Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives

Jiaxin Liu, Yi Yang, Kar Yan Tam


Abstract
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives. These narratives originate from the financial statements of the same company but correspond to different periods, such as year-over-year comparisons. Measuring the subtle semantic differences between these paired narratives enables market stakeholders to gauge changes over time in the company’s financial and operational situations, which is critical for financial decision-making. We find that existing pretrained embedding models and LLM embeddings fall short in discerning these subtle financial narrative shifts. To address this gap, we propose an LLM-augmented pipeline specifically designed for the Financial-STS task. Evaluation on a human-annotated dataset demonstrates that our proposed method outperforms existing methods trained on classic STS tasks and generic LLM embeddings.
Anthology ID:
2024.findings-naacl.168
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2641–2652
Language:
URL:
https://aclanthology.org/2024.findings-naacl.168
DOI:
Bibkey:
Cite (ACL):
Jiaxin Liu, Yi Yang, and Kar Yan Tam. 2024. Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2641–2652, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives (Liu et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.168.pdf
Copyright:
 2024.findings-naacl.168.copyright.pdf