An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls

Ramit Sawhney, Arshiya Aggarwal, Rajiv Ratn Shah


Abstract
Volatility prediction is complex due to the stock market’s stochastic nature. Existing research focuses on the textual elements of financial disclosures like earnings calls transcripts to forecast stock volatility and risk, but ignores the rich acoustic features in the company executives’ speech. Recently, new multimodal approaches that leverage the verbal and vocal cues of speakers in financial disclosures significantly outperform previous state-of-the-art approaches demonstrating the benefits of multimodality and speech. However, the financial realm is still plagued with a severe underrepresentation of various communities spanning diverse demographics, gender, and native speech. While multimodal models are better risk forecasters, it is imperative to also investigate the potential bias that these models may learn from the speech signals of company executives. In this work, we present the first study to discover the gender bias in multimodal volatility prediction due to gender-sensitive audio features and fewer female executives in earnings calls of one of the world’s biggest stock indexes, the S&P 500 index. We quantitatively analyze bias as error disparity and investigate the sources of this bias. Our results suggest that multimodal neural financial models accentuate gender-based stereotypes.
Anthology ID:
2021.naacl-main.294
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3751–3757
Language:
URL:
https://aclanthology.org/2021.naacl-main.294
DOI:
10.18653/v1/2021.naacl-main.294
Bibkey:
Cite (ACL):
Ramit Sawhney, Arshiya Aggarwal, and Rajiv Ratn Shah. 2021. An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3751–3757, Online. Association for Computational Linguistics.
Cite (Informal):
An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls (Sawhney et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.294.pdf
Video:
 https://aclanthology.org/2021.naacl-main.294.mp4
Code
 midas-research/multimodal-bias-naacl