A Graph per Persona: Reasoning about Subjective Natural Language Descriptions

EunJeong Hwang, Vered Shwartz, Dan Gutfreund, Veronika Thost


Abstract
Reasoning about subjective natural language descriptions, such as opinions and preferences, is a challenging topic that largely remains unsolved to date. In particular, state-of-the-art large language models (LLMs) perform disappointingly in this task, show strong biases, and do not meet the interpretability requirements often needed in these kinds of applications. We propose a novel approach for reasoning about subjective knowledge that integrates potential and implicit meanings and explicitly models the relational nature of the information. We apply supervised graph learning, offer explanations for the model’s reasoning, and show that our model performs well across all 15 topics of OpinionQA, outperforming several prominent LLMs. Our detailed analysis further shows its unique advantages and the complementary nature it offers in comparison to LLMs.
Anthology ID:
2024.findings-acl.115
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1928–1942
Language:
URL:
https://aclanthology.org/2024.findings-acl.115
DOI:
Bibkey:
Cite (ACL):
EunJeong Hwang, Vered Shwartz, Dan Gutfreund, and Veronika Thost. 2024. A Graph per Persona: Reasoning about Subjective Natural Language Descriptions. In Findings of the Association for Computational Linguistics ACL 2024, pages 1928–1942, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
A Graph per Persona: Reasoning about Subjective Natural Language Descriptions (Hwang et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.115.pdf