Denoising Attention for Query-aware User Modeling

Elias Bassani, Pranav Kasela, Gabriella Pasi


Abstract
Personalization of search results has gained increasing attention in the past few years, also thanks to the development of Neural Networks-based approaches for Information Retrieval. Recent works have proposed to build user models at query time by leveraging the Attention mechanism, which allows weighing the contribution of the user-related information w.r.t. the current query.This approach allows giving more importance to the user’s interests related to the current search performed by the user.In this paper, we discuss some shortcomings of the Attention mechanism when employed for personalization and introduce a novel Attention variant, the Denoising Attention, to solve them.Denoising Attention adopts a robust normalization scheme and introduces a filtering mechanism to better discern among the user-related data those helpful for personalization.Experimental evaluation shows improvements in MAP, MRR, and NDCG above 15% w.r.t. other Attention variants at the state-of-the-art.
Anthology ID:
2024.findings-naacl.153
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2368–2380
Language:
URL:
https://aclanthology.org/2024.findings-naacl.153
DOI:
Bibkey:
Cite (ACL):
Elias Bassani, Pranav Kasela, and Gabriella Pasi. 2024. Denoising Attention for Query-aware User Modeling. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2368–2380, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Denoising Attention for Query-aware User Modeling (Bassani et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.153.pdf
Copyright:
 2024.findings-naacl.153.copyright.pdf