Don’t Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention

Shu-Ting Pi, Pradeep Bagavan, Yejia Li, Disha Disha, Qun Liu


Abstract
Utilizing Large Language Models (LLM) as chatbots in diverse business scenarios often presents the challenge of maintaining topic continuity. Abrupt shifts in topics can lead to poor user experiences and inefficient utilization of computational resources. In this paper, we present a topic continuity model aimed at assessing whether a response aligns with the initial conversation topic. Our model is built upon the expansion of the corresponding natural language understanding (NLU) model into quantifiable terms using a Naive Bayes approach. Subsequently, we have introduced an attention mechanism and logarithmic nonlinearity to enhance its capability to capture topic continuity. This approach allows us to convert the NLU model into an interpretable analytical formula. In contrast to many NLU models constrained by token limits, our proposed model can seamlessly handle conversations of any length with linear time complexity. Furthermore, the attention mechanism significantly improves the model’s ability to identify topic continuity in complex conversations. According to our experiments, our model consistently outperforms traditional methods, particularly in handling lengthy and intricate conversations. This unique capability offers us an opportunity to ensure the responsible and interpretable use of LLMs.
Anthology ID:
2024.emnlp-industry.6
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
November
Year:
2024
Address:
Miami, Florida, US
Editors:
Franck Dernoncourt, Daniel Preoţiuc-Pietro, Anastasia Shimorina
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
65–72
Language:
URL:
https://aclanthology.org/2024.emnlp-industry.6
DOI:
Bibkey:
Cite (ACL):
Shu-Ting Pi, Pradeep Bagavan, Yejia Li, Disha Disha, and Qun Liu. 2024. Don’t Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 65–72, Miami, Florida, US. Association for Computational Linguistics.
Cite (Informal):
Don’t Shoot The Breeze: Topic Continuity Model Using Nonlinear Naive Bayes With Attention (Pi et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-industry.6.pdf
Poster:
 2024.emnlp-industry.6.poster.pdf