Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach

Masahiro Kaneko, Graham Neubig, Naoaki Okazaki


Abstract
Humans work together to solve common problems by having discussions, explaining, and agreeing or disagreeing with each other.Similarly, if a system can have discussions with human partners when solving tasks, it has the potential to improve the system’s performance and reliability.In previous research on explainability, it has only been possible for systems to make predictions and for humans to ask questions about them, rather than having a mutual exchange of opinions.This research aims to create a dataset and computational framework for systems that discuss and refine their predictions through dialogue. Through experiments, we show that the proposed system can have beneficial discussions with humans, improving the accuracy by up to 25 points on a natural language inference task.
Anthology ID:
2024.findings-eacl.114
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1644–1658
Language:
URL:
https://aclanthology.org/2024.findings-eacl.114
DOI:
Bibkey:
Cite (ACL):
Masahiro Kaneko, Graham Neubig, and Naoaki Okazaki. 2024. Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1644–1658, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach (Kaneko et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-eacl.114.pdf
Note:
 2024.findings-eacl.114.note.zip