WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia

Kokil Jaidka, Andrea Ceolin, Iknoor Singh, Niyati Chhaya, Lyle Ungar


Abstract
This study introduces and analyzes WikiTalkEdit, a dataset of conversations and edit histories from Wikipedia, for research in online cooperation and conversation modeling. The dataset comprises dialog triplets from the Wikipedia Talk pages, and editing actions on the corresponding articles being discussed. We show how the data supports the classic understanding of style matching, where positive emotion and the use of first-person pronouns predict a positive emotional change in a Wikipedia contributor. However, they do not predict editorial behavior. On the other hand, feedback invoking evidentiality and criticism, and references to Wikipedia’s community norms, is more likely to persuade the contributor to perform edits but is less likely to lead to a positive emotion. We developed baseline classifiers trained on pre-trained RoBERTa features that can predict editorial change with an F1 score of .54, as compared to an F1 score of .66 for predicting emotional change. A diagnostic analysis of persisting errors is also provided. We conclude with possible applications and recommendations for future work. The dataset is publicly available for the research community at https://github.com/kj2013/WikiTalkEdit/.
Anthology ID:
2021.naacl-main.177
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2191–2200
Language:
URL:
https://aclanthology.org/2021.naacl-main.177
DOI:
10.18653/v1/2021.naacl-main.177
Bibkey:
Cite (ACL):
Kokil Jaidka, Andrea Ceolin, Iknoor Singh, Niyati Chhaya, and Lyle Ungar. 2021. WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2191–2200, Online. Association for Computational Linguistics.
Cite (Informal):
WikiTalkEdit: A Dataset for modeling Editors’ behaviors on Wikipedia (Jaidka et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.177.pdf
Optional supplementary code:
 2021.naacl-main.177.OptionalSupplementaryCode.zip
Video:
 https://aclanthology.org/2021.naacl-main.177.mp4