Says Who…? Identification of Expert versus Layman Critics’ Reviews of Documentary Films

Ming Jiang, Jana Diesner


Abstract
We extend classic review mining work by building a binary classifier that predicts whether a review of a documentary film was written by an expert or a layman with 90.70% accuracy (F1 score), and compare the characteristics of the predicted classes. A variety of standard lexical and syntactic features was used for this supervised learning task. Our results suggest that experts write comparatively lengthier and more detailed reviews that feature more complex grammar and a higher diversity in their vocabulary. Layman reviews are more subjective and contextualized in peoples’ everyday lives. Our error analysis shows that laymen are about twice as likely to be mistaken as experts than vice versa. We argue that the type of author might be a useful new feature for improving the accuracy of predicting the rating, helpfulness and authenticity of reviews. Finally, the outcomes of this work might help researchers and practitioners in the field of impact assessment to gain a more fine-grained understanding of the perception of different types of media consumers and reviewers of a topic, genre or information product.
Anthology ID:
C16-1200
Volume:
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Yuji Matsumoto, Rashmi Prasad
Venue:
COLING
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
2122–2132
Language:
URL:
https://aclanthology.org/C16-1200
DOI:
Bibkey:
Cite (ACL):
Ming Jiang and Jana Diesner. 2016. Says Who…? Identification of Expert versus Layman Critics’ Reviews of Documentary Films. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2122–2132, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Says Who…? Identification of Expert versus Layman Critics’ Reviews of Documentary Films (Jiang & Diesner, COLING 2016)
Copy Citation:
PDF:
https://aclanthology.org/C16-1200.pdf