A Dataset for Multi-Scale Film Rating Inference from Reviews

Frankie Robertson, Stefano Leone


Abstract
This resource paper introduces a dataset for multi-scale rating inference of film review scores based upon review summaries. The dataset and task are unique in pairing a text regression problem with ratings given on multiple scales, e.g. the A-F letter scale and the 4-point star scale. It retains entity identifiers such as film and reviewer names. The paper describes the construction of the dataset before exploring potential baseline architectures for the task, and evaluating their performance. Baselines based on classifier-per-scale, affine-per-scale, and ordinal regression models are presented and evaluated with the BERT-base backbone. Additional experiments are used to ground a discussion of the different architectures’ merits and drawbacks with regards to explainability and model interpretation.
Anthology ID:
2024.nlperspectives-1.16
Volume:
Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Gavin Abercrombie, Valerio Basile, Davide Bernadi, Shiran Dudy, Simona Frenda, Lucy Havens, Sara Tonelli
Venues:
NLPerspectives | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
142–150
Language:
URL:
https://aclanthology.org/2024.nlperspectives-1.16
DOI:
Bibkey:
Cite (ACL):
Frankie Robertson and Stefano Leone. 2024. A Dataset for Multi-Scale Film Rating Inference from Reviews. In Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024, pages 142–150, Torino, Italia. ELRA and ICCL.
Cite (Informal):
A Dataset for Multi-Scale Film Rating Inference from Reviews (Robertson & Leone, NLPerspectives-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.nlperspectives-1.16.pdf