Using Machine Learning to Validate a Novel Taxonomy of Phenomenal Translation States

Michael Carl, Sheng Lu, Ali Al-Ramadan


Abstract
We report an experiment in which we use machine learning to validate the empirical objectivity of a novel annotation taxonomy for behavioral translation data. The HOF taxonomy defines three translation states according to which a human translator can be in a state of Orientation (O), Hesitation (H) or in a Flow state (F). We aim at validating the taxonomy based on a manually annotated dataset that consists of six English-Spanish translation sessions (approx 900 words) and 1813 HOF-annotated Activity Units (AUs). Two annotators annotated the data and obtain high average inter-annotator accuracy 0.76 (kappa 0.88). We train two classifiers, a Multi-layer Perceptron (MLP) and a Random Forest (RF) on the annotated data and tested on held-out data. The classifiers perform well on the annotated data and thus confirm the epistemological objectivity of the annotation taxonomy. Interestingly, inter-classifier accuracy scores are higher than between the two human annotators.
Anthology ID:
2024.eamt-1.40
Volume:
Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1)
Month:
June
Year:
2024
Address:
Sheffield, UK
Editors:
Carolina Scarton, Charlotte Prescott, Chris Bayliss, Chris Oakley, Joanna Wright, Stuart Wrigley, Xingyi Song, Edward Gow-Smith, Rachel Bawden, Víctor M Sánchez-Cartagena, Patrick Cadwell, Ekaterina Lapshinova-Koltunski, Vera Cabarrão, Konstantinos Chatzitheodorou, Mary Nurminen, Diptesh Kanojia, Helena Moniz
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation (EAMT)
Note:
Pages:
480–491
Language:
URL:
https://aclanthology.org/2024.eamt-1.40
DOI:
Bibkey:
Cite (ACL):
Michael Carl, Sheng Lu, and Ali Al-Ramadan. 2024. Using Machine Learning to Validate a Novel Taxonomy of Phenomenal Translation States. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 480–491, Sheffield, UK. European Association for Machine Translation (EAMT).
Cite (Informal):
Using Machine Learning to Validate a Novel Taxonomy of Phenomenal Translation States (Carl et al., EAMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eamt-1.40.pdf