%0 Conference Proceedings %T XD at SemEval-2020 Task 12: Ensemble Approach to Offensive Language Identification in Social Media Using Transformer Encoders %A Dong, Xiangjue %A Choi, Jinho D. %Y Herbelot, Aurelie %Y Zhu, Xiaodan %Y Palmer, Alexis %Y Schneider, Nathan %Y May, Jonathan %Y Shutova, Ekaterina %S Proceedings of the Fourteenth Workshop on Semantic Evaluation %D 2020 %8 December %I International Committee for Computational Linguistics %C Barcelona (online) %F dong-choi-2020-xd %X This paper presents six document classification models using the latest transformer encoders and a high-performing ensemble model for a task of offensive language identification in social media. For the individual models, deep transformer layers are applied to perform multi-head attentions. For the ensemble model, the utterance representations taken from those individual models are concatenated and fed into a linear decoder to make the final decisions. Our ensemble model outperforms the individual models and shows up to 8.6% improvement over the individual models on the development set. On the test set, it achieves macro-F1 of 90.9% and becomes one of the high performing systems among 85 participants in the sub-task A of this shared task. Our analysis shows that although the ensemble model significantly improves the accuracy on the development set, the improvement is not as evident on the test set. %R 10.18653/v1/2020.semeval-1.299 %U https://aclanthology.org/2020.semeval-1.299 %U https://doi.org/10.18653/v1/2020.semeval-1.299 %P 2244-2250