Yang Liu

University of Helsinki

Other people with similar names: Yang Liu (May refer to several people), Yang Liu (3M Health Information Systems), Yang Liu (National University of Defense Technology), Yang Liu (Edinburgh), Yang (Janet) Liu (刘洋; Georgetown), Yang Liu (Georgetown University), Yang Liu (The Chinese University of Hong Kong (Shenzhen)), Yang Liu (刘扬; Ph.D Purdue; ICSI, Dallas, Facebook, Liulishuo, Amazon), Yang Liu (刘洋; ICT, Tsinghua, Beijing Academy of Artificial Intelligence), Yang Liu (Microsoft Cognitive Services Research), Yang Liu (Peking University), Yang Liu (Univ. of Michigan, UC Santa Cruz)


pdf bib
Statistically Significant Detection of Semantic Shifts using Contextual Word Embeddings
Yang Liu | Alan Medlar | Dorota Glowacka
Proceedings of the 2nd Workshop on Evaluation and Comparison of NLP Systems

Detecting lexical semantic change in smaller data sets, e.g. in historical linguistics and digital humanities, is challenging due to a lack of statistical power. This issue is exacerbated by non-contextual embedding models that produce one embedding per word and, therefore, mask the variability present in the data. In this article, we propose an approach to estimate semantic shift by combining contextual word embeddings with permutation-based statistical tests. We use the false discovery rate procedure to address the large number of hypothesis tests being conducted simultaneously. We demonstrate the performance of this approach in simulation where it achieves consistently high precision by suppressing false positives. We additionally analyze real-world data from SemEval-2020 Task 1 and the Liverpool FC subreddit corpus. We show that by taking sample variation into account, we can improve the robustness of individual semantic shift estimates without degrading overall performance.