Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems

Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor 2022-11 Association for Computational Linguistics Online text conference publication eval4nlp-2022-evaluation 10.18653/v1/2022.eval4nlp-1.0 https://aclanthology.org/2022.eval4nlp-1.0/ A Japanese Corpus of Many Specialized Domains for Word Segmentation and Part-of-Speech Tagging Shohei Higashiyama author Masao Ideuchi author Masao Utiyama author Yoshiaki Oida author Eiichiro Sumita author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication higashiyama-etal-2022-japanese 10.18653/v1/2022.eval4nlp-1.1 https://aclanthology.org/2022.eval4nlp-1.1/ 2022-11 1 10 Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis Zachary Zhou author Alisha Zachariah author Devin Conathan author Jeffery Kline author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication zhou-etal-2022-assessing 10.18653/v1/2022.eval4nlp-1.2 https://aclanthology.org/2022.eval4nlp-1.2/ 2022-11 11 20 From COMET to COMES – Can Summary Evaluation Benefit from Translation Evaluation? Mateusz Krubiński author Pavel Pecina author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication krubinski-pecina-2022-comet 10.18653/v1/2022.eval4nlp-1.3 https://aclanthology.org/2022.eval4nlp-1.3/ 2022-11 21 31 Better Smatch = Better Parser? AMR evaluation is not so simple anymore Juri Opitz author Anette Frank author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication opitz-frank-2022-better 10.18653/v1/2022.eval4nlp-1.4 https://aclanthology.org/2022.eval4nlp-1.4/ 2022-11 32 43 GLARE: Generative Left-to-right AdversaRial Examples Ryan Andrew Chi author Nathan Kim author Patrick Liu author Zander Lack author Ethan A Chi author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication chi-etal-2022-glare 10.18653/v1/2022.eval4nlp-1.5 https://aclanthology.org/2022.eval4nlp-1.5/ 2022-11 44 50 Random Text Perturbations Work, but not Always Zhengxiang Wang author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication wang-2022-random 10.18653/v1/2022.eval4nlp-1.6 https://aclanthology.org/2022.eval4nlp-1.6/ 2022-11 51 57 A Comparative Analysis of Stance Detection Approaches and Datasets Parush Gera author Tempestt Neal author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication gera-neal-2022-comparative 10.18653/v1/2022.eval4nlp-1.7 https://aclanthology.org/2022.eval4nlp-1.7/ 2022-11 58 69 Why is sentence similarity benchmark not predictive of application-oriented task performance? Kaori Abe author Sho Yokoi author Tomoyuki Kajiwara author Kentaro Inui author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication abe-etal-2022-sentence 10.18653/v1/2022.eval4nlp-1.8 https://aclanthology.org/2022.eval4nlp-1.8/ 2022-11 70 87 Chat Translation Error Detection for Assisting Cross-lingual Communications Yunmeng Li author Jun Suzuki author Makoto Morishita author Kaori Abe author Ryoko Tokuhisa author Ana Brassard author Kentaro Inui author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication li-etal-2022-chat 10.18653/v1/2022.eval4nlp-1.9 https://aclanthology.org/2022.eval4nlp-1.9/ 2022-11 88 95 Evaluating the role of non-lexical markers in GPT-2’s language modeling behavior Roberta Rocca author Alejandro de la Vega author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication rocca-de-la-vega-2022-evaluating 10.18653/v1/2022.eval4nlp-1.10 https://aclanthology.org/2022.eval4nlp-1.10/ 2022-11 96 102 Assessing Neural Referential Form Selectors on a Realistic Multilingual Dataset Guanyi Chen author Fahime Same author Kees Van Deemter author 2022-11 text Proceedings of the 3rd Workshop on Evaluation and Comparison of NLP Systems Daniel Deutsch editor Can Udomcharoenchaikit editor Juri Opitz editor Yang Gao editor Marina Fomicheva editor Steffen Eger editor Association for Computational Linguistics Online conference publication chen-etal-2022-assessing 10.18653/v1/2022.eval4nlp-1.11 https://aclanthology.org/2022.eval4nlp-1.11/ 2022-11 103 114