Koen Dercksen
Author directory2026
ARR-Dv2: New Data from ACL Rolling Review and its Potential to Improve Reviewing in the ACL Community
Atnafu Lambebo Tonja | Sheng Lu | Nils Dycke | Dirk Hovy | Koen Dercksen | Lizhen Qu | Xiaodan Zhu | Thamar Solorio | Iryna Gurevych | Margot Mieskes
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Atnafu Lambebo Tonja | Sheng Lu | Nils Dycke | Dirk Hovy | Koen Dercksen | Lizhen Qu | Xiaodan Zhu | Thamar Solorio | Iryna Gurevych | Margot Mieskes
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
We present a new dataset to support AI-assisted and human studies of the peer reviewing process, based on the ARR Open Review Platform. Unlike previous similar data collections, ours, ARR-Dv2, not only contains the reviews but also author-reviewer discussions (aka rebuttal phase) and meta-reviews. We present statistical analyses of the data with respect to the overall score and the correlations among the overall score, confidence, and soundness scores. Additionally, we extend the task of Review Score Prediction also to include rebuttal data and analyze its effect on score prediction. Finally, we introduce a new task, Meta Review Score Prediction, based on a set of up to three reviews rather than a single review. Our initial results in a zero-shot setting indicate that the rebuttal data adds valuable information to the score prediction and enables reliable predictions.