%0 Conference Proceedings %T ISCAS at SemEval-2020 Task 5: Pre-trained Transformers for Counterfactual Statement Modeling %A Lu, Yaojie %A Li, Annan %A Lin, Hongyu %A Han, Xianpei %A Sun, Le %Y Herbelot, Aurelie %Y Zhu, Xiaodan %Y Palmer, Alexis %Y Schneider, Nathan %Y May, Jonathan %Y Shutova, Ekaterina %S Proceedings of the Fourteenth Workshop on Semantic Evaluation %D 2020 %8 December %I International Committee for Computational Linguistics %C Barcelona (online) %F lu-etal-2020-iscas %X ISCAS participated in two subtasks of SemEval 2020 Task 5: detecting counterfactual statements and detecting antecedent and consequence. This paper describes our system which is based on pretrained transformers. For the first subtask, we train several transformer-based classifiers for detecting counterfactual statements. For the second subtask, we formulate antecedent and consequence extraction as a query-based question answering problem. The two subsystems both achieved third place in the evaluation. Our system is openly released at https://github.com/casnlu/ISCASSemEval2020Task5. %R 10.18653/v1/2020.semeval-1.85 %U https://aclanthology.org/2020.semeval-1.85 %U https://doi.org/10.18653/v1/2020.semeval-1.85 %P 658-663