@inproceedings{kaneko-etal-2019-tmu,
    title = "{TMU} Transformer System Using {BERT} for Re-ranking at {BEA} 2019 Grammatical Error Correction on Restricted Track",
    author = "Kaneko, Masahiro  and
      Hotate, Kengo  and
      Katsumata, Satoru  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications",
    month = aug,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/W19-4422",
    doi = "10.18653/v1/W19-4422",
    pages = "207--212",
    abstract = "We introduce our system that is submitted to the restricted track of the BEA 2019 shared task on grammatical error correction1 (GEC). It is essential to select an appropriate hypothesis sentence from the candidates list generated by the GEC model. A re-ranker can evaluate the naturalness of a corrected sentence using language models trained on large corpora. On the other hand, these language models and language representations do not explicitly take into account the grammatical errors written by learners. Thus, it is not straightforward to utilize language representations trained from a large corpus, such as Bidirectional Encoder Representations from Transformers (BERT), in a form suitable for the learner{'}s grammatical errors. Therefore, we propose to fine-tune BERT on learner corpora with grammatical errors for re-ranking. The experimental results of the W{\&}I+LOCNESS development dataset demonstrate that re-ranking using BERT can effectively improve the correction performance.",
}