@inproceedings{nishino-etal-2020-reinforcement,
    title = "Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation",
    author = "Nishino, Toru  and
      Ozaki, Ryota  and
      Momoki, Yohei  and
      Taniguchi, Tomoki  and
      Kano, Ryuji  and
      Nakano, Norihisa  and
      Tagawa, Yuki  and
      Taniguchi, Motoki  and
      Ohkuma, Tomoko  and
      Nakamura, Keigo",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.findings-emnlp.202",
    doi = "10.18653/v1/2020.findings-emnlp.202",
    pages = "2223--2236",
    abstract = "Automated generation of medical reports that describe the findings in the medical images helps radiologists by alleviating their workload. Medical report generation system should generate correct and concise reports. However, data imbalance makes it difficult to train models accurately. Medical datasets are commonly imbalanced in their finding labels because incidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are significantly imbalanced. We propose a novel reinforcement learning method with a reconstructor to improve the clinical correctness of generated reports to train the data-to-text module with a highly imbalanced dataset. Moreover, we introduce a novel data augmentation strategy for reinforcement learning to additionally train the model on infrequent findings. From the perspective of a practical use, we employ a Two-Stage Medical Report Generator (TS-MRGen) for controllable report generation from input images. TS-MRGen consists of two separated stages: an image diagnosis module and a data-to-text module. Radiologists can modify the image diagnosis module results to control the reports that the data-to-text module generates. We conduct an experiment with two medical datasets to assess the data-to-text module and the entire two-stage model. Results demonstrate that the reports generated by our model describe the findings in the input image more correctly.",
}