Quick, get me a Dr. BERT: Automatic Grading of Evidence using Transfer Learning

Pradeesh Parameswaran, Andrew Trotman, Veronica Liesaputra, David Eyers


Abstract
We describe our methods for automatically grading the level of clinical evidence in medical papers, as part of the ALTA 2021 shared task. We use a combination of transfer learning and a hand-crafted, feature-based classifier. Our system (�orangutanV3�) obtained an accuracy score of 0.4918, which placed third in the leaderboard. From our failure analysis, we find that our classification techniques do not appropriately handle cases when the conclusions of across the medical papers are themselves inconclusive. We believe that this shortcoming can be overcome�thus improving the classification accuracy�by incorporating document similarity techniques.
Anthology ID:
2021.alta-1.24
Volume:
Proceedings of the 19th Annual Workshop of the Australasian Language Technology Association
Month:
December
Year:
2021
Address:
Online
Editors:
Afshin Rahimi, William Lane, Guido Zuccon
Venue:
ALTA
SIG:
Publisher:
Australasian Language Technology Association
Note:
Pages:
205–212
Language:
URL:
https://aclanthology.org/2021.alta-1.24
DOI:
Bibkey:
Cite (ACL):
Pradeesh Parameswaran, Andrew Trotman, Veronica Liesaputra, and David Eyers. 2021. Quick, get me a Dr. BERT: Automatic Grading of Evidence using Transfer Learning. In Proceedings of the 19th Annual Workshop of the Australasian Language Technology Association, pages 205–212, Online. Australasian Language Technology Association.
Cite (Informal):
Quick, get me a Dr. BERT: Automatic Grading of Evidence using Transfer Learning (Parameswaran et al., ALTA 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.alta-1.24.pdf
Code
 prasys/orangutanv3altasharedtask21
Data
ALTA 2021 Shared Task