Melanie McGrath
Author directory2025
A Dataset and Benchmark on Extraction of Novel Concepts on Trust in AI from Scientific Literature
Melanie McGrath | Harrison Bailey | Necva Bölücü | Xiang Dai | Sarvnaz Karimi | Andreas Duenser | Cécile Paris
Proceedings of the 23rd Annual Workshop of the Australasian Language Technology Association
Melanie McGrath | Harrison Bailey | Necva Bölücü | Xiang Dai | Sarvnaz Karimi | Andreas Duenser | Cécile Paris
Proceedings of the 23rd Annual Workshop of the Australasian Language Technology Association
Information extraction from the scientific literature is a long-standing technique for transforming unstructured knowledge hidden in text into structured data, which can then be used for further analytics and decision-making in downstream tasks. A large body of scientific literature discusses Trust in AI, where factors contributing to human trust in artificial intelligence (AI) applications and technology are studied. It explores questions such as why people may or may not trust a self-driving car, and what factors influence such trust. The relationships of these factors with human trust in AI applications are complex. We explore this space through the lens of information extraction. That is, we investigate how to extract these factors from the literature that studies them. The outcome could inform technology developers to improve the acceptance rate of their products. Our results indicate that (1) while Named Entity Recognition (NER) is largely considered a solved problem in many domains, it is far from solved in extracting factors of human trust in AI from the relevant scientific literature; and, (2) supervised learning is more effective for this task than prompt-based LLMs.