Xavier Coubez

Author directory

2026

The variation in writing style encapsulates nuanced characteristics, which are often exploited for author or demographic identification. In the medical domain, language models are frequently deployed to capture relevant information from unstructured or complex data, such as clinical notes that often include patients’ medical histories. Such data is largely free-form and unstructured, obtained through diverse clinician?patient interactions. In this work, we present a case study investigating whether variations in clinicians’ writing styles can lead to differences in medical context understanding capabilities for pre-trained language models (PLMs) on downstream tasks, such as medical event classification. Our findings indicate that variation in writing style, characterized by linguistic features, can indeed lead to suboptimal performance in deployed systems. Furthermore, we explore linguistic guided counterfactual reasoning in order to mitigate the impact of writing style variation which suggests LLM-based stylistic normalization to be effective for this purpose.
Language model (LM) capabilities have rapidly advanced over the past decade, driving their adoption across fields, including biomedical research. While domain-specific pretrained language models (PLMs) have long supported biomedical NLP, recent focus has shifted to medical large language models (LLMs) and their language understanding capabilities. In this paper, we investigate this capability by probing the medical conceptual alignment of various LMs against clinical expertise for identifying hallmarks of immunotherapy in breast cancer abstracts. Our results show a clear trade-off: LLMs achieve stronger performance and higher consistency on high-level concepts, whereas PLMs are more reliable for fine-grained, low-level concept identification. We find that hallucinations are the primary factor limiting LLM performance in these settings; once mitigated, LLMs surpass PLMs across all tiers of biomedical concepts. Despite this, PLMs remain competitive, particularly in closely aligning with medical expert judgments on detailed conceptual distinctions.

2024

The success of pretrained language models (PLMs) across a spate of use-cases has led to significant investment from the NLP community towards building domain-specific foundational models. On the other hand, in mission critical settings such as biomedical applications, other aspects also factor in—chief of which is a model’s ability to produce reasonable estimates of its own uncertainty. In the present study, we discuss these two desiderata through the lens of how they shape the entropy of a model’s output probability distribution. We find that domain specificity and uncertainty awareness can often be successfully combined, but the exact task at hand weighs in much more strongly.