Bi-encoder architectures for biomedical entity linking learn flat embedding spaces that ignore ontology structure. HiCo-EL encodes hierarchical knowledge into bi-encoder representations through hierarchical contrastive pretraining. The core idea is within-space hierarchical multi-label contrastive learning (HMLC): contrastive losses applied independently within each encoder space, preserving cross-space compatibility for retrieval. Our formulation drops the monotonicity constraints of prior HMLC work, handles variable-depth hierarchy paths, and adds depth-weighted alignment between encoder spaces. Treating each entity as a leaf node unifies coarse type structure with fine-grained entity discrimination. On MedMentions and NCBI Disease, HiCo-EL outperforms ArboEL by +3.7% and +1.8% Recall@1 respectively. The gains reveal a key insight: breadth amplifies, depth generalizes—broad hierarchies help common entities most (+6.3% R@1), while deep hierarchies benefit unseen entities (+9.6% R@1 zero-shot).

Biomedical Named Entity Recognition (NER) consists of identifying and classifying important biomedical entities mentioned in text. Traditionally, biomedical NER has heavily relied on domain-specific pre-trained language models; particularly variant of BERT models. With the emergence of large language models (LLMs), some studies have evaluated their performance on biomedical NLP tasks. These studies consistently show that, despite their general capabilities, LLMs still fall short compared to specialized BERT-based models for biomedical NER. However, as LLMs continue to advance at a remarkable pace, natural questions arise: Are they still far behind, or are they starting to be competitive? In this study, we investigate the performance of recent LLMs across multiple biomedical NER datasets under both clean and noisy dataset conditions. Our findings reveal that LLMs are progressively closing the performance gap with BERT-based models and demonstrate particular strengths in low-data settings. Moreover, our results suggest that in-context learning with LLMs exhibits a notable degree of robustness to noise, making them a promising alternative in settings where labeled data is scarce or noisy.

Reinforcement learning has become a common approach for improving language-model reasoning, but its effectiveness depends heavily on reward design, especially for compact domain-specialized models. We study structured biomedical question answering with two 4B-scale models, MedGemma-1.5-4B-it and Qwen3-4B-Thinking-2507, using a two-stage SFT+GRPO pipeline that trains models to produce tagged think/answer outputs. We compare three reward schemes: a simple format + accuracy reward, a depth incentivized reward that encourages multiple reasoning cycles, and a gated LLM-judge reward for intermediate reasoning milestones. Across five biomedical QA benchmarks, the simplest reward is the most robust: it substantially improves MedGemma in the structured-output setting, while depth and LLM-judge rewards often reduce accuracy. For Qwen3, SFT or native long-budget inference can outperform GRPO on several benchmarks, showing that reasoning-tuned models may not benefit from additional structural reinforcement. Our results suggest that, for lightweight biomedical QA, reward simplicity and model priors are more important than increasingly elaborate intermediate reasoning supervision. The codebase for this study is publicly available on GitHub: https://github.com/batuinmetz/When-more-supervision-hurts.

Entity linking involves normalizing a mention in medical text to a unique identifier in a knowledge base, such as UMLS or MeSH. Most entity linkers follow a two-stage process: first, a candidate generation step selects high-quality candidates, and then a named entity disambiguation phase determines the best candidate for final linking. This study demonstrates that leveraging a large language model (LLM) as an entity disambiguator significantly enhances entity linking models’ accuracy and recall. Specifically, the LLM disambiguator achieves remarkable improvements when applied to alias-matching entity linking methods. Without any fine-tuning, our approach establishes a new state-of-the-art (SOTA), surpassing previous methods on multiple prevalent biomedical datasets by up to 16 points in accuracy. We released our code on GitHub at https://github.com/ChristopheYe/llm_disambiguator