Denis Trautner
Author directory2026
NERQual: Evaluating the Robustness of Named Entity Recognition Models to Data Quality Issues
Denis Trautner | Philippe Thomas
Proceedings of the 11th Workshop on Natural User-generated Text
Denis Trautner | Philippe Thomas
Proceedings of the 11th Workshop on Natural User-generated Text
Transformer-based models for Named Entity Recognition (NER) achieve strong performance on clean benchmarks, yet their reliability in real-world settings remains insufficiently understood. In practical applications, NER systems are frequently exposed to degraded input originating from optical character recognition, automatic speech recognition, and user-generated text. We address this gap by deriving a taxonomy of data quality perturbations from a systematic literature review, consolidating fragmented prior work into a unified framework. Guided by this taxonomy, we conduct controlled experiments evaluating six widely used transformer architectures under varying perturbation types, intensities, and training configurations. Our results show that syntactic perturbations cause the most severe performance degradation across models, while architectural features such as character-level processing and disentangled attention confer specific robustness advantages. Furthermore, we demonstrate that targeted training with perturbed data can recover more than half of the lost performance with minimal impact on clean-data accuracy.