Large language models (LLMs) are increasingly being adopted in clinical practice, e.g., in medical diagnostic systems. However, their reliability is still a concern, as LLMs are prone to hallucinations and could lead to misdiagnosis, inappropriate treatment decisions, and potential harm to patients. To mitigate these risks, LLM-based medical diagnostic systems should incorporate safety guardrails capable of detecting suspicious predictions and flagging them for human review or further clinical assessment. In this work, we employ uncertainty quantification (UQ) as a safety mechanism within an enterprise diagnostic system. We evaluate 16 UQ methods and propose two task-specific modifications tailored to the medical diagnostic workflow. First, we show that methods based on semantic similarity between generated samples substantially benefit from replacing a general-purpose NLI model with a domain-specific model. Second, we adapt a supervised UQ method that relies on LLM internal states for the detection of erroneous generation from a black-box enterprise LLM accessible only through a text-generation API. Our approach leverages the internal states of an open-source, lightweight proxy white-box model. We demonstrate that this approach outperforms all other UQ methods in the considered setting.

In specialist industrial imagery the dominant cost is maintenance under domain shift: labeling and fine-tuning must be redone on every new field, and proprietary imagery often cannot leave the operator’s environment. For geological drill-core fracture segmentation we release CoreFrac, to our knowledge the first open pixel-level benchmark on drill-core photographs: 200 expert-annotated full-resolution images with an expert-agreement ceiling (soft-F1 0.86). We retarget a frozen concept segmentor (SAM 3) by evolving only its natural-language prompt — no gradient updates, just a small labeled probe. Under a source-grouped protocol the evolved prompt reaches 0.545 positives-only soft-F1, beating rank-8 LoRA at the same 96-patch cap (0.487; paired +0.058, 95% CI [+0.020, +0.094]) and the best of eight expert prompts by 2.7×. Under a controlled domain shift, re-evolving on a small probe lifts a 0.622 → 0.409 drop back to 0.534. Its masks distill into a small on-premises U-Net (7.8M parameters, 3.8 ms/patch) that retains ≈92% of the teacher’s positives-only soft-F1 (0.501) at ≈1700× lower latency than *unbatched* SAM 3 — a cheaper, leak-free deployment. Code and data are released.

We present CLARITY (Clinical Assistant for Routing, Inference and Triage), an AI-driven platform designed to facilitate patient-to-specialist routing, clinical consultations, and severity assessment of patient conditions. Its hybrid architecture combines a Finite State Machine (FSM) for structured dialogue flows with collaborative agents that employ Large Language Model (LLM) to analyze symptoms and prioritize referrals to appropriate specialists. Built on a modular microservices framework, CLARITY ensures safe, efficient, and robust performance, flexible and readily scalable to meet the demands of existing workflows and IT solutions in healthcare.We report integration of our clinical assistant into a large-scale national interhospital platform, with more than 55,000 content-rich userdialogues completed within the two months of deployment, 2,500 of which were expert-annotated for subsequent validation. The validation results show that CLARITY surpasses human-level performance in terms of the first-attempt routing precision, naturally requiring up to 3 times shorter duration of the consultation than with a human.

We present ALToolbox – an open-source framework for active learning (AL) annotation in natural language processing. Currently, the framework supports text classification, sequence tagging, and seq2seq tasks. Besides state-of-the-art query strategies, ALToolbox provides a set of tools that help to reduce computational overhead and duration of AL iterations and increase annotated data reusability. The framework aims to support data scientists and researchers by providing an easy-to-deploy GUI annotation tool directly in the Jupyter IDE and an extensible benchmark for novel AL methods. We prepare a small demonstration of ALToolbox capabilities available online. The code of the framework is published under the MIT license.

Annotating training data for sequence tagging of texts is usually very time-consuming. Recent advances in transfer learning for natural language processing in conjunction with active learning open the possibility to significantly reduce the necessary annotation budget. We are the first to thoroughly investigate this powerful combination for the sequence tagging task. We conduct an extensive empirical study of various Bayesian uncertainty estimation methods and Monte Carlo dropout options for deep pre-trained models in the active learning framework and find the best combinations for different types of models. Besides, we also demonstrate that to acquire instances during active learning, a full-size Transformer can be substituted with a distilled version, which yields better computational performance and reduces obstacles for applying deep active learning in practice.