Large language models (LLMs) frequently hallucinate when confronted with queries they cannot answer reliably. The root causes are multifaceted, including entity rarity, semantic ambiguity, temporal invalidity, false presuppositions, and other forms of epistemic uncertainty. Existing abstention mechanisms overwhelmingly rely on binary probes trained with manually curated “unanswerable” labels or calibrated confidence scores, both of which presuppose access to negative examples that are expensive to collect and fundamentally incomplete in coverage. We propose FLOW-ABSTAIN, an unsupervised framework that reframes query-aware abstention as density estimation in the model’s own latent space. Our method operates in three stages: (1) extracting intermediate hidden-state representations from a curated set of confidently answerable queries, (2) applying PCA-based dimensionality reduction to obtain a compact manifold, and (3) fitting a Masked Autoregressive Flow (MAF) to the resulting representations, yielding an exact, tractable log-likelihood for arbitrary new inputs. At inference time, queries whose latent projections receive low log-likelihood under the learned density are flagged for abstention—without requiring any negative or out-of-distribution training samples. Experiments across three instruction-tuned LLMs (Qwen-3-8B, Qwen-3-14B, Llama-3-8B-Instruct) and five diverse evaluation benchmarks demonstrate that FLOW-ABSTAIN achieves highly competitive abstention performance among methods with practical inference cost while providing a continuous, interpretable likelihood signal that naturally stratifies query difficulty along cognitive dimensions of unanswerability.

The rapid advancement of large language models (LLMs) has opened up promising opportunities for their downstream applications in question-answering (QA), such as ChatGPT, ChatGLM, etc. However, such LLMs do not perform very well in domain-specific QA tasks without fine-tuning. But directly fine-tuning LLMs on domain-specific corpus data may lead to catastrophic forgetting, causing the LLMs to lose their general language capability. To address this problem, we propose the Knowledge-Enhanced Fine-Tuning (KEFT) method, an unsupervised fine-tuning approach to enhance the knowledge capability of LLMs in domain-specific QA tasks while preserving their general language capability. KEFT leverages the inherent language comprehension of pre-trained LLMs to generate synthetic-QA datasets from domain-specific corpus data autonomously for fine-tuning, and adopts a Low-Rank Adaptation (LoRA) method to further alleviate over-fitting. Furthermore, to enhance the representation of domain-specific knowledge, we introduce a knowledge-enhanced fine-tuning loss function, which encourages the model to learn the knowledge-question connection, thereby generating natural and knowledgeable answers. Our evaluations across multiple domain-specific datasets demonstrate that KEFT surpasses state-of-the-art fine-tuning approaches, enhancing the performance of various LLMs in QA tasks in both English and Chinese languages.