Samuel Forster

Author directory

2026

Many domains require reasoning over unstructured text to identify attributes of interest that are not stated in a standardised form. Useful judgements on the presence of these attributes should also express calibrated confidence statements and cite supporting text as evidence. We define this task as Finding Attributes with Supporting Text (FAST). We introduce FAST-Eval, an uncertainty evaluation framework for confidence calibration, attribute identification and evidential grounding in FAST, together with a dataset of 1,000 human-annotated (document, attribute) pairs validated against an independent recruitment expert (Spearman ρs = 0.853, binary κ = 0.715). Taking employability skills in CVs as a case study, we use FAST-Eval to test whether FAST-LoRA, a Low-Rank Adaptation (LoRA) adapter applied to Qwen3-8B, improves performance in these three dimensions. Trained on 500 pairs, FAST-LoRA outperforms its untuned Qwen3-8B counterpart and GPT-5 on unseen skill identification (F1 0.877 vs 0.736 and 0.766) and calibration, while matching GPT-5 on citation fidelity. Both untuned models assign the highest confidence level to about half their judgements; FAST-LoRA instead distributes confidence across the scale and makes fewer unsupported positive predictions. These results suggest that modest domain adaptation is a more effective strategy than scaling model parameters for FAST.