Sidharth N Krishna

Author directory

2026

Health insurance policy documents justify coverage decisions through clinical evidence and regulatory criteria, but the underlying reasoning is buried in dense medical and legal text. A pipeline is presented that extracts argument components and relations from these documents and assembles them into directed argument graphs. A dataset of 5,230 sentences and 5,685 relations across 53 documents from Aetna, CMS, and United Healthcare is constructed along with two custom annotation tools. Six classical baselines, three fine-tuned transformer models, and three instruction-tuned LLMs are evaluated, with the LLMs assessed under zero-shot, few-shot, and chain-of-thought prompting. PubMedBERT and Legal-BERT, fine-tuned on the dataset, achieve the strongest results overall (0.731 and 0.653 Macro F1 on component classification and relation extraction, respectively). Notably, the much simpler TF-IDF + SVM baseline is competitive with these fine-tuned transformers (0.709 and 0.649 Macro F1) and outperforms the best prompted LLM by 12 and 21 F1 points respectively. The dataset and code are available at https://anonymous.4open.science/r/LLM-Driven-Argument-Mapping-3ACB/README.md