He Liu
Author directoryOther people with similar names: He Liu
Unverified author pages with similar names: He Liu
2026
PhysDox: Benchmarking LLMs on Physical Feasibility Auditing of Physiological Sensing Protocols
He Liu | Boyuan Gu | Shuaiqi Cheng | Haiyang Sun | Siyu You | Xuming Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
He Liu | Boyuan Gu | Shuaiqi Cheng | Haiyang Sun | Siyu You | Xuming Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) are increasingly used to assist scientific writing and experimental design, but a fluent protocol can still be physically infeasible. We introduce PhysDox, a benchmark for physical feasibility auditing of biomedical experimental protocols, comprising a 683-sample expert-curated Gold set and a 5,000-sample expanded Silver set across six sensing domains. PhysDox formulates protocol auditing as a two-stage task: severity detection over valid, minor issue, and fatal violation protocols, followed by diagnosis of fatal violations over coarse and fine-grained physical constraints. We evaluate 6 recent LLM variants under 4 inference strategies. Results show that current LLMs remain poorly calibrated at the severity gate: the best Gold Stage-1 macro-F1 is only 53.0, and strong oracle diagnosis often collapses after end-to-end gating. Qualitative case analyses reveal a shared failure mode: models reward complete experimental scaffolds while missing hidden physical constraints. These results show that protocol auditing requires calibrated feasibility reasoning over complete experimental procedures, rather than factual recall or isolated physics problem solving.