JingYuan Zhao
Author directoryAlso published as: Jingyuan Zhao
2026
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
To develop a reliable AI for psychological assessment, we introduce PsychEval, a multi-session, multi-therapy, and highly realistic benchmark designed to address three key challenges:1) Can we train a highly realistic AI counselor? Realistic counseling is a longitudinal task requiring sustained memory and dynamic goal tracking. We propose a multi-session benchmark (spanning 6-10 sessions across three distinct stages) that demands critical capabilities such as memory continuity, adaptive reasoning, and longitudinal planning. The dataset is annotated with extensive professional skills, comprising over 677 meta-skills and 4577 atomic skills. 2) How to train a multi-therapy AI counselor? While existing models often focus on a single therapy, complex cases frequently require flexible strategies among various therapies. We construct a diverse dataset covering five therapeutic modalities alongside an integrative therapy with a unified three-stage clinical framework across six core psychological topics. 3) How to systematically evaluate an AI counselor? We establish a holistic evaluation framework with 18 therapy-specific and therapy-shared metrics across Client-Level and Counselor-Level dimensions. To We also construct over 2,000 diverse client profiles. Extensive experimental analysis fully validates the superior quality and clinical fidelity of our dataset.Our datasets and evaluation framework are anonymously available at this repository.
Perception, Layout, and Validation: Calibrated Confidence for Reliable Straight-Through Processing of Financial Documents
Yichao Jin | Yushuo Wang | Han Yuxuan | Ching Yee Sonia Kwan | Weiyang Song | Kent Chiu Jin Chun | Chong Hwee Wong | Tiong Kiat Wong | Kenneth Zhu Ke | Jingyuan Zhao
The Proceedings of Workshop on Document Intelligence and Understanding
Yichao Jin | Yushuo Wang | Han Yuxuan | Ching Yee Sonia Kwan | Weiyang Song | Kent Chiu Jin Chun | Chong Hwee Wong | Tiong Kiat Wong | Kenneth Zhu Ke | Jingyuan Zhao
The Proceedings of Workshop on Document Intelligence and Understanding
Straight-through processing (STP) on extracted key-value fields from financial documents without human review requires a calibrated probability together with a bounded guarantee on the residual error of the auto-approved tier. The emergence of modern Vision Language Models (VLMs) provides an out-of-the-box capability for extracting the key-values, but their verbalized confidence signals are unreliable and weakly track field correctness. This paper introduces a decomposed confidence layer along three interpretable channels, including perception, layout, and validation. Together with a final conformal risk control, the score can be used for reliable STP of financial documents. The method is validated on three public datasets covering real invoices, synthetic invoices, and ad-buy forms, using two different VLM families (Qwen3.6-27B and Gemini-3.1-Flash-Lite). Our decomposed score consistently improves the separation of correct from incorrect extractions, substantially raising the AUROC from 0.54–0.74 for VLM verbalized signals to 0.90-0.99 with contributions from all three designed channels. Crucially for industrial deployment, this enables usable STP. The native VLM confidence signals could clear only 0.1%-7.0% of fields under risk control at a target error of ≤ 10%. In contrast, the proposed method auto-approves 49-72% of fields while holding the empirical error of the accepted tier at or below the target.