Xinyi Yang
Author directoryPapers on this page may belong to the following people: Xinyi Yang (Macau), Xinyi Yang, Xinyi Yang (Rutgers)
2026
FlexSciEval: Structure-Grounded Multi-agent Evaluation for Flexible Electronics Research
Ke Li | Xinyi Yang | Runzhe Zhan | Qianyong Hu | Derek F. Wong | Qingsong Li | Zhiyuan Liu
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Ke Li | Xinyi Yang | Runzhe Zhan | Qianyong Hu | Derek F. Wong | Qingsong Li | Zhiyuan Liu
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Evaluating research in flexible electronics requires weighing evidence across multiple technical and manufacturing dimensions. However, current LLM-based approaches largely rely on unstructured text and generic evaluation criteria, limiting their ability to assess scientific work at the device level. We present FlexSciEval, a multi-agent framework that first turns each paper or research plan into an auditable structured record, then uses dimension-specific agents to score academic impact, fabrication realizability, and novelty. We also construct a 60-paper expert-annotated benchmark for validation and scale the framework to 4,158 papers. FlexSciEval reduces overall mean absolute error by 10.4% compared with direct LLM evaluation on the benchmark annotated by experts, although gains vary by model and dimension. These results indicate that grounding LLM evaluation in structured and dimension-specific evidence improves the reliability of automated scientific assessment in the field of flexible electronics, providing a practical basis for more reliable agent-based evaluation in experimental device research in the future.
DeepResearch RewardBench: Evaluating Reward Models for Deep Research Report
Shudong Liu | Yuting Zhong | Guanhua Chen | Xinyi Yang | Junnan Liu | Lidia S. Chao | Jiaheng Liu | Derek F. Wong
Findings of the Association for Computational Linguistics: EMNLP 2026
Shudong Liu | Yuting Zhong | Guanhua Chen | Xinyi Yang | Junnan Liu | Lidia S. Chao | Jiaheng Liu | Derek F. Wong
Findings of the Association for Computational Linguistics: EMNLP 2026
Empowered by Large Language Models (LLMs), deep research systems integrate online information into research reports through multi-step web searching and information synthesis. Evaluating these reports presents a significant challenge due to their long context, high information density, and domain specificity, necessitating the use of AI evaluation as a substitute for prohibitively expensive human assessment. However, few studies have examined the effectiveness of existing reward models or LLMs in understanding and evaluating such research reports. To bridge this gap, we introduce DeepResearch RewardBench, the first benchmark dedicated to evaluating reward modeling for research reports. We collect diverse report candidates from multiple agents or tool-augmented LLMs. These candidates are scored using query-specific rubrics and judged LLMs, and we ensure data quality and difficulty through a combination of multi-round filtering and human verification. Our benchmark comprises preference datasets formatted as pairs, triplets, and quadruplets to test ranking capabilities across varying levels of difficulty. Extensive experiments demonstrate that while current reward models handle general coherence effectively, their performance significantly degrades when evaluating research reports. In contrast, generative LLMs acting as judges demonstrate comprehensive superiority in this domain, with comparative evaluation proving more effective than direct scoring methods. We anticipate that this benchmark will serve as a vital testbed for the evaluation and advancement of research agents.