Xinyi Yang

Author directory

Papers on this page may belong to the following people: Xinyi Yang (Macau), Xinyi Yang, Xinyi Yang (Rutgers)


2026

Evaluating research in flexible electronics requires weighing evidence across multiple technical and manufacturing dimensions. However, current LLM-based approaches largely rely on unstructured text and generic evaluation criteria, limiting their ability to assess scientific work at the device level. We present FlexSciEval, a multi-agent framework that first turns each paper or research plan into an auditable structured record, then uses dimension-specific agents to score academic impact, fabrication realizability, and novelty. We also construct a 60-paper expert-annotated benchmark for validation and scale the framework to 4,158 papers. FlexSciEval reduces overall mean absolute error by 10.4% compared with direct LLM evaluation on the benchmark annotated by experts, although gains vary by model and dimension. These results indicate that grounding LLM evaluation in structured and dimension-specific evidence improves the reliability of automated scientific assessment in the field of flexible electronics, providing a practical basis for more reliable agent-based evaluation in experimental device research in the future.
Empowered by Large Language Models (LLMs), deep research systems integrate online information into research reports through multi-step web searching and information synthesis. Evaluating these reports presents a significant challenge due to their long context, high information density, and domain specificity, necessitating the use of AI evaluation as a substitute for prohibitively expensive human assessment. However, few studies have examined the effectiveness of existing reward models or LLMs in understanding and evaluating such research reports. To bridge this gap, we introduce DeepResearch RewardBench, the first benchmark dedicated to evaluating reward modeling for research reports. We collect diverse report candidates from multiple agents or tool-augmented LLMs. These candidates are scored using query-specific rubrics and judged LLMs, and we ensure data quality and difficulty through a combination of multi-round filtering and human verification. Our benchmark comprises preference datasets formatted as pairs, triplets, and quadruplets to test ranking capabilities across varying levels of difficulty. Extensive experiments demonstrate that while current reward models handle general coherence effectively, their performance significantly degrades when evaluating research reports. In contrast, generative LLMs acting as judges demonstrate comprehensive superiority in this domain, with comparative evaluation proving more effective than direct scoring methods. We anticipate that this benchmark will serve as a vital testbed for the evaluation and advancement of research agents.