Logical anomalies in industrial inspection are rule violations: an object count is wrong, an item is misplaced, a component type does not match, or a required relation is broken. Existing anomaly understanding systems usually stop at diagnosis, answering whether an image is anomalous, where the anomaly is, or why it violates a rule. Reworkable inspection workflows require the next question: What should be fixed? We introduce an industry-oriented benchmark and training pipeline for rule-grounded corrective actions. The task asks a model to produce an operator-facing action that supports one of three downstream decisions: no action, rework, or reject/manual review. This setting exposes a Diagnosis-Action Gap: identifying an abnormal object does not imply knowing the valid, minimal action that restores normality. We construct from MVTec LOCO and from VID-AD, adding correction-oriented annotations that connect normal rules, visual evidence, repairability, corrective actions, and verification targets. The datasets support multiple-choice correction selection, structured JSON action prediction, and open-ended correction generation. We further train ReworkVLM, a Qwen3-VL-based model optimized with , which combines semantic correctness, trace-answer alignment, and minimal-edit rewards. Experiments on both benchmarks show that correction-oriented evaluation reveals failures hidden by diagnosis alone, including wrong action direction, unnecessary repair, and non-minimal edits, while ReworkVLM improves structured corrective success and open-ended correction quality over general and anomaly-specific VLM baselines.

This study evaluated automatic prompt engineering (APE) using one assignment in the PERSUADE 2.0 dataset. The APE approach achieved higher QWK (.812) than the research-informed, zero-shot baseline prompting approach (.646). Descriptive comparisons examined gender and English language learner subgroups. Findings support the benefits of APE for automated essay scoring (AES).

This study examined whether rubric-aligned generative-AI features could augment established linguistic features in trait-based automated essay scoring. Features from both sources showed meaningful associations with human scores and only partial overlap with one another. Scoring models combining both feature sets produced modest improvements that varied across traits and evaluation metrics.

Prompting functions as a measurement condition that alters LLM coding behavior. We examined whether few-shot prompting improves GPT-based qualitative coding in systematic reviews. Contrary to expectations, we found that the zero-shot approach produced the highest agreement with human coders, suggesting increasing conservatism as more prompt structure was added.

Ensuring the trustworthiness of Generative Foundation Models (GenFMs) is a pressing challenge as they gain widespread use. Existing evaluation toolkits are often limited in scope, dynamism, and flexibility. This paper introduces TRUSTEVAL, a dynamic and comprehensive toolkit designed for evaluating GenFMs across various dimensions. TRUSTEVAL supports both dynamic dataset generation and evaluation, offering advanced features including comprehensiveness, usability, and flexibility. TRUSTEVAL integrates diverse generative models, datasets, evaluation methods, metrics, inference efficiency enhancement, and evaluation report generation. Through case studies, we demonstrate TRUSTEVAL’s potential to advance the trustworthiness evaluation of GenFMs.

This study evaluates large language models (LLMs) for automated essay scoring (AES), comparing prompt strategies and fairness across student groups. We found that well-designed prompting helps LLMs approach traditional AES performance, but both differ from human scores for ELLs—the traditional model shows larger overrall gaps, while LLMs show subtler disparities.

This study examines reliability and comparability of Generative AI scores versus human ratings on two performance tasks—text-based and drawing-based—in a fourth-grade visual arts assessment. Results show GPT-4 is consistent, aligned with humans but more lenient, and its agreement with humans is slightly lower than that between human raters.

With the rapid development and widespread application of Large Language Models (LLMs), the use of Machine-Generated Text (MGT) has become increasingly common, bringing with it potential risks, especially in terms of quality and integrity in fields like news, education, and science. Current research mainly focuses on purely MGT detection, without adequately addressing mixed scenarios including AI-revised Human-Written Text (HWT) or human-revised MGT. To tackle this challenge, we define mixtext, a form of mixed text involving both AI and human-generated content. Then we introduce MixSet, the first dataset dedicated to studying these mixtext scenarios. Leveraging MixSet, we executed comprehensive experiments to assess the efficacy of prevalent MGT detectors in handling mixtext situations, evaluating their performance in terms of effectiveness, robustness, and generalization. Our findings reveal that existing detectors struggle to identify mixtext, particularly in dealing with subtle modifications and style adaptability. This research underscores the urgent need for more fine-grain detectors tailored for mixtext, offering valuable insights for future research. Code and Models are available at https://github.com/Dongping-Chen/MixSet.

Alignment has become a critical step for instruction-tuned Large Language Models (LLMs) to become helpful assistants. However, effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. To fill in this gap, we introduce AlignBench, a comprehensive multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese. We tailor a human-in-the-loop data curation pipeline, containing 8 main categories, 683 real-scenario rooted queries and corresponding human verified references.To ensure references’ correctness, each knowledge-intensive query is accompanied with evidences collected from reliable webpages (including the url and quotation) by our annotators.For automatic evaluation, our benchmark employs a rule-calibrated multi-dimensional LLM-as-Judge (CITATION) with Chain-of-Thought to generate explanations and final ratings as evaluations, ensuring high reliability and interpretability.All evaluation codes and data are publicly available at https://github.com/THUDM/AlignBench