Yingyi Fu
Author directory2026
UTSK26 at WAT2026 Patent Claims Translation/Evaluation Tasks
Yingyi Fu | Kosei Buma | Takehito Utsuro
Proceedings of the Thirteenth Workshop on Asian Translation (WAT 2026)
Yingyi Fu | Kosei Buma | Takehito Utsuro
Proceedings of the Thirteenth Workshop on Asian Translation (WAT 2026)
This paper describes the UTSK26 submission to the WAT2026 Patent Claims Translation/Evaluation Task for the Japanese-to-English (JA-EN) and English-to-Japanese (EN-JA) directions. The task requires participants to annotate translation errors in machine translation (MT) outputs of patent claims and to assign a quality score from 0 to 100 to each translation. For error annotation, we use GEMBA-Patent-Hybrid, which combines an LLM-as-a-Judge method for detecting error spans, categories, and severities using GPT-5.2 with another method for assigning categories and severities to encoder-detected error spans. For quality scoring, we use Patent-Rubric-Checklist, a rubric-based LLM-as-a-Judge method that instructs GPT-5.2 to answer patent-specific checklist items covering five evaluation dimensions and aggregates the answers into a score with predefined weights. On the training data, which consist of the human evaluation results of the WAT2025 task, Patent-Rubric-Checklist achieves the highest Pearson and Spearman correlations with human scores in both directions among the evaluated methods, including COMETKiwi, MetricX-24, GEMBA-DA, and GEMBA-MQM.
Towards a Rubric-based LLM-as-a-Judge for Patent Claim Translation Evaluation
Yingyi Fu | Takehito Utsuro | Masaaki Nagata
Proceedings of the Thirteenth Workshop on Asian Translation (WAT 2026)
Yingyi Fu | Takehito Utsuro | Masaaki Nagata
Proceedings of the Thirteenth Workshop on Asian Translation (WAT 2026)
Existing automatic machine translation evaluation metrics, such as COMETKiwi and MetricX, showed relatively low correlations with human evaluation in the WAT2025 Patent Claim Translation/Evaluation Tasks. We propose a rubric-based LLM-as-a-Judge method for patent claim translation evaluation. Based on a patent-specific rubric consisting of five evaluation dimensions with corresponding checklist items, we instruct a GPT model to assess translation quality. On a publicly available human evaluation dataset derived from the WAT2025 Patent Claims Translation/Evaluation Tasks test set, the proposed method achieves the highest Pearson and Spearman correlation coefficients with human evaluation among all evaluated methods.
2025
NTTSU at WMT2025 General Translation Task
Zhang Yin | Hiroyuki Deguchi | Haruto Azami | Guanyu Ouyang | Kosei Buma | Yingyi Fu | Katsuki Chousa | Takehito Utsuro
Proceedings of the Tenth Conference on Machine Translation
Zhang Yin | Hiroyuki Deguchi | Haruto Azami | Guanyu Ouyang | Kosei Buma | Yingyi Fu | Katsuki Chousa | Takehito Utsuro
Proceedings of the Tenth Conference on Machine Translation
This paper presents the submission of NTTSU for the constrained track of the English–Japanese and Japanese–Chinese at the WMT2025 general translation task.For each translation direction, we build translation models from a large language model by combining continual pretraining, supervised fine-tuning, and preference optimization based on the translation quality and adequacy.We finally generate translations via context-aware MBR decoding to maximize translation quality and document-level consistency.
Search