Takashi Ieda

Author directory

2026

This paper presents the results and findings of the 2nd Patent Claims Translation/Evaluation Task at WAT2026. Building on the first task, which collected human error annotations for machine-translated patent claims, this year we introduce an evaluation task: given the source claims and the machine translations gathered last year, participants automatically annotate translation errors (spans, categories, and severities) and assign a holistic quality score to each translation. We received submissions from three teams covering the Japanese–English (JA–EN) and English–Japanese (EN–JA) directions. We evaluated the submissions against the organizers’ reference annotations in terms of error-span detection, severity, and quality-score correlation and correlated the results with the design of each participating system. Our analysis highlights two observations. First, giving an LLM explicit word alignments mainly helps it detect errors that hinge on the source–target correspondence (mistranslation and terminology), while leaving target-only fluency errors unaffected. Second, locating errors and producing a good overall quality score behave as largely independent abilities: the system whose score correlated best with the reference detected the fewest errors, because it scored quality with a module separate from its error annotator.

2025

This paper presents the results and findings of the first shared task of translating patent claims. We provide training, development, and test data for participants and perform human evaluation of the submitted translations. This time, 2 teams submitted their translation results. Our analysis of the human-annotated translation errors revealed not only general, domain-independent errors but also errors specific to patent translation. We also found that the human annotation itself exhibited some serious issues. In this paper, we report on these findings.