Wuttikrai Lertprasertphakorn
Author directory2026
Beyond the Single-Expert Ceiling: Localizing LLM-as-Judge Stability on Free-Form Legal Essays
Pawitsapak Akarajaradwong | Wuttikrai Lertprasertphakorn | Chompakorn Chaksangchaichot | Sarana Nutanong
Proceedings of the Natural Legal Language Processing Workshop 2026
Pawitsapak Akarajaradwong | Wuttikrai Lertprasertphakorn | Chompakorn Chaksangchaichot | Sarana Nutanong
Proceedings of the Natural Legal Language Processing Workshop 2026
Free-form legal essay evaluation in NLP treats expert inter-rater stability as a single ceiling number: a correlation score between credentialed examiners is reported and used as the target that an LLM judge should approach. Whether that ceiling is uniform across items has rarely been tested on the expert side. We test it on Thai bar exam essays. Three Bar Council examiners and three anchor LLM judges (Claude 4.6 Opus, Gemini 3.1 Pro, GPT-5.4 Pro) independently graded the same 15 answers from the same four inputs: question, official rubric, gold answer, and student answer. On 10 of the 15 answers, both panels reach 𝛼 = 0.75. On the remaining 5, drawn from one question where the rubric does not prescribe how to grade a correct answer that omits a decisive citation, the human panel collapses to 𝛼 = -0.34 while the LLM panel holds at 𝛼 = 0.68. One examiner’s stricter reading of this underspecified rubric drives the gap. The cluster pattern replicates across seven of eight LLMs that took the exam in Phase 1 and were reused as judges, regardless of their own exam scores: the same model behaves differently in the two roles. The expert stability ceiling is not flat: free-form essay rubrics combine a deterministic correctness element (grounded by the gold answer) with an interpretive quality element (judged without one), and expert disagreement lives entirely on the interpretive element. The bottleneck of free-form essay evaluation is the rubric’s interpretive coverage, not LLM capability.
What the Public Asked to Change, and What It Would Touch? Complaint-to-Statute Grounding for Thailand’s Regulatory Reform
Pawitsapak Akarajaradwong | Wuttikrai Lertprasertphakorn | Chompakorn Chaksangchaichot | Sarana Nutanong
Proceedings of the Natural Legal Language Processing Workshop 2026
Pawitsapak Akarajaradwong | Wuttikrai Lertprasertphakorn | Chompakorn Chaksangchaichot | Sarana Nutanong
Proceedings of the Natural Legal Language Processing Workshop 2026
When a government reviews its laws, the public speaks in everyday words and rarely names a provision directly, so turning complaints into amendment targets is still expert work done by hand. We describe a pipeline deployed inside Thailand’s 2026 Regulatory Guillotine that maps 199,771 social media posts and 63 proposals from business associations to the related specific provisions. Our pipeline first removes irrelevant posts. A legal language model then labels each post with a written rationale, and the posts’ embeddings are clustered into 309 issues. The model grounds each issue in specific provisions using a Thai legal knowledge graph, which it must read before it cites, leaving 799 verified sections in 80 laws. The same graph then shows what an amendment would touch, namely the other sections, definitions, and case law that depend on each targeted section, a check that drafters currently do by hand. We report the system, what the Thai public asked to change, and the blast radius of the sections they targeted.