Khangembam Alex D Nelson
Author directory2026
Aura 💫: A Self-Hostable Platform for Human Evaluation of NLG for Under-resourced Languages and Beyond
Rudali Huidrom | Khangembam Alex D Nelson | Nicholas Khumukcham | Hoomexsun Pangsatabam | Jesu Nazarene Roy L | Kanal Trivedi | Vikas Kumar | Pinaki Das | Kshetrimayum Boynao Singh | Tarun Bisht | Justice Konjengbam | Konjengbam Anand | Vikram Goyal
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations
Rudali Huidrom | Khangembam Alex D Nelson | Nicholas Khumukcham | Hoomexsun Pangsatabam | Jesu Nazarene Roy L | Kanal Trivedi | Vikas Kumar | Pinaki Das | Kshetrimayum Boynao Singh | Tarun Bisht | Justice Konjengbam | Konjengbam Anand | Vikram Goyal
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics: System Demonstrations
Human evaluation is the gold standard for Natural Language Generation (NLG), but the tooling is fragmented: most platforms serve a single task or a single protocol, and none treat the growing practice of assisting annotators with an LLM as a measurable object of study. These gaps fall hardest on indigenous and under-resourced languages, where paid crowds are scarce, references are unavailable, and data governance matters. We present Aura, an open, self-hostable platform that unifies five evaluation protocols (Likert/DA, pairwise, ranking, error-span, post-editing) across eight task families under one segment schema and one result model, records writing-system and speaker provenance as first-class fields, and runs entirely offline. Its distinguishing feature is a grounded, advisory-only, logged LLM assistant paired with a built-in randomised experiment that estimates the effect of AI assistance on inter-annotator agreement and time-on-task. Aura’s agreement statistics match reference implementations to machine precision. We conducted a human evaluation experiment on three English→Indian-language ranking campaigns (N=9 annotators) which returned a mean usability rating of 4.06/5. Aura is released as open-source software.