Rahul Mehta
2024
Halu-NLP at SemEval-2024 Task 6: MetaCheckGPT - A Multi-task Hallucination Detection using LLM uncertainty and meta-models
Rahul Mehta
|
Andrew Hoblitzell
|
Jack O’keefe
|
Hyeju Jang
|
Vasudeva Varma
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Hallucinations in large language models(LLMs) have recently become a significantproblem. A recent effort in this directionis a shared task at Semeval 2024 Task 6,SHROOM, a Shared-task on Hallucinationsand Related Observable Overgeneration Mis-takes. This paper describes our winning so-lution ranked 1st and 2nd in the 2 sub-tasksof model agnostic and model aware tracks re-spectively. We propose a meta-regressor basedensemble of LLMs based on a random forestalgorithm that achieves the highest scores onthe leader board. We also experiment with var-ious transformer based models and black boxmethods like ChatGPT, Vectara, and others. Inaddition, we perform an error analysis com-paring ChatGPT against our best model whichshows the limitations of the former
2023
LLM-RM at SemEval-2023 Task 2: Multilingual Complex NER Using XLM-RoBERTa
Rahul Mehta
|
Vasudeva Varma
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Named Entity Recognition(NER) is a task ofrecognizing entities at a token level in a sen-tence. This paper focuses on solving NER tasksin a multilingual setting for complex named en-tities. Our team, LLM-RM participated in therecently organized SemEval 2023 task, Task 2:MultiCoNER II,Multilingual Complex NamedEntity Recognition. We approach the problemby leveraging cross-lingual representation pro-vided by fine-tuning XLM-Roberta base modelon datasets of all of the 12 languages provided - Bangla, Chinese, English, Farsi, French,German, Hindi, Italian, Portuguese, Spanish,Swedish and Ukrainian.
2004
Learning to Resolve Bridging References
Massimo Poesio
|
Rahul Mehta
|
Axel Maroudas
|
Janet Hitzeman
Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)
Search
Co-authors
- Vasudeva Varma 2
- Massimo Poesio 1
- Axel Maroudas 1
- Janet Hitzeman 1
- Andrew Hoblitzell 1
- show all...