Fermi at SemEval-2019 Task 8: An elementary but effective approach to Question Discernment in Community QA Forums

Bakhtiyar Syed, Vijayasaradhi Indurthi, Manish Shrivastava, Manish Gupta, Vasudeva Varma


Abstract
Online Community Question Answering Forums (cQA) have gained massive popularity within recent years. The rise in users for such forums have led to the increase in the need for automated evaluation for question comprehension and fact evaluation of the answers provided by various participants in the forum. Our team, Fermi, participated in sub-task A of Task 8 at SemEval 2019 - which tackles the first problem in the pipeline of factual evaluation in cQA forums, i.e., deciding whether a posed question asks for a factual information, an opinion/advice or is just socializing. This information is highly useful in segregating factual questions from non-factual ones which highly helps in organizing the questions into useful categories and trims down the problem space for the next task in the pipeline for fact evaluation among the available answers. Our system uses the embeddings obtained from Universal Sentence Encoder combined with XGBoost for the classification sub-task A. We also evaluate other combinations of embeddings and off-the-shelf machine learning algorithms to demonstrate the efficacy of the various representations and their combinations. Our results across the evaluation test set gave an accuracy of 84% and received the first position in the final standings judged by the organizers.
Anthology ID:
S19-2203
Volume:
Proceedings of the 13th International Workshop on Semantic Evaluation
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota, USA
Editors:
Jonathan May, Ekaterina Shutova, Aurelie Herbelot, Xiaodan Zhu, Marianna Apidianaki, Saif M. Mohammad
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1160–1164
Language:
URL:
https://aclanthology.org/S19-2203
DOI:
10.18653/v1/S19-2203
Bibkey:
Cite (ACL):
Bakhtiyar Syed, Vijayasaradhi Indurthi, Manish Shrivastava, Manish Gupta, and Vasudeva Varma. 2019. Fermi at SemEval-2019 Task 8: An elementary but effective approach to Question Discernment in Community QA Forums. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1160–1164, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
Cite (Informal):
Fermi at SemEval-2019 Task 8: An elementary but effective approach to Question Discernment in Community QA Forums (Syed et al., SemEval 2019)
Copy Citation:
PDF:
https://aclanthology.org/S19-2203.pdf