Humans Keep It One Hundred: an Overview of AI Journey

Tatiana Shavrina, Anton Emelyanov, Alena Fenogenova, Vadim Fomin, Vladislav Mikhailov, Andrey Evlampiev, Valentin Malykh, Vladimir Larin, Alex Natekin, Aleksandr Vatulin, Peter Romov, Daniil Anastasiev, Nikolai Zinov, Andrey Chertok


Abstract
Artificial General Intelligence (AGI) is showing growing performance in numerous applications - beating human performance in Chess and Go, using knowledge bases and text sources to answer questions (SQuAD) and even pass human examination (Aristo project). In this paper, we describe the results of AI Journey, a competition of AI-systems aimed to improve AI performance on knowledge bases, reasoning and text generation. Competing systems pass the final native language exam (in Russian), including versatile grammar tasks (test and open questions) and an essay, achieving a high score of 69%, with 68% being an average human result. During the competition, a baseline for the task and essay parts was proposed, and 80+ systems were submitted, showing different approaches to task understanding and reasoning. All the data and solutions can be found on github https://github.com/sberbank-ai/combined_solution_aij2019
Anthology ID:
2020.lrec-1.277
Volume:
Proceedings of the 12th Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
2276–2284
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.277
DOI:
Bibkey:
Cite (ACL):
Tatiana Shavrina, Anton Emelyanov, Alena Fenogenova, Vadim Fomin, Vladislav Mikhailov, Andrey Evlampiev, Valentin Malykh, Vladimir Larin, Alex Natekin, Aleksandr Vatulin, Peter Romov, Daniil Anastasiev, Nikolai Zinov, and Andrey Chertok. 2020. Humans Keep It One Hundred: an Overview of AI Journey. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2276–2284, Marseille, France. European Language Resources Association.
Cite (Informal):
Humans Keep It One Hundred: an Overview of AI Journey (Shavrina et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.277.pdf
Code
 sberbank-ai/combined_solution_aij2019