Industry vs Academia: Running a Course on Transformers in Two Setups

Irina Nikishina, Maria Tikhonova, Viktoriia Chekalina, Alexey Zaytsev, Artem Vazhentsev, Alexander Panchenko


Abstract
This paper presents a course on neural networks based on the Transformer architecture targeted at diverse groups of people from academia and industry with experience in Python, Machine Learning, and Deep Learning but little or no experience with Transformers. The course covers a comprehensive overview of the Transformers NLP applications and their use for other data types. The course features 15 sessions, each consisting of a lecture and a practical part, and two homework assignments organized as CodaLab competitions. The first six sessions of the course are devoted to the Transformer and the variations of this architecture (e.g., encoders, decoders, encoder-decoders) as well as different techniques of model tuning. Subsequent sessions are devoted to multilingualism, multimodality (e.g., texts and images), efficiency, event sequences, and tabular data.We ran the course for different audiences: academic students and people from industry. The first run was held in 2022. During the subsequent iterations until 2024, it was constantly updated and extended with recently emerged findings on GPT-4, LLMs, RLHF, etc. Overall, it has been ran six times (four times in industry and twice in academia) and received positive feedback from academic and industry students.
Anthology ID:
2024.teachingnlp-1.3
Volume:
Proceedings of the Sixth Workshop on Teaching NLP
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Sana Al-azzawi, Laura Biester, György Kovács, Ana Marasović, Leena Mathur, Margot Mieskes, Leonie Weissweiler
Venues:
TeachingNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7–22
Language:
URL:
https://aclanthology.org/2024.teachingnlp-1.3
DOI:
Bibkey:
Cite (ACL):
Irina Nikishina, Maria Tikhonova, Viktoriia Chekalina, Alexey Zaytsev, Artem Vazhentsev, and Alexander Panchenko. 2024. Industry vs Academia: Running a Course on Transformers in Two Setups. In Proceedings of the Sixth Workshop on Teaching NLP, pages 7–22, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Industry vs Academia: Running a Course on Transformers in Two Setups (Nikishina et al., TeachingNLP-WS 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.teachingnlp-1.3.pdf