Overview of the Second Shared Task on Automatic Minuting (AutoMin) at INLG 2023

Tirthankar Ghosal, Ondřej Bojar, Marie Hledíková, Tom Kocmi, Anna Nedoluzhko


Abstract
In this article, we report the findings of the second shared task on Automatic Minuting (AutoMin) held as a Generation Challenge at the 16th International Natural Language Generation (INLG) Conference 2023. The second Automatic Minuting shared task is a successor to the first AutoMin which took place in 2021. The primary objective of the AutoMin shared task is to garner participation of the speech and natural language processing and generation community to create automatic methods for generating minutes from multi-party meetings. Five teams from diverse backgrounds participated in the shared task this year. A lot has changed in the Generative AI landscape since the last AutoMin especially with the emergence and wide adoption of Large Language Models (LLMs) to different downstream tasks. Most of the contributions are based on some form of an LLM and we are also adding current outputs of GPT4 as a benchmark. Furthermore, we examine the applicability of GPT-4 for automatic scoring of minutes. Compared to the previous instance of AutoMin, we also add another domain, the minutes for EU Parliament sessions, and we experiment with a more fine-grained manual evaluation. More details on the event can be found at https://ufal.github.io/automin-2023/.
Anthology ID:
2023.inlg-genchal.19
Volume:
Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges
Month:
September
Year:
2023
Address:
Prague, Czechia
Editor:
Simon Mille
Venues:
INLG | SIGDIAL
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
138–167
Language:
URL:
https://aclanthology.org/2023.inlg-genchal.19
DOI:
Bibkey:
Cite (ACL):
Tirthankar Ghosal, Ondřej Bojar, Marie Hledíková, Tom Kocmi, and Anna Nedoluzhko. 2023. Overview of the Second Shared Task on Automatic Minuting (AutoMin) at INLG 2023. In Proceedings of the 16th International Natural Language Generation Conference: Generation Challenges, pages 138–167, Prague, Czechia. Association for Computational Linguistics.
Cite (Informal):
Overview of the Second Shared Task on Automatic Minuting (AutoMin) at INLG 2023 (Ghosal et al., INLG-SIGDIAL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.inlg-genchal.19.pdf