Vincenzo Deufemia
Author directory2026
Meta-Prompting Follow-Ups for Unsupervised Dialogue Evaluation Using Open-Source Large Language Models
Gaetano Cimino | Chuyuan Li | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Gaetano Cimino | Chuyuan Li | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Automatically evaluating dialogue quality remains a major challenge due to the complexity and contextual variability of human interactions. This paper introduces DIET, a novel unsupervised, reference-free metric that uses follow-up utterances to assess dialogue quality. Unlike existing reference-free metrics, which rely on follow-ups derived from annotated data and apply a uniform set of utterances across all dialogues, DIET generates follow-ups using open-source Large Language Models (LLMs) and refines them through a selection process. Two strategies are explored: SELFMAP, where generation and evaluation are performed by the same model to ensure internal coherence, and CRAFT, where multiple models collaborate to generate diverse and complementary follow-ups, enhancing robustness and reducing model bias. Dialogue quality is measured via the likelihood of an LLM continuing the dialogue from selected follow-ups. Experiments show DIET better correlates with human judgments than existing reference-free metrics across multiple meta-evaluation datasets.
MIMIC: Multi-party Dialogue Augmentation via Speaker Stylistic Transfer
Gaetano Cimino | Giuseppe Carenini | Vincenzo Deufemia
Findings of the Association for Computational Linguistics: EACL 2026
Gaetano Cimino | Giuseppe Carenini | Vincenzo Deufemia
Findings of the Association for Computational Linguistics: EACL 2026
Annotated data scarcity has long hindered progress in dialogue discourse parsing. To fill this gap, we introduce MIMIC, a framework for augmenting discourse-annotated corpora via speaker stylistic transfer using Large Language Models (LLMs). MIMIC rephrases utterances while preserving discourse coherence, using the MASK metric to identify speakers for replacement that enrich structural diversity and the MIRROR method to select substitute speakers who have experienced similar discourse interactions. Experimental results on STAC and Molweni corpora show that parsers trained with MIMIC-augmented data improve both link prediction and relation classification, with consistent gains for underrepresented discourse patterns and in low-resource scenarios.
CLARUS: Hierarchical Hypergraph Modeling for Unsupervised Discharge Summarization
Gaetano Cimino | Mir Rayat Imtiaz Hossain | Laura De Santis | Amirhossein Abaskohi | Raymond T. Ng | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Gaetano Cimino | Mir Rayat Imtiaz Hossain | Laura De Santis | Amirhossein Abaskohi | Raymond T. Ng | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Discharge summary generation from clinical records is challenging due to long, heterogeneous inputs and the need to preserve clinically relevant information. Although Large Language Models (LLMs) have improved abstractive summarization, their direct application to full patient records is limited by context length and difficulties in aggregating evidence across notes. We present CLARUS, a fully unsupervised framework that models patient records as hypergraphs encoding multi-dimensional relations. A hierarchical hypergraph encoder with depth-aware concept attention strengthens sparse representations, while a variational information bottleneck with Gumbel-Softmax relaxation enables differentiable selection of compact, informative subsets for LLM generation. Experiments on MIMIC-III show that CLARUS outperforms strong unsupervised baselines across lexical, semantic, and clinical metrics, demonstrating the value of higher-order structure and concept hierarchy for faithful multi-document clinical summarization.
MUDIPA: A Human-in-the-Loop AI-Assisted Platform for Discourse Graph Annotation in Multi-party Dialogues
Laura De Santis | Gaetano Cimino | Nils Feldhus | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Laura De Santis | Gaetano Cimino | Nils Feldhus | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Annotating discourse structure in multi-party dialogues is a cognitively demanding task that requires specialized expertise, yet existing annotation tools are either fully manual or provide only limited assistance. We present MUDIPA, a web-based, human-in-the-loop platform for discourse graph annotation in multi-party dialogues, built around two pluggable AI backends: a suggestion engine that produces confidence-ranked arc proposals, and an LLM-based reasoning engine that generates on-demand natural-language rationales for candidate attachments and relation labels. In a counterbalanced, within-subjects study with eight annotators, we compare AI-assisted annotation with MUDIPA against manual annotation. The AI-assisted condition achieves higher inter-annotator agreement on relation labels, higher accuracy against the gold standard, and lower perceived workload, while requiring only a modest increase in annotation time. These findings indicate that the principal value of LLMs in discourse annotation lies in supporting human decision-making, improving both annotation quality and inter-annotator agreement relative to unassisted annotation.
2024
Coherence-based Dialogue Discourse Structure Extraction using Open-Source Large Language Models
Gaetano Cimino | Chuyuan Li | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Gaetano Cimino | Chuyuan Li | Giuseppe Carenini | Vincenzo Deufemia
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Despite the challenges posed by data sparsity in discourse parsing for dialogues, unsupervised methods have been underexplored. Leveraging recent advances in Large Language Models (LLMs), in this paper we investigate an unsupervised coherence-based method to build discourse structures for multi-party dialogues using open-source LLMs fine-tuned on conversational data. Specifically, we propose two algorithms that extract dialogue structures by identifying their most coherent sub-dialogues: DS-DP employs a dynamic programming strategy, while DS-FLOW applies a greedy approach. Evaluation on the STAC corpus demonstrates a micro-F1 score of 58.1%, surpassing prior unsupervised methods. Furthermore, on a cleaned subset of the Molweni corpus, the proposed method achieves a micro-F1 score of 74.7%, highlighting its effectiveness across different corpora.
2003
Visual Language Editors based on LR Parsing Techniques
Gennaro Costagliola | Vincenzo Deufemia
Proceedings of the Eighth International Workshop on Parsing Technologies
Gennaro Costagliola | Vincenzo Deufemia
Proceedings of the Eighth International Workshop on Parsing Technologies
Visual language editors should provide a user-friendly environment where users are supported in an effective way in the construction of visual sentences. In this paper, we propose an approach for the construction of syntax-directed visual language editors by integrating incremental parsers into freehand editors. The approach combines the LR-based techniques for parsing visual languages with the more general incremental Generalized LR parsing techniques developed for string languages.