Combining Hierachical VAEs with LLMs for clinically meaningful timeline summarisation in social media

Jiayu Song, Jenny Chim, Adam Tsakalidis, Julia Ive, Dana Atzil-Slonim, Maria Liakata


Abstract
We introduce a hybrid abstractive summarisation approach combining hierarchical VAEs with LLMs to produce clinically meaningful summaries from social media user timelines, appropriate for mental health monitoring. The summaries combine two different narrative points of view: (a) clinical insights in third person, generated by feeding into an LLM clinical expert-guided prompts, and importantly, (b) a temporally sensitive abstractive summary of the user’s timeline in first person, generated by a novel hierarchical variational autoencoder, TH-VAE. We assess the generated summaries via automatic evaluation against expert summaries and via human evaluation with clinical experts, showing that timeline summarisation by TH-VAE results in more factual and logically coherent summaries rich in clinical utility and superior to LLM-only approaches in capturing changes over time.
Anthology ID:
2024.findings-acl.873
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14651–14672
Language:
URL:
https://aclanthology.org/2024.findings-acl.873
DOI:
Bibkey:
Cite (ACL):
Jiayu Song, Jenny Chim, Adam Tsakalidis, Julia Ive, Dana Atzil-Slonim, and Maria Liakata. 2024. Combining Hierachical VAEs with LLMs for clinically meaningful timeline summarisation in social media. In Findings of the Association for Computational Linguistics ACL 2024, pages 14651–14672, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Combining Hierachical VAEs with LLMs for clinically meaningful timeline summarisation in social media (Song et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-acl.873.pdf