Soëlie Lerch


2024

We propose EMOLIS Dataset that contains annotated emotional transcripts of scenes from Walt Disney cartoons at the same time as physiological signals from spectators (breathing, ECG, eye movements). The dataset is used in EMOLIS App, our second proposal. EMOLIS App allows to display the identified emotions while a video is playing and suggest emotionally comparable videos. We propose to estimate an emotional distance between videos using multimodal neural representations (text, audio, video) that also combine physiological signals. This enables personalized results that can be used for cognitive therapies focusing on awareness of felt emotions. The dataset is designed to be suitable for all audiences and autistic people who have difficulties to recognize and express emotions.