Antony D’Avirro
Author directory2026
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
Jeffrey M. Girard | Jason Zheng | Jacqueline Romero Vertino | Antony D’Avirro | Benjamin Peloquin
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Jeffrey M. Girard | Jason Zheng | Jacqueline Romero Vertino | Antony D’Avirro | Benjamin Peloquin
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of prompt, so only the manner of interaction can reveal the answer. Across text, audio, and video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models favor "stranger." This is a difference in effective prior, not in discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of speech. We release the stimuli, human ratings, and model predictions.
2024
EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models
Maureen de Seyssel | Antony D’Avirro | Adina Williams | Emmanuel Dupoux
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Maureen de Seyssel | Antony D’Avirro | Adina Williams | Emmanuel Dupoux
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
We introduce EmphAssess, a prosodic benchmark designed to evaluate the capability of speech-to-speech models to encode and reproduce prosodic emphasis. We apply this to two tasks: speech resynthesis and speech-to-speech translation. In both cases, the benchmark evaluates the ability of the model to encode emphasis in the speech input and accurately reproduce it in the output, potentially across a change of speaker and language. As part of the evaluation pipeline, we introduce EmphaClass, a new model that classifies emphasis at the frame or word level.