Benjamin Peloquin
Author directory2026
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
Jeffrey M. Girard | Jason Zheng | Jacqueline Romero Vertino | Antony D’Avirro | Benjamin Peloquin
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Jeffrey M. Girard | Jason Zheng | Jacqueline Romero Vertino | Antony D’Avirro | Benjamin Peloquin
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of prompt, so only the manner of interaction can reveal the answer. Across text, audio, and video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models favor "stranger." This is a difference in effective prior, not in discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of speech. We release the stimuli, human ratings, and model predictions.
2024
Textless Acoustic Model with Self-Supervised Distillation for Noise-Robust Expressive Speech-to-Speech Translation
Min-Jae Hwang | Ilia Kulikov | Benjamin Peloquin | Hongyu Gong | Peng-Jen Chen | Ann Lee
Findings of the Association for Computational Linguistics: ACL 2024
Min-Jae Hwang | Ilia Kulikov | Benjamin Peloquin | Hongyu Gong | Peng-Jen Chen | Ann Lee
Findings of the Association for Computational Linguistics: ACL 2024
In this paper, we propose a textless acoustic model with a self-supervised distillation strategy for noise-robust expressive speech-to-speech translation (S2ST).Recently proposed expressive S2ST systems have achieved impressive expressivity preservation performances by cascading unit-to-speech (U2S) generator to the speech-to-unit translation model. However, these systems are vulnerable to the presence of noise in input speech, which is an assumption in real-world translation scenarios. To address this limitation, we propose a U2S generator that incorporates a distillation with no label (DINO) self-supervised training strategy into it’s pretraining process.Because the proposed method captures noise-agnostic expressivity representation, it can generate qualified speech even in noisy environment.Objective and subjective evaluation results verified that the proposed method significantly improved the performance of the expressive S2ST system in noisy environments while maintaining competitive performance in clean environments.