Fréchet Gesture Distance (FGD) is the standard automatic metric for co-speech gesture generation, the models that synthesize the hand and body motion accompanying speech. Human perceptual studies remain the gold standard, and the GENEA challenges rank systems by them, but such studies are slow and costly, so papers routinely compare systems by FGD instead. FGD is a Fréchet distance in the latent space of a learned pose autoencoder, so it inherits the many choices behind that encoder, most of which papers never report. We ask whether FGD is a reliable proxy for perceptual evaluation under these unreported choices. Holding the systems fixed, we vary the reference tool’s own settings (analysis window, seed, feature versus raw space) and the corpus the feature extractor is trained on, on both tiers of the GENEA 2022 evaluation and on the GENEA 2023 challenge. First, the FGD ranking is not reproducible where it matters: the "best" system is one of two that a per-clip bootstrap shows are statistically tied, so which one wins is a configuration choice, though aggregate rank correlation stays moderate rather than collapsing. Second, and more consequential, FGD’s agreement with human ratings is modest and itself tunable, moving by 0.18 to 0.24 Kendall τ across these undocumented choices: FGD can be made to look appreciably more or less human-valid without changing the systems. These effects replicate across two challenge years and three evaluation tiers. We conclude that FGD numbers are not comparable across papers unless the window, seed, scoring space, and feature-extractor training corpus are pinned and reported, and that FGD cannot resolve small differences between closely performing systems.

Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score.

The use of emojis affords a visual modality to, often private, textual communication.The task of predicting emojis however provides a challenge for machine learning as emoji use tends to cluster into the frequently used and the rarely used emojis.Much of the machine learning research on emoji use has focused on high resource languages and has conceptualised the task of predicting emojis around traditional server-side machine learning approaches.However, traditional machine learning approaches for private communication can introduce privacy concerns, as these approaches require all data to be transmitted to a central storage.In this paper, we seek to address the dual concerns of emphasising high resource languages for emoji prediction and risking the privacy of people’s data.We introduce a new dataset of 118k tweets (augmented from 25k unique tweets) for emoji prediction in Hindi, and propose a modification to the federated learning algorithm, CausalFedGSD, which aims to strike a balance between model performance and user privacy. We show that our approach obtains comparative scores with more complex centralised models while reducing the amount of data required to optimise the models and minimising risks to user privacy.

The task of sentiment analysis has been extensively studied in high-resource languages. Even though sentiment analysis is studied for some resource-constrained languages, the corpora and the datasets available in other low resource languages are scarce and fragmented. This prevents further research of resource-constrained languages and also inhibits model performance for these languages. Privacy concerns may also be raised while aggregating some datasets for training central models. Our work tries to steer the research of sentiment analysis for resource-constrained languages in the direction of Federated Learning. We conduct various experiments to compare server based and federated approaches for 4 Indic Languages - Marathi, Hindi, Bengali, and Telugu. Specifically, we show that a privacy preserving approach, Federated Learning surpasses traditional server trained LSTM model and exhibits comparable performance to other servers-side transformer models.