AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP

Ramit Sawhney, Shrey Pandit, Vishwa Shah, Megh Thakkar, Shafiq Joty


Abstract
The Euclidean space is the familiar space for training neural models and performing arithmetic operations.However, many data types inherently possess complex geometries, and model training methods involve operating over their latent representations, which cannot be effectively captured in the Euclidean space.The hyperbolic space provides a more generalized representative geometry to model the hierarchical complexities of the tree-like structure of natural language.We propose AdaPT a set of guidelines for initialization, parametrization, and training of neural networks, which adapts to the dataset and can be used with different manifolds. AdaPT can be generalized over any existing neural network training methodology and leads to more stable training without a substantial increase in training time.We apply AdaPT guidelines over two state-of-the-art deep learning approaches and empirically demonstrate its effectiveness through experiments on three tasks over 12 languages across speech and text.Through extensive qualitative analysis, we put forward the applicability of AdaPT as a set of guidelines optimally utilizing the manifold geometry, which can be extended to various downstream tasks across languages and modalities.
Anthology ID:
2024.findings-naacl.114
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1757–1771
Language:
URL:
https://aclanthology.org/2024.findings-naacl.114
DOI:
Bibkey:
Cite (ACL):
Ramit Sawhney, Shrey Pandit, Vishwa Shah, Megh Thakkar, and Shafiq Joty. 2024. AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1757–1771, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP (Sawhney et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.114.pdf
Copyright:
 2024.findings-naacl.114.copyright.pdf