Hyper-CL: Conditioning Sentence Representations with Hypernetworks

Young Yoo, Jii Cha, Changhyeon Kim, Taeuk Kim


Abstract
While the introduction of contrastive learning frameworks in sentence representation learning has significantly contributed to advancements in the field, it still remains unclear whether state-of-the-art sentence embeddings can capture the fine-grained semantics of sentences, particularly when conditioned on specific perspectives.In this paper, we introduce Hyper-CL, an efficient methodology that integrates hypernetworks with contrastive learning to compute conditioned sentence representations.In our proposed approach, the hypernetwork is responsible for transforming pre-computed condition embeddings into corresponding projection layers. This enables the same sentence embeddings to be projected differently according to various conditions.Evaluation on two representative conditioning benchmarks, namely conditional semantic text similarity and knowledge graph completion, demonstrates that Hyper-CL is effective in flexibly conditioning sentence representations, showcasing its computational efficiency at the same time.We also provide a comprehensive analysis of the inner workings of our approach, leading to a better interpretation of its mechanisms.
Anthology ID:
2024.acl-long.41
Volume:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
700–711
Language:
URL:
https://aclanthology.org/2024.acl-long.41
DOI:
Bibkey:
Cite (ACL):
Young Yoo, Jii Cha, Changhyeon Kim, and Taeuk Kim. 2024. Hyper-CL: Conditioning Sentence Representations with Hypernetworks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 700–711, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Hyper-CL: Conditioning Sentence Representations with Hypernetworks (Yoo et al., ACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.acl-long.41.pdf