Rong Jin


2024

pdf bib
SumCSE: Summary as a transformation for Contrastive Learning
Raghuveer Thirukovalluru | Xiaolan Wang | Jun Chen | Shuyang Li | Jie Lei | Rong Jin | Bhuwan Dhingra
Findings of the Association for Computational Linguistics: NAACL 2024

Sentence embedding models are typically trained using contrastive learning (CL), either using human annotations directly or by repurposing other annotated datasets. In this work, we explore the recently introduced paradigm of generating CL data using generative language models (LM). In CL for computer vision (CV), compositional transformations (series of operations applied over an image. e.g. cropping + color distortion) which modify the input/image to retain minimal information were shown to be very effective. We show that composition of a ‘Summary’ transformation with diverse paraphrasing/contradicting transformations accomplishes the same and works very well in CL for sentence embeddings. Our final generated dataset (using Vicuna-13B) significantly outperforms the previous best unsupervised method (using ChatGPT) by 1.8 points, and SimCSE, a strong supervised baseline by 0.3 points on the semantic text similarity (STS) benchmark.

2022

pdf bib
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation
Xiangpeng Wei | Heng Yu | Yue Hu | Rongxiang Weng | Weihua Luo | Rong Jin
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

The principal task in supervised neural machine translation (NMT) is to learn to generate target sentences conditioned on the source inputs from a set of parallel sentence pairs, and thus produce a model capable of generalizing to unseen instances. However, it is commonly observed that the generalization performance of the model is highly influenced by the amount of parallel data used in training. Although data augmentation is widely used to enrich the training data, conventional methods with discrete manipulations fail to generate diverse and faithful training samples. In this paper, we present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT), which augments each training instance with an adjacency semantic region that could cover adequate variants of literal expression under the same meaning. We conduct extensive experiments on both rich-resource and low-resource settings involving various language pairs, including WMT14 English{German,French}, NIST ChineseEnglish and multiple low-resource IWSLT translation tasks. The provided empirical evidences show that CsaNMT sets a new level of performance among existing augmentation techniques, improving on the state-of-the-art by a large margin. The core codes are contained in Appendix E.

2007

pdf bib
Automated Vocabulary Acquisition and Interpretation in Multimodal Conversational Systems
Yi Liu | Joyce Chai | Rong Jin
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

2004

pdf bib
Discourse Structure for Context Question Answering
Joyce Y. Chai | Rong Jin
Proceedings of the Workshop on Pragmatics of Question Answering at HLT-NAACL 2004

2002

pdf bib
A New Probabilistic Model for Title Generation
Rong Jin | Alexander G. Hauptmann
COLING 2002: The 19th International Conference on Computational Linguistics

2001

pdf bib
Automatic Title Generation for Spoken Broadcast News
Rong Jin | Alexander G. Hauptmann
Proceedings of the First International Conference on Human Language Technology Research