Wasserstein Selective Transfer Learning for Cross-domain Text Mining

Lingyun Feng, Minghui Qiu, Yaliang Li, Haitao Zheng, Ying Shen


Abstract
Transfer learning (TL) seeks to improve the learning of a data-scarce target domain by using information from source domains. However, the source and target domains usually have different data distributions, which may lead to negative transfer. To alleviate this issue, we propose a Wasserstein Selective Transfer Learning (WSTL) method. Specifically, the proposed method considers a reinforced selector to select helpful data for transfer learning. We further use a Wasserstein-based discriminator to maximize the empirical distance between the selected source data and target data. The TL module is then trained to minimize the estimated Wasserstein distance in an adversarial manner and provides domain invariant features for the reinforced selector. We adopt an evaluation metric based on the performance of the TL module as delayed reward and a Wasserstein-based metric as immediate rewards to guide the reinforced selector learning. Compared with the competing TL approaches, the proposed method selects data samples that are closer to the target domain. It also provides better state features and reward signals that lead to better performance with faster convergence. Extensive experiments on three real-world text mining tasks demonstrate the effectiveness of the proposed method.
Anthology ID:
2021.emnlp-main.770
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9772–9783
Language:
URL:
https://aclanthology.org/2021.emnlp-main.770
DOI:
10.18653/v1/2021.emnlp-main.770
Bibkey:
Cite (ACL):
Lingyun Feng, Minghui Qiu, Yaliang Li, Haitao Zheng, and Ying Shen. 2021. Wasserstein Selective Transfer Learning for Cross-domain Text Mining. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9772–9783, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Wasserstein Selective Transfer Learning for Cross-domain Text Mining (Feng et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.770.pdf
Software:
 2021.emnlp-main.770.Software.zip
Video:
 https://aclanthology.org/2021.emnlp-main.770.mp4