Zhuoqun Ma


2024

pdf bib
Joint Similarity Guidance Hash Coding Based on Adaptive Weight Mixing Strategy For Cross-Modal Retrieval
Sun Yaqi | Yun Jing | Zhuoqun Ma
Proceedings of the 23rd Chinese National Conference on Computational Linguistics (Volume 1: Main Conference)

“There is a continuous and explosive growth of multimodal data. Efficient cross-modal hash-ing retrieval is of significant importance in conserving computational resources.To further en-hance the attention to informative data within modalities and capture the semantic correlationsin cross-modal data, we propose an enhanced deep Joint-Semantics Reconstructing Hashing al-gorithm, which is the Joint Similarity Guidance Hash Coding Based on Adaptive Weight MixingStrategy(JSGHCA). The algorithm focuses on delving deeper into the correlations of the data incross-modal. We introduce the adaptive weight mixing strategy to construct the semantic affinitymatrix, so that the matrix can identify each modal data with specific weight in each batch. Atthe same time, in the process of the hash code generation, we introduce collaborative attentionmechanism. It helps the model to pay more attention to the local information of each modality,thereby capturing the semantic features within each modality more accurately. Additionally, itenables the model to jointly process the attention across different modalities and extract sharedsemantic features more precisely. Experimental results show that the proposed model is signifi-cantly better than the deep joint semantic reconstruction hash algorithm on multiple benchmarkdatasets.”