Multi-similarity Semantic Correctional Hashing For Cross Modal Retrieval

Given the benefits of their low storage requirements and high retrieval efficiency, hashing methods have attracted considerable attention for large scale cross-modal retrieval and significant progress has been made recently. However, the existing methods generally use the label-guided similarity matrix to measure the similarities of sample pairs, which limits their semantic representation capability. Moreover, the sample imbalance of different classes would bias the learning process toward majority classes and affect the retrieval performance. To boost the semantic representation, to alleviate the impact of data imbalance, and to obtain a high-ranking correlation of hash code pairs, we propose a novel hashing method that uses a semantic correctional similarity matrix to enhance the embedded representation of sample pairs. Furthermore, we propose a novel cross-modal multi-similarity loss based on the general pair weighting framework to collect and weight informative pairs efficiently and accurately, thus improving the retrieval performance. Our analysis and experimental results demonstrate that, compared with recent cross-modal retrieval methods, our methods achieve greater retrieval performance on two datasets MIRFlickr-25K and NUS-WIDE.