Unsupervised Deep Cross-modality Spectral Hashing

Author(s)

• Tuan Hoang, Singapore University of Technology and Design
• Thanh-Toan Do, University of Liverpool
• Tam V. Nguyen, University of DaytonFollow
• Ngai-Man Cheung, Singapore University of Technology and Design

Document Type

Article

Publication Date

8-12-2020

Publication Source

IEEE Transactions on Image Processing

Abstract

This paper presents a novel framework, namely Deep Cross-modality Spectral Hashing (DCSH), to tackle the unsupervised learning problem of binary hash codes for efficient cross-modal retrieval. The framework is a two-step hashing approach which decouples the optimization into (1) binary optimization and (2) hashing function learning. In the first step, we propose a novel spectral embedding-based algorithm to simultaneously learn single-modality and binary cross-modality representations. While the former is capable of well preserving the local structure of each modality, the latter reveals the hidden patterns from all modalities. In the second step, to learn mapping functions from informative data inputs (images and word embeddings) to binary codes obtained from the first step, we leverage the powerful CNN for images and propose a CNN-based deep architecture to learn text modality. Quantitative evaluations on three standard benchmark datasets demonstrate that the proposed DCSH method consistently outperforms other state-of-the-art methods.

Inclusive pages

8391 - 8406

ISBN/ISSN

1057-714

Publisher

IEEE

Volume

29

Keywords

Cross-modal retrieval, spectral hashing, image search, constraint optimization.

eCommons Citation

Hoang, Tuan; Do, Thanh-Toan; Nguyen, Tam V.; and Cheung, Ngai-Man, "Unsupervised Deep Cross-modality Spectral Hashing" (2020). Computer Science Faculty Publications. 227.
https://ecommons.udayton.edu/cps_fac_pub/227