Video Instance Search via Spatial Fusion of Visual Words and Object Proposals

Author(s)

• Vinh-Tiep Nguyen, University of Information Technology
• Duy Dinh Le, University of Information Technology
• Minh-Triet Tran, Vietnam National University
• Tam V. Nguyen, University of DaytonFollow
• Thanh Duc Ngo, University of Information Technology
• Shin’ichi Satoh, National Institute of Informatics
• Duc Anh Duong, University of Information Technology

Document Type

Article

Publication Date

4-29-2019

Publication Source

International Journal of Multimedia Information Retrieval

Abstract

Most popular systems for object instance search are based on the bag-of-visual-word model. The inherent weaknesses of this standard model such as quantization error, unstructured representation, burstiness phenomenon are to some extent solved. However, it has a serious problem of searching small objects on a database with cluttered background. In many situations, even the irrelevant objects which share the same texture or shape with a query object get higher score than relevant ones. To overcome this problem, we propose a novel fusion method to significantly boost the accuracy of instance search systems. Firstly, we use the state-of-the-art object detector with denser feature for finding object bounding box and similarity score. Secondly, to exploit the spatial relationship of each visual word with an object proposal, a detected area that might contain a query object, we define three categories of visual word pairs, i.e., discriminative, weak relevant, and context inferred ones. Finally, we propose a new re-ranking scheme with three weighting functions corresponding to the three categories of visual word pairs to compute the final similarity score between a query topic and a video shot. To illustrate the efficiency of the proposed method, we conduct experiments on datasets which have a wide variety of types of query objects. Experimental results on TRECVID Instance Search datasets (INS2013 and INS2014) show the superiority of our proposed method over the state-of-the-art approaches.

Inclusive pages

181–192

ISBN/ISSN

2192-662X

Publisher

Springer

Volume

8

Issue

3

Keywords

Video instance search, Bag-of-visual-word model, Object proposal, Spatial fusion

eCommons Citation

Nguyen, Vinh-Tiep; Le, Duy Dinh; Tran, Minh-Triet; Nguyen, Tam V.; Ngo, Thanh Duc; Satoh, Shin’ichi; and Duong, Duc Anh, "Video Instance Search via Spatial Fusion of Visual Words and Object Proposals" (2019). Computer Science Faculty Publications. 234.
https://ecommons.udayton.edu/cps_fac_pub/234