Multi-scale image retrieval based on HSV and directional gradient features

Jiang Man; Zhang Haoxiang; Cheng Deqiang; Guo Lin; Kou Qiqi; Zhao Lei

doi:10.12086/oee.2021.210310

Article navigation > Opto-Electronic Engineering > 2021 Vol. 48 > No. 11 > 210310

Next Article Previous Article

Jiang M, Zhang H X, Cheng D Q, et al. Multi-scale image retrieval based on HSV and directional gradient features[J]. Opto-Electron Eng, 2021, 48(11): 210310. doi: 10.12086/oee.2021.210310

Citation:

Jiang M, Zhang H X, Cheng D Q, et al. Multi-scale image retrieval based on HSV and directional gradient features[J]. Opto-Electron Eng, 2021, 48(11): 210310. doi: 10.12086/oee.2021.210310

Multi-scale image retrieval based on HSV and directional gradient features

1.
School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China
2.
Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China
3.
School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu 221116, China

Fund Project: National Natural Science Foundation of China (51774281)

More Information

^*Corresponding author: Cheng Deqiang, E-mail: cdqcumt@126.com

Received Date 24 September 2021

Revised Date 05 November 2021

Published Date 30 November 2021

Abstract

Abstract

Aiming at the problems of poor robustness of rotation change, high feature dimension, and long retrieval time of existing color image retrieval algorithms, this paper proposed an innovative image retrieval method by fusing color features and improved directional gradient features. It proposed an improved directional gradient algorithm based on the principal curvatures (P-FHOG) by combining the geometric curvature information of the image surface into the FHOG descriptor from multiple scales. At the same time, the color information of the image was further fused to obtain the multi-scale image retrieval method based on the color features and the improved directional gradient features (CP-FHOG). The experiment was compared with the advanced image retrieval methods on the Corel-1000 and Coil-100 data sets, and the average accuracy rates of 85.89% and 93.38% were achieved, respectively. The results show that the proposed algorithm is more accurate and robust (in rotation change) than other algorithms.
- image retrieval /
- color information /
- directional gradient /
- multiple scales /
- features fusion

FullText(HTML)

References

[1]	Yan C G, Gong B, Wei Y X, et al. Deep multi-view enhancement hashing for image retrieval[J]. IEEE Trans Pattern Mach Intell, 2021, 43(4): 1445–1451. doi: 10.1109/TPAMI.2020.2975798 CrossRef Google Scholar
[2]	寇旗旗, 程德强, 于文洁, 等. 融合CLBP和局部几何特征的纹理目标分类[J]. 光电工程, 2019, 46(11): 180604. doi: 10.12086/oee.2019.180604 CrossRef Google Scholar Kou Q Q, Cheng D Q, Yu W J, et al. Texture target classification with CLBP and local geometric features[J]. Opto-Electron Eng, 2019, 46(11): 180604. doi: 10.12086/oee.2019.180604 CrossRef Google Scholar
[3]	刘芳, 吴志威, 杨安喆, 等. 基于多尺度特征融合的自适应无人机目标检测[J]. 光学学报, 2020, 40(10): 1015002. Google Scholar Liu F, Wu Z W, Yang A Z, et al. Multi-scale feature fusion based adaptive object detection for UAV[J]. Acta Opt Sin, 2020, 40(10): 1015002. Google Scholar
[4]	Celik C, Bilge H S. Content based image retrieval with sparse representations and local feature descriptors: a comparative study[J]. Pattern Recognit, 2017, 68: 1–13. doi: 10.1016/j.patcog.2017.03.006 CrossRef Google Scholar
[5]	Agarwal M, Maheshwari R P. HOG feature and vocabulary tree for content-based image retrieval[J]. Int J Signal Imaging Syst Eng, 2011, 3(4): 246–254. Google Scholar
[6]	Hu R, Barnard M, Collomosse J. Gradient field descriptor for sketch based retrieval and localization[C]//Proceedings of 2010 IEEE International Conference on Image Processing, Hong Kong, China, 2010: 1025–1028. Google Scholar
[7]	Joolee J B, Lee Y K. Video retrieval based on image queries using THOG for augmented reality environments[C]//Proceedings of 2018 IEEE International Conference on Big Data and Smart Computing, Shanghai, China, 2018: 557–560. Google Scholar
[8]	程德强, 张皓翔, 江曼, 等. 融合主曲率与颜色信息的彩色图像检索算法[J]. 计算机辅助设计与图形学学报, 2021, 33(2): 223–231. Google Scholar Cheng D Q, Zhang H X, Jiang M, et al. Color image retrieval method fusing principal curvature and color information[J]. J Comput Aided Des Comput Graph, 2021, 33(2): 223–231. Google Scholar
[9]	Pavithra L K, Sharmila T S. An efficient framework for image retrieval using color, texture and edge features[J]. Comput Elect Eng, 2018, 70: 580–593. doi: 10.1016/j.compeleceng.2017.08.030 CrossRef Google Scholar
[10]	Bella M I T, Vasuki A. An efficient image retrieval framework using fused information feature[J]. Comput Elect Eng, 2019, 75: 46–60. doi: 10.1016/j.compeleceng.2019.01.022 CrossRef Google Scholar
[11]	Garg M, Dhiman G. A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants[J]. Neural Comput Appl, 2020, 33(4): 1311–1328. doi: 10.1007/s00521-020-05017-z CrossRef Google Scholar
[12]	Danapur N, Dizaj S A A, Rostami V. An efficient image retrieval based on an integration of HSV, RLBP, and CENTRIST features using ensemble classifier learning[J]. Multimed Tools Appl, 2020, 79(33): 24463–24486. doi: 10.1007/s11042-020-09109-9 CrossRef Google Scholar
[13]	Khwildi R, Ouled Zaid A. HDR image retrieval by using color-based descriptor and tone mapping operator[J]. Vis Comput, 2020, 36(8): 1111–1126. doi: 10.1007/s00371-019-01719-1 CrossRef Google Scholar
[14]	Farid H, Simoncelli E P. Differentiation of discrete multidimensional signals[J]. IEEE Trans Image Process, 2004, 13(4): 496–508. doi: 10.1109/TIP.2004.823819 CrossRef Google Scholar
[15]	Felzenszwalb P F, Girshick R B, McAllester D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Trans Pattern Anal Mach Intell, 2010, 32(9): 1627–1645. doi: 10.1109/TPAMI.2009.167 CrossRef Google Scholar
[16]	Kou Q Q, Cheng D Q, Zhuang H D, et al. Cross-complementary local binary pattern for robust texture classification[J]. IEEE Signal Process Lett, 2018, 26(1): 129–133. Google Scholar
[17]	Zhang H X, Jiang M, Kou Q Q. Color image retrieval algorithm fusing color and principal curvatures information[J]. IEEE Access, 2020, 8: 184945–184954. doi: 10.1109/ACCESS.2020.3030056 CrossRef Google Scholar
[18]	Wang J Z, Li J, Wiederhold G. SIMPLIcity: semantics-sensitive integrated matching for picture libraries[J]. IEEE Trans Pattern Anal Mach Intell, 2001, 23(9): 947–963. doi: 10.1109/34.955109 CrossRef Google Scholar
[19]	Nene S A, Nayar S K, Murase H. Columbia object image library (COIL-100)[R]. New York: Columbia University, 1996. Google Scholar
[20]	Kavitha H, Sudhamani M V. Object Based Image Retrieval from Database Using Combined Features[C]//Proceedings of the 2014 Fifth International Conference on Signal and Image Processing. IEEE, Bangalore, INDIA, 2014: 161–165. Google Scholar
[21]	吕晨, 程德强, 寇旗旗, 等. 基于YOLOv3和ASMS的目标跟踪算法[J]. 光电工程, 2021, 48(2): 200175. doi: 10.12086/oee.2021.200175 CrossRef Google Scholar Lv C, Cheng D Q, Kou Q Q, et al. Target tracking algorithm based on YOLOv3 and ASMS[J]. Opto-Electron Eng, 2021, 48(2): 200175. doi: 10.12086/oee.2021.200175 CrossRef Google Scholar
[22]	Kundu M K, Chowdhury M, Bulo S R. A graph-based relevance feedback mechanism in content-based image retrieval[J]. Knowl-Based Syst, 2015, 73: 254–264. doi: 10.1016/j.knosys.2014.10.009 CrossRef Google Scholar
[23]	Dubey S R, Singh S K, Singh R K. Multichannel decoded local binary patterns for content-based image retrieval[J]. IEEE Trans Image Process, 2016, 25(9): 4018–4032. doi: 10.1109/TIP.2016.2577887 CrossRef Google Scholar
[24]	孙奇平. 基于深度学习的图像检索研究[J]. 景德镇学院学报, 2018, 33(3): 15–18. doi: 10.3969/j.issn.1008-8458.2018.03.009 CrossRef Google Scholar Sun Q P. Research on image retrieval based on deep learning[J]. Jingdezhen Compr Coll J, 2018, 33(3): 15–18. doi: 10.3969/j.issn.1008-8458.2018.03.009 CrossRef Google Scholar
[25]	Somnugpong S, Khiewwan K. Content-based image retrieval using a combination of color correlograms and edge direction histogram[C]//Proceedings of the 2016 13th International Joint Conference on Computer Science and Software Engineering, Khon Kaen, Thailand, 2016: 1–5. Google Scholar
[26]	Xiao Y, Wu J X, Yuan J S. mCENTRIST: a multi-channel feature generation mechanism for scene categorization[J]. IEEE Trans Image Process, 2014, 23(2): 823–836. doi: 10.1109/TIP.2013.2295756 CrossRef Google Scholar
[27]	Ahmed K T, Ummesafi S, Iqbal A. Content based image retrieval using image features information fusion[J]. Inf Fusion, 2019, 51: 76–99. doi: 10.1016/j.inffus.2018.11.004 CrossRef Google Scholar

Overview

Overview

Overview: With the rapid development of computer vision and digital media, image retrieval has been successfully applied to search engines, digital libraries, medical image management, and other fields. For current color image retrieval, the extraction of a single image feature is often too limited, and it is difficult to achieve the purpose of efficient and fast retrieval. Color feature and directional gradient feature are two important features of an image, which are widely used in the field of image retrieval. Color information represents the overall features of the image, and the directional gradient feature represents the partial features information of the image by extracting the texture information of the image. Aiming at the problems of poor rotation change robustness, high feature dimension, and long retrieval time in current retrieval methods, a color image retrieval method that combines color feature with improved directional gradient feature is proposed. First, the input color image is converted into a grayscale image through Gaussian space, and the surface geometric curvature information and texture information of the grayscale image are extracted and integrated into the FHOG descriptor, and the main curvature information is multi-sampled to construct a mixed sampling direction gradient feature (P-FHOG1, P-FHOG2, P-FHOG3) based on the main curvature, and the improved directional gradient feature (P-FHOGs) based on the main curvature is obtained by merging the features of three scales. At the same time, the image is converted from RGB color space to HSV color space and the color information of the image is extracted after quantization to construct the color feature histogram, and the color feature of the image is obtained. On this basis, the two features are merged to obtain an image retrieval method based on color feature and improved direction gradient feature (CP-FHOG). The experiment was compared with the advanced image retrieval methods on the Corel-1000 and Coil-100 data sets, and the average accuracy rates of 85.89% and 93.38% were achieved, respectively. On the Corel-1000 data set, the features extraction time and retrieval time of the algorithm in this paper are 0.067 s and 0.048 s, respectively, which are improved by 0.075 s and 1.06 s, respectively, compared with the second-performing algorithm. At the same time, ablation experiments were performed in the two data sets to verify the effectiveness of the fusion algorithm. The experimental results show that, compared with HSV and P-FHOGs algorithms, CP-FHOG extracts richer detailed features, has stronger rotation robustness, and significantly improves retrieval accuracy in datasets containing complex backgrounds and targets with different rotation angles. Besides, retrieval time and feature dimension have also been greatly improved. The color image retrieval method proposed in this paper introduces main curvature information and color information based on FHOG descriptors, combines the advantages of color feature and directional gradient feature, and extracts rich overall and detailed features. The experimental result proves that the retrieval accuracy of the method in this paper is higher and the method has rotation robustness.