Citation: | Han Ze, Lin Suzhen. Multiband fusion image evaluation method based on correlation between subject and object evaluation[J]. Opto-Electronic Engineering, 2017, 44(9): 895-902. doi: 10.3969/j.issn.1003-501X.2017.09.006 |
[1] | Li Shutao, Kang Xudong, Fang Leyuan, et al. Pixel-level image fusion: a survey of the state of the art [J]. Information Fusion, 2017, 33: 100–112. doi: 10.1016/j.inffus.2016.05.004 |
[2] | Ding Li, Huang Hua, Zang Yu. Image quality assessment using directional anisotropy structure measurement [J]. IEEE Transactions on Image Processing, 2017, 26(4): 1799–1809. doi: 10.1109/TIP.2017.2665972 |
[3] | Krasula L, Le Callet P, Fliegel K, et al. Quality assessment of sharpened images: challenges, methodology, and objective metrics [J]. IEEE Transactions on Image Processing, 2017, 26(3): 1496–1508. doi: 10.1109/TIP.2017.2651374 |
[4] | Vega M T, Mocanu D C, Stavrou S, et al. Predictive no-reference assessment of video quality [J]. Signal Processing: Image Communication, 2017, 52: 20–32. doi: 10.1016/j.image.2016.12.001 |
[5] | Alaql O Ghazinour K, Lu Cheng Chang. Classification of image distortions for image quality assessment [C]// Proceedings of International Conference on Computational Science and Computational Intelligence. 2016: 653–658. |
[6] | 颜文, 龚飞, 周颖, 等.基于NSST与自适应PCNN相结合的卫星云图融合[J].光电工程, 2016, 43(10): 70–76, 83. doi: 10.3969/j.issn.1003-501X.2016.10.012 Yan Wen, Gong Fei, Zhou Ying, et al. Satellite cloud image fusion based on adaptive PCNN and NSST [J]. Opto-Electronic Engineering, 2016, 43(10): 70–76, 83. doi: 10.3969/j.issn.1003-501X.2016.10.012 |
[7] | 殷明, 段普宏, 褚标, 等.结合SIST和压缩感知的CT与MRI图像融合[J].光电工程, 2016, 43(8): 47–52. Yin Ming, Duan Puhong, Chu Biao, et al. CT and MRI medical image fusion based on shift-invariant shearlet transform and compressed sensing [J]. Opto–Electronic Engineering, 2016, 43(8): 47–52. |
[8] | 张学典, 汪泓, 江旻珊, 等.显著性分析在对焦图像融合方面的应用[J].光电工程, 2017, 44(4): 435–441. Zhang Xuedian, Wang Hong, Jiang Minshan, et al. Applications of saliency analysis in focus image fusion [J]. Opto-Electronic Engineering, 2017, 44(4): 435–441. |
[9] | Liu Yu, Chen Xun, Peng Hu, et al. Multi-focus image fusion with a deep convolutional neural network [J]. Information Fusion, 2017, 36: 191–207. doi: 10.1016/j.inffus.2016.12.001 |
[10] | Zhang Kai, Wang Min, Yang Shuyuan. Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(3): 1363–1371. doi: 10.1109/TGRS.2016.2623626 |
[11] | Wang Zhou, Bovik A C. A universal image quality index [J]. IEEE Signal Processing Letters, 2002, 9(3): 81–84 doi: 10.1109/97.995823 |
[12] |
He Guiqing, LiangFan, Xing Siyuan, et al. Study on algorithm evaluation of image fusion based on multi-hierarchical synthetic analysis[C]// Proceedings of 2016 IEEE International Conference on Signal Processing, Communications and Computing, 2016: 1–6. |
[13] | 朱亚辉. 红外与可见光图像融合质量评价方法研究[D]. 西安: 西北工业大学, 2015. Zhu Yahui. Research on quality evaluation methods of infrared and visible image fusion[D]. Xi'an: Northwestern Polytechnical University, 2015. |
[14] | Xydeas C S, Petrovic V. Objective image fusion performance measure[J]. Electronics Letters, 2000, 36(4): 308–309. doi: 10.1049/el:20000267 |
[15] | Piella G, Heijmans H. A new quality metric for image fusion [C]// Proceedings of 2003 International Conference on Image Pro-cessing, 2003, 2: Ⅲ-173–176. |
[16] |
Nizami I F, Majid M, Khurshid K. Efficient feature selection for blind image quality assessment based on natural scene statistics [C]// Proceedings of 2017 14th International Bhurban Conference on Applied Sciences and Technology, 2017: 318–322. |
[17] | Ding Yong, Zhao Yang, Zhao Xinyu. Image quality assessment based on multi-feature extraction and synthesis with support vector regression [J]. Signal Processing: Image Communication, 2017, 54: 81–92. doi: 10.1016/j.image.2017.03.001 |
[18] | Mukherjee R, Debattista K, Bashford-Rogers T, et al. Objective and subjective evaluation of high dynamic range video com-pression [J]. Signal Processing: Image Communication, 2016, 47: 426–437. doi: 10.1016/j.image.2016.08.001 |
[19] | Liu Yu, Liu Shuping, Wang Zengfu. A general framework for image fusion based on multi-scale transform and sparse representation [J]. Information Fusion, 2015, 24: 147–164. doi: 10.1016/j.inffus.2014.09.004 |
[20] | Jagalingam P, Hegde A V. A Review of quality metrics for fused image [J]. Aquatic Procedia, 2015, 4: 133–142. doi: 10.1016/j.aqpro.2015.02.019 |
[21] | 张小利, 李雄飞, 李军.融合图像质量评价指标的相关性分析及性能评估[J].自动化学报, 2014, 40(2): 306–315. Zhang Xiaoli, LI Xiongfei, LI Jun. Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40(2): 306–315. |
[22] | Han Yu, Cai Yunze, Cao Yin, et al. A new image fusion performance metric based on visual information fidelity[J]. Information Fusion, 2013, 14(2): 127–135. doi: 10.1016/j.inffus.2011.08.002 |
[23] | Warne R T. Testing Spearman's hypothesis with advanced placement examination data[J]. Intelligence, 2016, 57: 87–95. doi: 10.1016/j.intell.2016.05.002 |
Abstract: Image fusion is an important branch of multi-sensor information fusion, which is to synthesize several images or sequential detective images about one scene into a more complete and thorough image. At present, this technology has achieved a universal usage in remote sense detection, computer vision, target detection and recognition, etc. However, because of the variances of fusion image type, there is no standard evaluation method. Researchers have to select some appropriate evaluation indicators from a number of objective evaluation indicators by experience. The result is that different studies select different evaluation indicators, and it is hard to compare, which leads to lower persuasion in theory study. The hot issue on nowadays study is to choose relative evaluation indicators according to evaluation targets, and synthesize the chosen evaluation indicators to a comprehensive indicator. Indicator accuracy can be achieved through complementary advantages among indicators. An evaluation method of multiband fusion image is proposed based on the correlation of subjective and objective evaluations. This evaluation method includes the following steps. First, subjectively evaluate a variety of fusion results from four aspects. They are the clarity of edge, natural sense, information quantity and comprehensive evaluation. The evaluation level is divided into five levels:"good", "better", "normal", "poor" and"bad". Secondly, calculate the 14 objective evaluation indicators of the fusion results. Thirdly, normalize the subjective and objective evaluation results. Fourthly, use relative Spearman coefficient to calculate the correlation among each evaluation aspect and the 14 objective evaluation indicators. Fifthly, use the correlation to calculate the occupation weight of each objective evaluation indicator in the comprehensive evaluation indicator. Finally, construct a comprehensive index based on the correlation of the 14 indexes for every objective evaluation.
The experimental results show that the synthesis indicator based on correlation between subject and object evaluation is more relevant to the objective evaluations than the individual evaluation indicator, CMSVD (complex matrix singular value decomposition) and MSA (multi-hierarchical synthesis analysis). The correlation of clarity of edge, natural sense, information quantity and comprehensive evaluation are 0.634, 0.630, 0.737, and 0.661, respectively. As for different evaluation aspects, the correlations between the objective evaluation and subjective evaluation are different. However, the correlations of AG (average gradient), SF (spatial frequency) and VIFF (visual information fidelity for fusion) are relatively higher than other aspects.
Basic idea of multiband fusion image evaluation method.
Group 1 image to be evaluated. (a) RP fusion result. (b) NSCT fusion result. (c) PCA fusion result. (d) SVT fusion result. (e) AVG fusion result. (f) LPSR fusion result. (g) SR fusion result. (h) DCNN fusion result.
Group 2 image to be evaluated. (a) RP fusion result. (b) NSCT fusion result. (c) PCA fusion result. (d) SVT fusion result. (e) AVG fusion result. (f) LPSR fusion result. (g) SR fusion result. (h) DCNN fusion result.
Correlation coefficient between three comprehensive indexes and subjective evaluation. (a) Clarity of edge correlation coefficient. (b) Natural sense correlation coefficient. (c) Information correlation coefficient. (d) Comprehensive evaluation correlation coefficient.
Correlation coefficient between three comprehensive indexes and subjective evaluation of validation experiment. (a) Clarity of edge correlation coefficient. (b) Natural sense correlation coefficient. (c) Information correlation coefficient. (d) Comprehensive evaluation correlation coefficient.