Color space focusing evaluation algorithm for color overlay microscopy

Shi Yanqiong; Yang Yonghui; Zha Zhao; Zhu Guang; Zheng Yu

doi:10.12086/oee.2024.240078

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 7 > 240078

Next Article Previous Article

Shi Y Q, Yang Y H, Zha Z, et al. Color space focusing evaluation algorithm for color overlay microscopy[J]. Opto-Electron Eng, 2024, 51(7): 240078. doi: 10.12086/oee.2024.240078

Citation:

Shi Y Q, Yang Y H, Zha Z, et al. Color space focusing evaluation algorithm for color overlay microscopy[J]. Opto-Electron Eng, 2024, 51(7): 240078. doi: 10.12086/oee.2024.240078

Color space focusing evaluation algorithm for color overlay microscopy

School of Mechanical and Electrical Engineering, Anhui Jianzhu University, Hefei, Anhui 230601, China

Fund Project: Project supported by Anhui Province Major Science and Technology Special Project(202203a05020022); Anhui Jianzhu University Talent Introduction and Doctoral Initiation Fund (2019QDZ16)

More Information

^*Corresponding author: 1213814242@qq.com

Received Date 29 March 2024

Revised Date 19 June 2024

Accepted Date 19 June 2024

Published Date 20 August 2024

Abstract

Abstract

Focusing evaluation is the key to extending the depth of field in microscopy with stacked focus. To accurately and quickly obtain the pixel focusing position of the stacked focus image sequences and generate high-quality all-in-focus images, a focusing evaluation algorithm based on color vector space is proposed. This algorithm directly calculates color image gradients in the RGB vector space, fully utilizing the correlation between color channels, avoiding the information loss caused by traditional focus evaluation algorithms when converting color images into grayscale images, and has higher accuracy compared to simple stacking of color component gradients; Using the average Manhattan distance between the center pixel and neighboring pixels in RGB space as the focus evaluation weight can enhance the sensitivity of the focusing part, reduce the evaluation value of the defocused part, and make the focus evaluation curve characteristics tend to be idealized. Seven focusing evaluation algorithms in spatial domain, frequency domain, and statistics were selected for performance comparison experiments with the proposed algorithm. The results indicate that the proposed algorithm has better sensitivity, focusing resolution, and noise resistance in simulated and real microscopic images. The curve characteristics were significantly improved, and its application in microscope depth extension can further improve the quality of stacked focal large-depth imaging.
- focusing evaluation /
- focus stacking image fusion /
- color space /
- depth of field /
- color image gradient /
- microscope

FullText(HTML)

References

[1]	张子建, 徐欣, 王吉祥, 等. 光片荧光显微镜研究进展[J]. 光电工程, 2023, 50(5): 220045. doi: 10.12086/oee.2023.220045 CrossRef Google Scholar Zhang Z J, Xu X, Wang J X, et al. Review of the development of light sheet fluorescence microscopy[J]. Opto-Electron Eng, 2023, 50(5): 220045. doi: 10.12086/oee.2023.220045 CrossRef Google Scholar
[2]	李晟, 王博文, 管海涛, 等. 远场合成孔径计算光学成像技术: 文献综述与最新进展[J]. 光电工程, 2023, 50(10): 230090. doi: 10.12086/oee.2023.230090 CrossRef Google Scholar Li S, Wang B W, Guan H T, et al. Far-field computational optical imaging techniques based on synthetic aperture: a review[J]. Opto-Electron Eng, 2023, 50(10): 230090. doi: 10.12086/oee.2023.230090 CrossRef Google Scholar
[3]	张平, 吴嘉敏, 林靖宇, 等. 显微镜景深拓展技术研究[J]. 应用光学, 2014, 35(6): 1075−1082. doi: 10.5768/JAO201435.0605005 CrossRef Google Scholar Zhang P, Wu J M, Lin J Y, et al. Extended depth of field in microscopy[J]. J Appl Opt, 2014, 35(6): 1075−1082. doi: 10.5768/JAO201435.0605005 CrossRef Google Scholar
[4]	王伟, 张露鹤, 傅天文. 基于波前编码的扩展景深短波红外成像系统[J]. 激光与光电子学进展, 2023, 60(10): 1011005. doi: 10.3788/LOP223156 CrossRef Google Scholar Wang W, Zhang L H, Fu T W. Wavefront coding-based short-wave infrared imaging system for extended depth of field[J]. Laser Optoelectron Prog, 2023, 60(10): 1011005. doi: 10.3788/LOP223156 CrossRef Google Scholar
[5]	胡杰, 唐紫依, 蓝翔, 等. 基于相变材料Ge₂Sb₂Se₄Te₁的可切换边缘检测与聚焦成像超表面[J]. 光电工程, 2023, 50(8): 220284. doi: 10.12086/oee.2023.220284 CrossRef Google Scholar Hu J, Tang Z Y, Lan X, et al. Switchable edge detection and imaging based on a phase-change metasurface with Ge₂Sb₂Se₄Te₁[J]. Opto-Electron Eng, 2023, 50(8): 220284. doi: 10.12086/oee.2023.220284 CrossRef Google Scholar
[6]	Seyler T, Fratz M, Beckmann T, et al. Extending the depth of field beyond geometrical imaging limitations using phase noise as a focus measure in multiwavelength digital holography[J]. Appl Sci, 2018, 8(7): 1042. doi: 10.3390/app8071042 CrossRef Google Scholar
[7]	于春水, 卢荣胜. 叠焦大景深成像中的聚焦评价算子性能评估方法[J]. 激光与光电子学进展, 2022, 59(14): 1415027. doi: 10.3788/LOP202259.1415027 CrossRef Google Scholar Yu C S, Lu R S. Performance evaluation method for focusing evaluation operator in superposed large depth imaging[J]. Laser Optoelectron Prog, 2022, 59(14): 1415027. doi: 10.3788/LOP202259.1415027 CrossRef Google Scholar
[8]	Ali U, Mahmood M T. Energy minimization for image focus volume in shape from focus[J]. Pattern Recognit, 2022, 126: 108559. doi: 10.1016/J.PATCOG.2022.108559 CrossRef Google Scholar
[9]	梁鑫, 卿粼波, 要小涛, 等. 一种基于中频滤波的动态步长聚焦算法[J]. 智能计算机与应用, 2022, 12(1): 35−40. doi: 10.3969/j.issn.2095-2163.2022.01.007 CrossRef Google Scholar Liang X, Qing L B, Yao X T, et al. A dynamic step-size focusing algorithm based on intermediate frequency filtering[J]. Intell Comput Appl, 2022, 12(1): 35−40. doi: 10.3969/j.issn.2095-2163.2022.01.007 CrossRef Google Scholar
[10]	熊锐, 顾乃庭, 徐洪艳. 一种适应多方向灰度梯度变化的自动对焦评价函数[J]. 激光与光电子学进展, 2022, 59(4): 0418001. doi: 10.3788/LOP202259.0418001 CrossRef Google Scholar Xiong R, Gu N T, Xu H Y. An auto-focusing evaluation function adapted to multi-directional gray gradient change[J]. Laser Optoelectron Prog, 2022, 59(4): 0418001. doi: 10.3788/LOP202259.0418001 CrossRef Google Scholar
[11]	吕美妮, 玉振明. 基于DCT零系数和局部标准差的自动聚焦算法[J]. 激光技术, 2018, 42(1): 66−71. doi: 10.7510/jgjs.issn.1001-3806.2018.01.013 CrossRef Google Scholar Lv M N, Yu Z M. Automatic focusing algorithm based on DCT coefficient of zero and local standard deviation[J]. Laser Technol, 2018, 42(1): 66−71. doi: 10.7510/jgjs.issn.1001-3806.2018.01.013 CrossRef Google Scholar
[12]	Mahmood M T. Cross-scale focus measure aggregation in depth recovery of microscopic objects[J]. Microsc Res Tech, 2019, 82(6): 872−877. doi: 10.1002/jemt.23230 CrossRef Google Scholar
[13]	Shim S O. Multidirectional focus measure for accurate three-dimensional shape recovery of microscopic objects[J]. Microsc Res Tech, 2022, 85(3): 940−947. doi: 10.1002/jemt.23963 CrossRef Google Scholar
[14]	Pertuz S, Puig D, Garcia M A. Analysis of focus measure operators for shape-from-focus[J]. Pattern Recognit, 2013, 46(5): 1415−1432. doi: 10.1016/j.patcog.2012.11.011 CrossRef Google Scholar
[15]	Jang H S, Yun G, Mutahira H, et al. A new focus measure operator for enhancing image focus in 3D shape recovery[J]. Microsc Res Tech, 2021, 84(10): 2483−2493. doi: 10.1002/jemt.23781 CrossRef Google Scholar
[16]	曹震, 巢渊, 徐魏, 等. 基于改进梯度加权的零件图像高精度聚焦方法[J]. 电子测量与仪器学报, 2023, 37(11): 132−142. doi: 10.13382/j.jemi.B2306767 CrossRef Google Scholar Cao Z, Chao Y, Xu W, et al. High-precision focusing method for parts image based on improved gradient weighting[J]. J Electron Meas Instrum, 2023, 37(11): 132−142. doi: 10.13382/j.jemi.B2306767 CrossRef Google Scholar
[17]	王秀峰, 孙小伟, 王加科, 等. 余弦变换与拉普拉斯算子结合的聚焦评价方法[J]. 激光与光电子学进展, 2021, 58(24): 2410005. doi: 10.3788/LOP202158.2410005 CrossRef Google Scholar Wang X F, Sun X W, Wang J K, et al. Focus measure operator combining cosine transform and Laplacian operator[J]. Laser Optoelectron Prog, 2021, 58(24): 2410005. doi: 10.3788/LOP202158.2410005 CrossRef Google Scholar
[18]	Zhang Z, Liu Y, Xiong Z H, et al. Focus and blurriness measure using reorganized DCT coefficients for an autofocus application[J]. IEEE Trans Circuits Syst Video Technol, 2018, 28(1): 15−30. doi: 10.1109/TCSVT.2016.2602308 CrossRef Google Scholar
[19]	刘斌, 谯倩, 赵静, 等. 基于高频方差熵清晰度评价函数的聚焦三维测量方法[J]. 红外与激光工程, 2021, 50(5): 20200326. doi: 10.3788/IRLA20200326 CrossRef Google Scholar Liu B, Qiao Q, Zhao J, et al. 3D profile measurement based on depth from focus method using high-frequency component variance weighted entropy image sharpness evaluation function[J]. Infrared Laser Eng, 2021, 50(5): 20200326. doi: 10.3788/IRLA20200326 CrossRef Google Scholar
[20]	翟永平, 周东翔, 刘云辉, 等. 聚焦函数性能评价指标设计及最优函数选取[J]. 光学学报, 2011, 31(4): 0418002. doi: 10.3788/AOS201131.0418002 CrossRef Google Scholar Zhai Y P, Zhou D X, Liu Y H, et al. Design of evaluation index for auto-focusing function and optimal function selection[J]. Acta Opt Sin, 2011, 31(4): 0418002. doi: 10.3788/AOS201131.0418002 CrossRef Google Scholar
[21]	Mutahira H, Ahmad B, Muhammad M S, et al. Focus measurement in color space for shape from focus systems[J]. IEEE Access, 2021, 9: 103291−103310. doi: 10.1109/ACCESS.2021.3098753 CrossRef Google Scholar
[22]	Chatoux H, Richard N, Lecellier F, et al. Gradient in spectral and color images: from the Di Zenzo initial construction to a generic proposition[J]. J Opt Soc Am A, 2019, 36(11): C154−C165. doi: 10.1364/JOSAA.36.00C154 CrossRef Google Scholar
[23]	Di Zenzo S. A note on the gradient of a multi-image[J]. Comput Vision Graphics Image Process, 1986, 33(1): 116−125. doi: 10.1016/0734-189X(86)90223-9 CrossRef Google Scholar
[24]	Koschan A, Abidi M. Detection and classification of edges in color images[J]. IEEE Signal Process Mag, 2005, 22(1): 64−73. doi: 10.1109/MSP.2005.1407716 CrossRef Google Scholar
[25]	董正琼, 杨清锋, 黄贤文, 等. 基于梯度方差聚焦评价的三维显微测量方法[J]. 激光与光电子学进展, 2024. https://doi.org/10.3788/LOP232028. Google Scholar Dong Z Q, Yang Q F, Huang X W, et al. Three-dimensional microscopic measurement method based on gradient variance focus evaluation[J]. Laser Optoelectron Prog, 2024. https://doi.org/10.3788/LOP232028. Google Scholar
[26]	Honauer K, Johannsen O, Kondermann D, et al. A dataset and evaluation methodology for depth estimation on 4D light fields[C]//13th Asian Conference on Computer Vision, 2017: 19–34. https://doi.org/10.1007/978-3-319-54187-7_2. Google Scholar
[27]	张雅玲, 吉琳娜, 杨风暴, 等. 基于余弦相似性的双模态红外图像融合性能表征[J]. 光电工程, 2019, 46(10): 190059. doi: 10.12086/oee.2019.190059 CrossRef Google Scholar Zhang Y L, Ji L N, Yang F B, et al. Characterization of dual-mode infrared images fusion based on cosine similarity[J]. Opto-Electron Eng, 2019, 46(10): 190059. doi: 10.12086/oee.2019.190059 CrossRef Google Scholar
[28]	Li Y, Li X Y, Wang J Q, et al. Multi-focus image fusion for microscopic depth-of-field extension of waterjet-assisted laser processing[J]. Int J Adv Manuf Technol, 2024, 131(3-4): 1717−1734. doi: 10.1007/s00170-024-13118-5 CrossRef Google Scholar

Overview

Overview

Focusing evaluation is the key to stacked focus extended depth of field imaging, and traditional spatial domain focusing evaluation algorithms mostly use the degree of drastic changes in image grayscale values as the basis for clarity evaluation. However, converting color images to grayscale images can result in multiple pixels with different color values being mapped onto the same grayscale pixels. This imprecise pixel mapping relationship can cause serious loss of image information, greatly affecting the accuracy of the focus evaluation algorithm, thereby reducing the overall accuracy of the stacked depth of field. Moreover, color images formed by this calculation method of stacked focus may yield results that are inconsistent with human visual characteristics. Based on the above issues, this article proposes a focus evaluation algorithm based on color vector space to more accurately and quickly obtain the pixel focus position of color image sequences and generate high-quality panoramic deep images. This algorithm extends the concept of gradients to vector functions, directly calculating color image gradients in the RGB color vector space, preserving image color information, and fully utilizing the correlation of various color channels. Compared to calculating gradients based on image grayscale and individual color components, it has higher accuracy and sensitivity. The average Manhattan distance between the central pixel and neighboring pixels in RGB space is used as the focus evaluation weight to enhance the sensitivity of the focusing part and reduce the evaluation value of the defocus part, effectively improving the resolution and anti-interference ability of the focusing evaluation function. This article selects seven focusing evaluation algorithms in the spatial domain, frequency domain, and statistics to conduct simulation comparison experiments and real environment comparison experiments with the proposed algorithm from two aspects: focusing evaluation function curve characteristics and stacked focus extended depth of field imaging performance. The experimental results show that compared with several selected focusing evaluation operators, the color vector space focusing evaluation algorithm achieved the best peak sensitivity, steepness, and gentle fluctuation indicators on three sets of simulated images and two sets of real microscopic images, and generated higher-quality panoramic depth images. Especially for the focus evaluation problem of images with a wide variety of colors and rich information, the proposed focus evaluation algorithm can accurately calculate the pixel focus value and has a significant overlapping fusion effect, which can meet the requirements of expanding the depth of field in microscopy and has practical application value.