Study on retinal OCT segmentation with dual-encoder

Chen Minghui; Wang Teng; Yuan Yuan; Ke Shuting

doi:10.12086/oee.2023.230146

Article navigation > Opto-Electronic Engineering > 2023 Vol. 50 > No. 10 > 230146

Next Article Previous Article

Chen M H, Wang T, Yuan Y, et al. Study on retinal OCT segmentation with dual-encoder[J]. Opto-Electron Eng, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146

Citation:

Chen M H, Wang T, Yuan Y, et al. Study on retinal OCT segmentation with dual-encoder[J]. Opto-Electron Eng, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146

Study on retinal OCT segmentation with dual-encoder

Shanghai Engineering Research Center of Interventional Medical, Shanghai Institute for Interventional Medical Devices, School of Health Science and Engineering College of Health Sciences and Engineering, University of Shanghai for Science and Technology, Shanghai 200082, China

Fund Project: Project supported by Shanghai Science and Technology Commission Industry-university-research Medical Project (15DZ1940400)

More Information

^*Corresponding author: cmhui.43@163.com

Received Date 26 June 2023

Revised Date 10 November 2023

Accepted Date 14 November 2023

Published Date 25 October 2023

Abstract

Abstract

There are noises and speckles in OCT retinal images, and a single extraction of spatial features is often easy to miss some important information. Therefore, the target region cannot be accurately segmented. OCT images themselves have spectral frequency domain characteristics. Aiming at the frequency domain characteristics of OCT images, this paper proposes a new dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The proposed frequency encoder can extract image frequency domain information and convert it into spatial information through fast Fourier convolution. The lack of feature information that can be omitted by a single space encoder will be well-complemented. After comparison with other classical models and ablation experiments, the results show that with the addition of a frequency domain encoder, the model can effectively improve the segmentation performance of the retinal layer and liquid. Both average Dice coefficient and mIoU are increased by 2% compared with U-Net. They are increased by 8% and 4% compared with ReLayNet, respectively. Among them, the improvement of liquid segszmentation is particularly obvious, and the Dice coefficient is increased by 10% compared with the U-Net model.
- optical coherence tomography /
- convolutional neural network /
- segmentation of image /
- dual-encoder

FullText(HTML)

References

[1]	Hayreh S S, Zimmerman M B. Fundus changes in branch retinal arteriolar occlusion[J]. Retina, 2015, 35(10): 2060−2066. doi: 10.1097/IAE.0000000000000585 CrossRef Google Scholar
[2]	Fraz M M, Remagnino P, Hoppe A, et al. Blood vessel segmentation methodologies in retinal images—A survey[J]. Comput Methods Programs Biomed, 2012, 108(1): 407−433. doi: 10.1016/j.cmpb.2012.03.009 CrossRef Google Scholar
[3]	李云耀, 樊金宇, 蒋天亮, 等. 光学相干层析技术在眼科手术导航方面的研究进展[J]. 光电工程, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027 CrossRef Google Scholar Li Y Y, Fan J Y, Jiang T L, et al. Review of the development of optical coherence tomography imaging navigation technology in ophthalmic surgery[J]. Opto-Electron Eng, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027 CrossRef Google Scholar
[4]	姜盼秋, 汪平河. 谱域光学相干层析系统的色散补偿技术研究[J]. 光电工程, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184 CrossRef Google Scholar Jiang P Q, Wang P H. Research on dispersion compensation technology for SDOCT system[J]. Opto-Electron Eng, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184 CrossRef Google Scholar
[5]	Huang D, Swanson E A, Lin C P, et al. Optical coherence tomography[J]. Science, 1991, 254(5035): 1178−1181. doi: 10.1126/science.1957169 CrossRef Google Scholar
[6]	Tranos P G, Wickremasinghe S S, Stangos N T, et al. Macular edema[J]. Surv Ophthalmol, 2004, 49(5): 470−490. doi: 10.1016/j.survophthal.2004.06.002 CrossRef Google Scholar
[7]	Monemian M, Rabbani H. Mathematical analysis of texture indicators for the segmentation of optical coherence tomography images[J]. Optik, 2020, 219: 165227. doi: 10.1016/j.ijleo.2020.16522 CrossRef Google Scholar
[8]	Liu Y H, Carass A, He Y F, et al. Layer boundary evolution method for macular OCT layer segmentation[J]. Biomed Opt Express, 2019, 10(3): 1064−1080. doi: 10.1364/BOE.10.001064 CrossRef Google Scholar
[9]	杨平, 彭清, 刘维平, 等. 一种眼底黄斑水肿OCT图像分割方法[J]. 生物医学工程学杂志, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015 CrossRef Google Scholar Yang P, Peng Q, Liu W P, et al. A segmentation algorithm of OCT image for macula edema[J]. J Biomed Eng, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015 CrossRef Google Scholar
[10]	何锦涛, 陈明惠, 贾文宇, 等. 眼底OCT图像中糖尿病性黄斑水肿的分割[J]. 光电工程, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605 CrossRef Google Scholar He J T, Chen M H, Jia W Y, et al. Segmentation of diabetic macular edema in OCT retinal images[J]. Opto-Electron Eng, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605 CrossRef Google Scholar
[11]	Hussain A, Bhuiyan A, Turpin A, et al. Automatic identification of pathology-distorted retinal layer boundaries using SD-OCT imaging[J]. IEEE Trans Biomed Eng, 2017, 64(7): 1638−1649. doi: 10.1109/TBME.2016.2619120 CrossRef Google Scholar
[12]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90. Google Scholar
[13]	Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, 2017. https://doi.org/10.1609/aaai.v31i1.11231. Google Scholar
[14]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations, San Diego, 2015. Google Scholar
[15]	吴玉超, 林岚, 王婧璇. 基于卷积神经网络的语义分割在医学图像中的应用[J]. 生物医学工程学杂志, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067 CrossRef Google Scholar Wu Y C, Lin L, Wang J X. Application of semantic segmentation based on convolutional neural network in medical images[J]. J Biomed Eng, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067 CrossRef Google Scholar
[16]	秦传波, 宋子玉, 曾军英, 等. 联合多尺度和注意力-残差的深度监督乳腺癌分割[J]. 光学精密工程, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877 CrossRef Google Scholar Qin C B, Song Z Y, Zeng J Y, et al. Deeply supervised breast cancer segmentation combined with multi-scale and attention-residuals[J]. Opt Precis Eng, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877 CrossRef Google Scholar
[17]	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical Image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234–241. https://doi.org/10.1007/978-3-319-24574-4_28. Google Scholar
[18]	Roy A G, Conjeti S, Karri S P K, et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J]. Biomed Opt Express, 2017, 8(8): 3627−3642. doi: 10.1364/BOE.8.003627 CrossRef Google Scholar
[19]	袁坤, 霍力. 用于视网膜OCT图像分层的多尺度补全卷积神经网络[J]. 中国激光, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004 CrossRef Google Scholar Yuan K, Huo L. Multiple-scale inpainting convolutional neural network for retinal OCT image segmentation[J]. Chin J Lasers, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004 CrossRef Google Scholar
[20]	Liu W, Sun Y K, Ji Q G. MDAN-UNet: multi-scale and dual attention enhanced nested U-net architecture for segmentation of optical coherence tomography images[J]. Algorithms, 2020, 13(3): 60. doi: 10.3390/a13030060 CrossRef Google Scholar
[21]	Ngo L, Cha J, Han J H. Deep neural network regression for automated retinal layer segmentation in optical coherence tomography images[J]. IEEE Trans Image Process, 2020, 29: 303−312. doi: 10.1109/TIP.2019.2931461 CrossRef Google Scholar
[22]	Chi L, Jiang B R, Mu Y D. Fast Fourier convolution[C]//Proceedings of the 34th International Conference on Neural Information Processing System, Vancouver, 2020: 376. https://dl.acm.org/doi/abs/10.5555/3495724.3496100 Google Scholar
[23]	Nair V, Chatterjee M, Tavakoli N, et al. Optimizing CNN using fast Fourier transformation for object recognition[C]//Proceedings of the 19th IEEE International Conference on Machine Learning and Applications, Miami, 2020: 234–239. https://doi.org/10.1109/ICMLA51294.2020.00046. Google Scholar
[24]	Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[Z]. arXiv: 1511.07122, 2016. https://arxiv.org/abs/1511.07122. Google Scholar
[25]	Chiu S J, Allingham M J, Mettu P S, et al. Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema[J]. Biomed Opt Express, 2015, 6(4): 1172−1194. doi: 10.1364/BOE.6.001172 CrossRef Google Scholar
[26]	Mondal P K, Prodhan U K, AI Mamun S, et al. Segmentation of white blood cells using fuzzy C means segmentation algorithm[J]. IOSR J Comput Eng, 2014, 16(3): 1−5. doi: 10.9790/0661-16390105 CrossRef Google Scholar
[27]	Singh V K, Romani S, Rashwan H A, et al. Conditional generative adversarial and convolutional networks for X-ray breast mass segmentation and shape classification[C]//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, 2018: 833–840. https://doi.org/10.1007/978-3-030-00934-2_92. Google Scholar

Overview

Overview

Deep learning methods have already had a profound impact on medical image processing. However, some noises and speckles contained in OCT images affect the quality of the images, coupled with the elongated and complex retinal layer and the irregular distribution of pathological fluid in it, which brings great challenges to the automatic segmentation task. At the same time, depending on the limited manpower and time, it is also difficult to sketch a large number of existing images by relying on the professional knowledge of doctors. For the above reasons, automatic medical image segmentation, a scientific medical auxiliary support, is of great clinical significance.

The research content of this paper mainly focuses on the analysis and processing of retinal OCT images. When it comes to monitoring the state of the patient's retina layer, due to the noise and speckle in OCT images and the subtle and complex structure of the retina layer itself, the performance of the model is limited by a single extraction space feature, and the target region cannot be accurately segmented. Aiming at the frequency domain characteristics of OCT images, this paper proposes a dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The frequency domain encoder extracts the image frequency domain information and converts it into spatial information by fast Fourier convolution to complete the feature extraction of a single spatial encoder. The experimental results show that the model can effectively improve the segmentation performance of the retinal layer and liquid, both average Dice coefficient and mIoU are increased by 2% compared with U-Net, and the Dice coefficient of liquid is increased by 10%.