Adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm

Liang Liming; Lu Baohe; Long Pengwei; Yang Yuan

doi:10.12086/oee.2023.230161

Article navigation > Opto-Electronic Engineering > 2023 Vol. 50 > No. 10 > 230161

Next Article Previous Article

Liang L M, Lu B H, Long P W, et al. Adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm[J]. Opto-Electron Eng, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161

Citation:

Liang L M, Lu B H, Long P W, et al. Adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm[J]. Opto-Electron Eng, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161

Adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm

School of Electrical Engineering and Automation, Jiangxi University of Science and Technology, Ganzhou, Jiangxi 341000, China

Fund Project: Project supported by National Natural Science Foundation of China (51365017, 6146301), and Natural Science Foundation of Jiangxi Province (20192BAB205084)

More Information

Corresponding author: 939175848@qq.com

Received Date 03 July 2023

Revised Date 28 September 2023

Accepted Date 07 October 2023

Published Date 25 October 2023

Abstract

Abstract

An adaptive feature fusion cascaded Transformer retinal vessel segmentation algorithm is proposed in this paper to address issues such as pathological artifacts interference, incomplete segmentation of small vessels, and low contrast between vascular foreground and non-vascular background. Firstly, image preprocessing is performed through contrast-limited histogram equalization and Gamma correction to enhance vascular texture features. Secondly, an adaptive enhancing attention module is designed in the encoding part to reduce computational redundancy while eliminating noise in retinal background images. Furthermore, a cascaded ensemble Transformer module is introduced at the bottom of the encoding-decoding structure to establish dependencies between long and short-distance vascular features. Lastly, a gate-controlled feature fusion module is introduced in the decoding part to achieve semantic fusion between encoding and decoding, enhancing the smoothness of retinal vessel segmentation. Validation on public datasets DRIVE, CHASE_DB1, and STARE yielded accuracy rates of 97.09%, 97.60%, and 97.57%, sensitivity rates of 80.38%, 81.05%, and 80.32%, and specificity rates of 98.69%, 98.71%, and 98.99%, respectively. Experimental results indicate that the overall performance of this algorithm surpasses that of most existing state-of-the-art methods and holds potential value in the diagnosis of clinical ophthalmic diseases.
- retinal vessel segmentation /
- Transformer /
- adaptive enhancement of attention /
- gating feature fusion

FullText(HTML)

References

[1]	梁礼明, 周珑颂, 陈鑫, 等. 鬼影卷积自适应视网膜血管分割算法[J]. 光电工程, 2021, 48(10): 210291. doi: 10.12086/oee.2021.210291 CrossRef Google Scholar Liang L M, Zhou L S, Chen X, et al. Ghost convolution adaptive retinal vessel segmentation algorithm[J]. Opto-Electron Eng, 2021, 48(10): 210291. doi: 10.12086/oee.2021.210291 CrossRef Google Scholar
[2]	梁礼明, 阳渊, 何安军, 等. 跨级可变形Transformer编解码视网膜图像分割算法[J]. 无线电工程, 2023, 53(9): 1990−2001. doi: 10.3969/j.issn.1003-3106.2023.09.002 CrossRef Google Scholar Liang L M, Yang Y, He A J, et al. Cross-stage deformable transformer encoding and decoding algorithm for retinal image segmentation[J]. Radio Eng, 2023, 53(9): 1990−2001. doi: 10.3969/j.issn.1003-3106.2023.09.002 CrossRef Google Scholar
[3]	梁礼明, 董信, 李仁杰, 等. 基于注意力机制多特征融合的视网膜病变分级算法[J]. 光电工程, 2023, 50(1): 220199. doi: 10.12086/oee.2023.220199 CrossRef Google Scholar Liang L M, Dong X, Li R J, et al. Classification algorithm of retinopathy based on attention mechanism and multi feature fusion[J]. Opto-Electron Eng, 2023, 50(1): 220199. doi: 10.12086/oee.2023.220199 CrossRef Google Scholar
[4]	吕佳, 王泽宇, 梁浩城. 边界注意力辅助的动态图卷积视网膜血管分割[J]. 光电工程, 2023, 50(1): 220116. doi: 10.12086/oee.2023.220116 CrossRef Google Scholar Lv J, Wang Z Y, Liang H C. Boundary attention assisted dynamic graph convolution for retinal vascular segmentation[J]. Opto-Electron Eng, 2023, 50(1): 220116. doi: 10.12086/oee.2023.220116 CrossRef Google Scholar
[5]	Vlachos M, Dermatas E. Multi-scale retinal vessel segmentation using line tracking[J]. Comput Med Imaging Graph, 2010, 34(3): 213−227. doi: 10.1016/j.compmedimag.2009.09.006 CrossRef Google Scholar
[6]	Azzopardi G, Strisciuglio N, Vento M, et al. Trainable COSFIRE filters for vessel delineation with application to retinal images[J]. Med Image Analy, 2015, 19(1): 46−57. doi: 10.1016/j.media.2014.08.002 CrossRef Google Scholar
[7]	王晓红, 赵于前, 廖苗, 等. 基于多尺度2D Gabor小波的视网膜血管自动分割[J]. 自动化学报, 2015, 41(5): 970−980. doi: 10.16383/j.aas.2015.c140185 CrossRef Google Scholar Wang X H, Zhao Y Q, Liao M, et al. Automatic segmentation for retinal vessel based on multi-scale 2D Gabor wavelet[J]. Acta Autom Sin, 2015, 41(5): 970−980. doi: 10.16383/j.aas.2015.c140185 CrossRef Google Scholar
[8]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 2015: 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965. Google Scholar
[9]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234–241. https://doi.org/10.1007/978-3-319-24574-4_28. Google Scholar
[10]	Bakas S, Reyes M, Jakab A, et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge[Z]. arXiv: 1811.02629, 2018. https://doi.org/10.48550/arXiv.1811.02629. Google Scholar
[11]	Heller N, Sathianathen N, Kalapara A, et al. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes[Z]. arXiv: 1904.00445, 2019. https://doi.org/10.48550/arXiv.1904.00445. Google Scholar
[12]	Li Y, Zhang Y, Cui W G, et al. Dual encoder-based dynamic-channel graph convolutional network with edge enhancement for retinal vessel segmentation[J]. IEEE Trans Med Imag, 2022, 41(8): 1975−1989. doi: 10.1109/TMI.2022.3151666 CrossRef Google Scholar
[13]	Wang X, Li Z S, Huang Y P, et al. Multimodal medical image segmentation using multi-scale context-aware network[J]. Neurocomputing, 2022, 486: 135−146. doi: 10.1016/j.neucom.2021.11.017 CrossRef Google Scholar
[14]	Yang B, Qin L, Peng H, et al. SDDC-Net: A U-shaped deep spiking neural P convolutional network for retinal vessel segmentation[J]. Digital Signal Process, 2023, 136: 104002. doi: 10.1016/j.dsp.2023.104002 CrossRef Google Scholar
[15]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 2017: 6000–6010. https://dl.acm.org/doi/10.5555/3295222.3295349 Google Scholar
[16]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]//Proceedings of the 9th International Conference on Learning Representations, 2021. Google Scholar
[17]	Liu X Y, Peng H W, Zheng N X, et al. EfficientViT: memory efficient vision transformer with cascaded group attention[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 2023: 14420–14430. https://doi.org/10.1109/CVPR52729.2023.01386. Google Scholar
[18]	Azad R, Jia Y W, Aghdam E K, et al. Enhancing medical image segmentation with TransCeption: a multi-scale feature fusion approach[Z]. arXiv: 2301.10847, 2023. https://doi.org/10.48550/arXiv.2301.10847. Google Scholar
[19]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018: 7132–7141. https://doi.org/10.1109/CVPR.2018.00745. Google Scholar
[20]	Ni J J, Sun H Z, Xu J X, et al. A feature aggregation and feature fusion network for retinal vessel segmentation[J]. Biomed Signal Process Control, 2023, 85: 104829. doi: 10.1016/j.bspc.2023.104829 CrossRef Google Scholar
[21]	Zhang R F, Liu S S, Yu Y Z, et al. Self-supervised correction learning for semi-supervised biomedical image segmentation[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, 2021: 134–144. https://doi.org/10.1007/978-3-030-87196-3_13. Google Scholar
[22]	Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: Learning where to look for the pancreas[Z]. arXiv: 1804.03999, 2018. https://doi.org/10.48550/arXiv.1804.03999. Google Scholar
[23]	Wang C, Zhao Z Y, Ren Q Q, et al. Dense U-net based on patch-based learning for retinal vessel segmentation[J]. Entropy, 2019, 21(2): 168. doi: 10.3390/e21020168 CrossRef Google Scholar
[24]	Liu W T, Yang H H, Tian T, et al. Full-resolution network and dual-threshold iteration for retinal vessel and coronary angiograph segmentation[J]. IEEE J Biomed Health Inform, 2022, 26(9): 4623−4634. doi: 10.1109/JBHI.2022.3188710 CrossRef Google Scholar
[25]	Zhang H B, Zhong X, Li Z J, et al. TiM-Net: transformer in M-Net for retinal vessel segmentation[J]. J Healthcare Eng, 2022, 2022: 9016401. doi: 10.1155/2022/9016401 CrossRef Google Scholar
[26]	Tchinda B S, Tchiotsop D, Noubom M, et al. Retinal blood vessels segmentation using classical edge detection filters and the neural network[J]. Inf Med Unlocked, 2021, 23: 100521. doi: 10.1016/j.imu.2021.100521 CrossRef Google Scholar
[27]	Du X F, Wang J S, Sun W Z. UNet retinal blood vessel segmentation algorithm based on improved pyramid pooling method and attention mechanism[J]. Phys Med Biol, 2021, 66(17): 175013. doi: 10.1088/1361-6560/ac1c4c CrossRef Google Scholar
[28]	Khan T M, Khan M A U, Rehman N U, et al. Width-wise vessel bifurcation for improved retinal vessel segmentation[J]. Biomed Signal Process Control, 2022, 71: 103169. doi: 10.1016/j.bspc.2021.103169 CrossRef Google Scholar
[29]	Guo S. CSGNet: Cascade semantic guided net for retinal vessel segmentation[J]. Biomed Signal Process Control, 2022, 78: 103930. doi: 10.1016/j.bspc.2022.103930 CrossRef Google Scholar
[30]	Li X, Jiang Y C, Li M L, et al. Lightweight attention convolutional neural network for retinal vessel image segmentation[J]. IEEE Trans Ind Inf, 2021, 17(3): 1958−1967. doi: 10.1109/TII.2020.2993842 CrossRef Google Scholar
[31]	Yang X, Li Z Q, Guo Y Q, et al. DCU-net: A deformable convolutional neural network based on cascade U-net for retinal vessel segmentation[J]. Multimedia Tools Appl, 2022, 81(11): 15593−15607. doi: 10.1007/s11042-022-12418-w CrossRef Google Scholar
[32]	Khan T M, Naqvi S S, Robles-Kelly A, et al. Retinal vessel segmentation via a Multi-resolution Contextual Network and adversarial learning[J]. Neural Netw, 2023, 165: 310−320. doi: 10.1016/j.neunet.2023.05.029 CrossRef Google Scholar

Overview

Overview

Retinal blood vessel images contain rich geometric structures, such as vessel diameter, branching angle, and length, which allow ophthalmologists to prevent and diagnose diseases such as hypertension, diabetes, and atherosclerosis by observing information about retinal blood vessel structure. However, the topology of the fundus blood vessels is intricate and difficult to extract medically, so it is important to study a retinal vessel segmentation algorithm that can be efficient and automatic for clinicopathologic diagnosis. The contemporary retinal vessel segmentation methods are mainly categorized into traditional machine- and deep-learning-based methods. Traditional machine learning methods include morphology-based processing, matched filter-based, and wavelet transform, etc. Such methods usually do not require a priori labeling information, but rather utilize the similarity between the data for analysis. The deep learning method is an end-to-end learning method, that can automatically extract the bottom and high-level feature information of the image, compared with the traditional segmentation methods to avoid the process of manual feature extraction, and at the same time reduce the subjectivity of segmentation, and its generalization ability is significantly better than that of the traditional methods. However, the fundus retinal segmentation task still suffers from pathologic artifact interference, incomplete segmentation of tiny vessels, and low contrast between the vascular foreground and the nonvascular background. To solve the above problems, an adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm is proposed. The original image of the retina dataset was first subjected to dataset expansion to ensure adequate training and prediction of the model, and operations such as gamma correction were performed to perform dataset image enhancement and to improve the contrast of the blood vessel texture. Secondly, the adaptive enhancement attention module is designed in the encoding part to improve the information interaction ability between different channels, and at the same time, the background noise information of the image is eliminated to reduce the interference of pathological artifacts and enhance the nonlinear ability of the vascular image. Then the cascade group Transformer module is added at the bottom end of the codec to effectively aggregate the contextual vascular feature information and fully capture the local features of tiny blood vessels. Finally, a gated feature fusion module is introduced in the decoding part to capture the spatial feature information of different sizes in the codec layer, which improves the feature utilization and algorithm robustness. Validated on the public datasets DRIVE, CHASE_DB1, and STARE, the accuracy reaches 97.09%, 97.60%, and 97.57%, the sensitivity reaches 80.38%, 81.05%, and 80.32%, and the specificity reaches 98.69%, 98.71%, and 98.99%. The experimental results show that the overall performance of the algorithm in this paper is better than most of the existing state-of-the-art algorithms, and it has a certain application value for the diagnosis of clinical ophthalmic diseases.