Super-resolution reconstruction of retinal OCT image using multi-teacher knowledge distillation network

Chen Minghui; Lu Yanqi; Yang Wenyi; Wang Yuanzhu; Shao Yi

doi:10.12086/oee.2024.240114

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 7 > 240114

Next Article Previous Article

Chen M H, Lu Y Q, Yang W Y, et al. Super-resolution reconstruction of retinal OCT image using multi-teacher knowledge distillation network[J]. Opto-Electron Eng, 2024, 51(7): 240114. doi: 10.12086/oee.2024.240114

Citation:

Chen M H, Lu Y Q, Yang W Y, et al. Super-resolution reconstruction of retinal OCT image using multi-teacher knowledge distillation network[J]. Opto-Electron Eng, 2024, 51(7): 240114. doi: 10.12086/oee.2024.240114

Super-resolution reconstruction of retinal OCT image using multi-teacher knowledge distillation network

1.
Shanghai Engineering Research Center of Interventional Medical, Shanghai Institute for Interventional Medical Devices, School of Health Sciences and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
2.
Shanghai Raykeen Laser Technology Co., Ltd., Shanghai 200120, China
3.
Shanghai General Hospital, Shanghai 200080, China

Fund Project: Project supported by Shanghai Science and Technology Commission's Industry University Research Medical Project (15DZ1940400)

More Information

^*Corresponding author: cmhui.43@163.com

Received Date 15 May 2024

Revised Date 08 August 2024

Accepted Date 09 August 2024

Published Date 20 August 2024

Abstract

Abstract

Optical coherence tomography (OCT) is widely used in ophthalmic diagnosis and adjuvant therapy, but its imaging quality is inevitably affected by speckle noise and motion artifacts. This article proposes a multi teacher knowledge distillation network MK-OCT for OCT super-resolution tasks, which uses teacher networks with different advantages to train balanced, lightweight, and efficient student networks. The use of efficient channel distillation method ECD in MK-OCT also enables the model to better preserve the texture information of retinal images, meeting clinical needs. The experimental results show that compared with classical super-resolution networks, the model proposed in this paper performs well in both reconstruction accuracy and perceptual quality, with smaller model size and less computational complexity.
- medical images /
- optical coherence tomography images /
- super-resolution /
- knowledge distillation /
- contrastive learning

FullText(HTML)

References

[1]	陆冬筱, 房文汇, 李玉瑶, 等. 光学相干层析成像技术原理及研究进展[J]. 中国光学, 2020, 13(5): 919−935. doi: 10.37188/CO.2020-0037 CrossRef Google Scholar Lu D X, Fang W H, Li Y Y, et al. Optical coherence tomography: principles and recent developments[J]. Chin Opt, 2020, 13(5): 919−935. doi: 10.37188/CO.2020-0037 CrossRef Google Scholar
[2]	Huang Y Q, Lu Z X, Shao Z M, et al. Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network[J]. Opt Express, 2019, 27(9): 12289−12307. doi: 10.1364/OE.27.012289 CrossRef Google Scholar
[3]	Das V, Dandapat S, Bora P K. Unsupervised super-resolution of OCT images using generative adversarial network for improved age-related macular degeneration diagnosis[J]. IEEE Sensors J, 2020, 20(15): 8746−8756. doi: 10.1109/JSEN.2020.2985131 CrossRef Google Scholar
[4]	Qiu B, You Y F, Huang Z Y, et al. N2NSR‐OCT: simultaneous denoising and super‐resolution in optical coherence tomography images using semisupervised deep learning[J]. J Biophotonics, 2021, 14(1): e202000282. doi: 10.1002/jbio.202000282 CrossRef Google Scholar
[5]	芦焱琦, 陈明惠, 秦楷博, 等. 基于金字塔长程Transformer的OCT图像超分辨率重建[J]. 中国激光, 2023, 50(15): 1507107. doi: 10.3788/CJL230624 CrossRef Google Scholar Lu Y Q, Chen M H, Qin K B, et al. Super-resolution reconstruction of OCT image based on pyramid long-range transformer[J]. Chin J Lasers, 2023, 50(15): 1507107. doi: 10.3788/CJL230624 CrossRef Google Scholar
[6]	柯舒婷, 陈明惠, 郑泽希, 等. 生成对抗网络对OCT视网膜图像的超分辨率重建[J]. 中国激光, 2022, 49(15): 1507203. doi: 10.3788/CJL202249.1507203 CrossRef Google Scholar Ke S T, Chen M H, Zheng Z X, et al. Super-resolution reconstruction of optical coherence tomography retinal images by generating adversarial network[J]. Chin J Lasers, 2022, 49(15): 1507203. doi: 10.3788/CJL202249.1507203 CrossRef Google Scholar
[7]	Ma Y H, Chen X J, Zhu W F, et al. Speckle noise reduction in optical coherence tomography images based on edge-sensitive cGAN[J]. Biomed Opt Express, 2018, 9(11): 5129−5146. doi: 10.1364/BOE.9.005129 CrossRef Google Scholar
[8]	汪荣贵, 雷辉, 杨娟, 等. 基于自相似特征增强网络结构的图像超分辨率重建[J]. 光电工程, 2022, 49(5): 210382. doi: 10.12086/oee.2022.210382 CrossRef Google Scholar Wang R G, Lei H, Yang J. Self-similarity enhancement network for image super-resolution[J]. Opto-Electron Eng, 2022, 49(5): 210382. doi: 10.12086/oee.2022.210382 CrossRef Google Scholar
[9]	Ma C, Rao Y M, Cheng Y, et al. Structure-preserving super resolution with gradient guidance[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7766–7775. https://doi.org/10.1109/CVPR42600.2020.00779. Google Scholar
[10]	Park S J, Son H, Cho S, et al. SRFeat: single image super-resolution with feature discrimination[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 455–471. https://doi.org/10.1007/978-3-030-01270-0_27. Google Scholar
[11]	Wang X T, Yu K, Wu S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 63–79. https://doi.org/10.1007/978-3-030-11021-5_5. Google Scholar
[12]	Yao G Q, Li Z, Bhanu B, et al. MTKDSR: multi-teacher knowledge distillation for super resolution image reconstruction[C]//Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), 2022: 352–358. https://doi.org/10.1109/ICPR56361.2022.9956250. Google Scholar
[13]	Shu C Y, Liu Y F, Gao J F, et al. Channel-wise knowledge distillation for dense prediction[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision, 2021: 5291–5300. https://doi.org/10.1109/ICCV48922.2021.00526. Google Scholar
[14]	Zhao T L, Hu L, Zhang Y M, et al. Super-resolution network with information distillation and multi-scale attention for medical CT image[J]. Sensors, 2021, 21(20): 6870. doi: 10.3390/s21206870 CrossRef Google Scholar
[15]	Wang Q L, Wu B G, Zhu P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155. Google Scholar
[16]	Zhou Z D, Zhuge C R, Guan X W, et al. Channel distillation: channel-wise attention for knowledge distillation[Z]. arXiv: 2006.01683, 2020. https://doi.org/10.48550/arXiv.2006.01683. Google Scholar
[17]	Yoo J, Ahn N, Sohn K A. Rethinking data augmentation for image super-resolution: a comprehensive analysis and a new strategy[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 8372–8381. https://doi.org/10.1109/CVPR42600.2020.00840. Google Scholar
[18]	Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of the 13th European Conference on Computer Vision, 2014: 184–199. https://doi.org/10.1007/978-3-319-10593-2_13. Google Scholar
[19]	Hui Z, Gao X B, Yang Y C, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia, 2019: 2024–2032. https://doi.org/10.1145/3343031.3351084. Google Scholar
[20]	Liu J, Tang J, Wu G S. Residual feature distillation network for lightweight image super-resolution[C]//Proceedings of the 16th European Conference on Computer Vision, 2020: 41–55. https://doi.org/10.1007/978-3-030-67070-2_2. Google Scholar
[21]	Wang Y B, Lin S H, Qu Y Y, et al. Towards compact single image super-resolution via contrastive self-distillation[C]// Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021: 1122–1128. https://doi.org/10.24963/ijcai.2021/155. Google Scholar
[22]	Bogunović H, Venhuizen F, Klimscha S, et al. RETOUCH: the retinal OCT fluid detection and segmentation benchmark and challenge[J]. IEEE Trans Med Imaging, 2019, 38(8): 1858−1874. doi: 10.1109/TMI.2019.2901398 CrossRef Google Scholar

Overview

Overview

Optical coherence technology (OCT), which is widely used in the diagnosis of ophthalmic diseases, can reconstruct three-dimensional cross-sectional images inside biological tissues through the mutual interference of weakly coherent light. However, due to the inevitable scattering of weakly coherent light when it enters the tissue, there is speckle noise in the OCT retinal image, which covers up the subtle and very important details in the image. Secondly, unconscious movements such as eye movements (drift, tremors, and micro jumps), head movements, and cardiopulmonary system during the image acquisition process can lead to artifacts in OCT images, affecting clinical diagnosis and interfering with subsequent automated analysis of images. To solve the problem of existing OCT super-resolution networks being solely focused on reconstruction accuracy and perceptual quality, reduce the model complexity of the network, and be more suitable for clinical applications, this paper proposes a multi teacher knowledge distillation network MK-OCT for OCT image super-resolution. Through knowledge distillation, the student network can combine the different abilities of the teacher network to achieve balance, lightweight, and efficiency. At the same time, an efficient channel distillation method ECD was proposed, which enables the student network to extract rich channel attention information from the middle layer of the teacher network and transmit it to the middle layer of the student network in the form of a loss function, improving model performance without increasing the parameters and computational complexity of the student network. During the training process, both the student network and the teacher network use low-resolution images as input, and after the three networks respectively obtain reconstructed images, different loss functions are used to calculate the loss between the output images of each network. This allows the student network to simultaneously learn both reconstruction accuracy and perceptual quality from the two teacher networks. In addition, the student network additionally uses contrastive learning, which can provide external knowledge with upper and lower bounds, reducing the optimization space for the OCT image super-resolution task, thereby further improving the performance of the student network. We compared our model to five classic lightweight super-resolution reconstruction models, namely SRCNN, CSD, IMDN, and RFDN. Experiments have verified the effectiveness and superiority of MK-OCT in OCT image super-resolution reconstruction. At the same time, our research group also conducted ablation experiments, which further confirmed the effectiveness of multi teacher knowledge distillation. The generalization performance experiment also proves that the MK-OCT model has a good generalization ability.