Surface defect detection of solar cells using local and global feature fusion

Tao Zhiyong; He Yan; Lin Sen; Yi Tingjun; Zhang Yaosheng

doi:10.12086/oee.2024.230292

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 1 > 230292

Next Article Previous Article

Tao Z Y, He Y, Lin S, et al. Surface defect detection of solar cells using local and global feature fusion[J]. Opto-Electron Eng, 2024, 51(1): 230292. doi: 10.12086/oee.2024.230292

Citation:

Tao Z Y, He Y, Lin S, et al. Surface defect detection of solar cells using local and global feature fusion[J]. Opto-Electron Eng, 2024, 51(1): 230292. doi: 10.12086/oee.2024.230292

Surface defect detection of solar cells using local and global feature fusion

1.
School of Electronic and Information Engineering, Liaoning Technical University, Huludao, Liaoning 125105, China
2.
School of Automation and Electrical Engineering, Shenyang Ligong University, Shenyang, Liaoning 110159, China

Fund Project: Project supported by Applied Basic Research Projects of Department of Science & Technology of Liaoning Province(2022JH2/101300274), Research Project on Teaching Reform of Graduate Education in Liaoning Province (LNYJG2023117), and Basic Research Project of Liaoning Provincial Department of Education (LJKMZ20220676)

More Information

^*Corresponding author: 2575561115@qq.com

Received Date 01 December 2023

Revised Date 02 February 2024

Accepted Date 02 February 2024

Published Date 25 January 2024

Abstract

Abstract

The surface defects of solar cells exhibit significant intra-class differences, minor inter-class differences, and complex background features, making high-precision identification of surface defects a challenging task. This paper proposes a Convolutional -Vision Transformer Network (CViT-Net) that combines local and global features to address this issue. First, a Ghost-Convolution two-fusion (G-C2F) module is used to extract local features of the solar cell panel defects. Then, a coordinate attention mechanism is introduced to emphasize defect features and suppress background features. Finally, a Ghost-Vision Transformer (G-ViT) module is constructed to fuse local and global features of the solar cell panel defects. Meanwhile, CViT-Net-S and CViT-Net-L network structures are provided for low-resource and high-resource environments. Experimental results show that compared to classic lightweight networks such as MobileVit, MobileNetV3, and GhostNet, CViT-Net-S improves the classification accuracy of solar cell panels by 1.4%, 2.3%, and 1.3%, respectively, and improves the mAP50 for defect detection by 2.7%, 0.3%, and 0.8% respectively. Compared to ResNet50 and RegNet, CViT-Net-L enhances the classification accuracy by 0.72% and 0.7%, respectively, and improves the mAP50 for defect detection by 3.9% and 1.3%, respectively. Compared to advanced YOLOV6, YOLOV7, and YOLOV8 detection networks, CViT-Net-S and CViT-Net-L structures, as backbone networks, still maintain good detection performance in terms of mAP and mAP50 metrics, demonstrating the application value of the proposed algorithm in the field of solar cell panel surface defect detection.
- deep learning /
- feature fusion /
- solar cells /
- defect classification /
- defect detection

FullText(HTML)

References

[1]	余星, 严俊森, 吴志鹏, 等. 激光微纳制造太阳能海水淡化材料研究进展[J]. 光电工程, 2022, 49(1): 210313. doi: 10.12086/oee.2022.210313 CrossRef Google Scholar Yu X, Yan J S, Wu Z P, et al. Research progress of solar desalination materials produced by laser micro-nano fabrication[J]. Opto-Electron Eng, 2022, 49(1): 210313. doi: 10.12086/oee.2022.210313 CrossRef Google Scholar
[2]	Herraiz Á H, Marugán A P, Márquez F P G. Photovoltaic plant condition monitoring using thermal images analysis by convolutional neural network-based structure[J]. Renew Energy, 2020, 153: 334−348. doi: 10.1016/j.renene.2020.01.148 CrossRef Google Scholar
[3]	曾德宇, 梁泽逍, 吴宗泽. 基于加权核范数和L_{2, 1}范数的最优均值线性分类器[J]. 电子与信息学报, 2022, 44(5): 1602−1609. doi: 10.11999/JEIT211434 CrossRef Google Scholar Zeng D Y, Liang Z X, Wu Z Z. Optimal mean linear classifier via weighted nuclear norm and L_{2, 1} norm[J]. J Electron Inf Technol, 2022, 44(5): 1602−1609. doi: 10.11999/JEIT211434 CrossRef Google Scholar
[4]	Juan R O S, Kim J. Photovoltaic cell defect detection model based-on extracted electroluminescence images using SVM classifier[C]//2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan, 2020: 578–582. https://doi.org/10.1109/ICAIIC48513.2020.9065065. Google Scholar
[5]	王超, 蒋晓瑜, 柳效辉. 基于电致发光成像理论的硅太阳电池缺陷检测[J]. 光电子·激光, 2011, 22(9): 1332–1336. Google Scholar Wang C, Jiang X Y, Liu X H. Defect detection in crystalline silicon solar cells based on electroluminescence imaging[J]. J Optoelectron·Laser‚ 2011, 22(9): 1332–1336. Google Scholar
[6]	Firuzi K, Vakilian M, Phung B T, et al. Partial discharges pattern recognition of transformer defect model by LBP & HOG features[J]. IEEE Trans Power Delivery, 2019, 34(2): 542−550. doi: 10.1109/TPWRD.2018.2872820 CrossRef Google Scholar
[7]	Juan R O S, Kim J. Photovoltaic cell defect detection model based-on extracted electroluminescence images using SVM classifier[C]//2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC). IEEE, 2020: 578-582. https://doi.org/10.1109/ICAIIC48513.2020.9065065. Google Scholar
[8]	李原, 李燕君, 刘进超, 等. 基于改进Res-UNet网络的钢铁表面缺陷图像分割研究[J]. 电子与信息学报, 2022, 44(5): 1513−1520. doi: 10.11999/JEIT211350 CrossRef Google Scholar Li Y, Li Y J, Liu J C, et al. Research on segmentation of steel surface defect images based on improved Res-UNet network[J]. J Electron Inf Technol, 2022, 44(5): 1513−1520. doi: 10.11999/JEIT211350 CrossRef Google Scholar
[9]	Masita K, Hasan A, Shongwe T. 75MW AC PV module field anomaly detection using drone-based IR orthogonal images with Res-CNN3 detector[J]. IEEE Access, 2022, 10: 83711−83722. doi: 10.1109/ACCESS.2022.3194547 CrossRef Google Scholar
[10]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90. Google Scholar
[11]	Du B L, He Y G, He Y Z, et al. Intelligent classification of silicon photovoltaic cell defects based on eddy current thermography and convolution neural network[J]. IEEE Trans Industr Inform, 2020, 16(10): 6242−6251. doi: 10.1109/TII.2019.2952261 CrossRef Google Scholar
[12]	Chen H Y, Pang Y, Hu Q D, et al. Solar cell surface defect inspection based on multispectral convolutional neural network[J]. J Intell Manuf, 2020, 31(2): 453−468. doi: 10.1007/s10845-018-1458-z CrossRef Google Scholar
[13]	Su B Y, Chen H Y, Zhou Z. BAF-detector: an efficient CNN-based detector for photovoltaic cell defect detection[J]. IEEE Trans Ind Electron, 2022, 69(3): 3161−3171. doi: 10.1109/TIE.2021.3070507 CrossRef Google Scholar
[14]	陈海永, 赵鹏, 闫皓炜. 融合注意力的多尺度Faster RCNN的裂纹检测[J]. 光电工程, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112 CrossRef Google Scholar Chen H Y, Zhao P, Yan H W. Crack detection based on multi-scale Faster RCNN with attention[J]. Opto-Electron Eng, 2021, 48(1): 200112. doi: 10.12086/oee.2021.200112 CrossRef Google Scholar
[15]	Zhang M, Yin L J. Solar cell surface defect detection based on improved YOLO v5[J]. IEEE Access, 2022, 10: 80804−80815. doi: 10.1109/ACCESS.2022.3195901 CrossRef Google Scholar
[16]	陈旭, 彭冬亮, 谷雨. 基于改进YOLOv5s的无人机图像实时目标检测[J]. 光电工程, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372 CrossRef Google Scholar Chen X, Peng D L, Gu Y. Real-time object detection for UAV images based on improved YOLOv5s[J]. Opto-Electron Eng, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372 CrossRef Google Scholar
[17]	Howard A, Sandler M, Chen B, et al. Searching for MobileNetV3[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019: 1314–1324. https://doi.org/10.1109/ICCV.2019.00140. Google Scholar
[18]	Zhang X Y, Zhou X Y, Lin M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6848–6856. https://doi.org/10.1109/CVPR.2018.00716. Google Scholar
[19]	Koonce B. EfficientNet[M]//Koonce B. Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization. Berkeley: Apress, 2021: 109–123. https://doi.org/10.1007/978-1-4842-6168-2_10. Google Scholar
[20]	Han K, Wang Y H, Tian Q, et al. GhostNet: more features from cheap operations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020: 1577–1586. https://doi.org/10.1109/CVPR42600.2020.00165. Google Scholar
[21]	Radosavovic I, Kosaraju R P, Girshick R, et al. Designing network design spaces[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020: 10425–10433. https://doi.org/10.1109/CVPR42600.2020.01044. Google Scholar
[22]	Han K, Xiao A, Wu E H, et al. Transformer in transformer[C]//35th Conference on Neural Information Processing Systems, 2021: 15908–15919. Google Scholar
[23]	Wu H P, Xiao B, Codella N, et al. CvT: introducing convolutions to vision transformers[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, Canada, 2021: 22–31. https://doi.org/10.1109/ICCV48922.2021.00009. Google Scholar
[24]	d’Ascoli S, Touvron H, Leavitt M L, et al. ConViT: improving vision transformers with soft convolutional inductive biases[J]. J Stat Mech Theory Exp, 2022, 2022(11): 114005. doi: 10.1088/1742-5468/ac9830 CrossRef Google Scholar
[25]	Mehta S, Rastegari M. MobileViT: light-weight, general-purpose, and mobile-friendly vision transformer[C]//Tenth International Conference on Learning Representations, 2022. Google Scholar
[26]	Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, 2023: 7464–7475. https://doi.org/10.1109/CVPR52729.2023.00721. Google Scholar
[27]	Pawar N, Waghmare A, Pratap A, et al. Miniscule object detection in aerial images using YOLOR: a review[M]//Kumar S, Hiranwal S, Purohit S D, et al. Proceedings of International Conference on Communication and Computational Technologies: ICCCT 2022. Singapore: Springer, 2023: 697–708. https://doi.org/10.1007/978-981-19-3951-8_52. Google Scholar
[28]	Norkobil Saydirasulovich S, Abdusalomov A, Jamil M K, et al. A YOLOv6-based improved fire detection approach for smart city environments[J]. Sensors, 2023, 23(6): 3161. doi: 10.3390/s23063161 CrossRef Google Scholar
[29]	Talaat F M, Zaineldin H. An improved fire detection approach based on YOLO-v8 for smart cities[J]. Neural Comput Appl, 2023, 35(28): 20939−20954. doi: 10.1007/s00521-023-08809-1 CrossRef Google Scholar
[30]	Zhang Y C, Zhang W B, Yu J Y, et al. Complete and accurate holly fruits counting using YOLOX object detection[J]. Comput Electron Agric, 2022, 198: 107062. doi: 10.1016/j.compag.2022.107062 CrossRef Google Scholar
[31]	Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: 2261–2269. https://doi.org/10.1109/CVPR.2017.243. Google Scholar
[32]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: 936–944. https://doi.org/10.1109/CVPR.2017.106. Google Scholar
[33]	Liu S, Qi L, Qin H F, et al. Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 8759–8768. https://doi.org/10.1109/CVPR.2018.00913. Google Scholar
[34]	Hou Q B, Zhou D Q, Feng J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, USA, 2021: 13708–13717. https://doi.org/10.1109/CVPR46437.2021.01350. Google Scholar
[35]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7132–7141. https://doi.org/10.1109/CVPR.2018.00745. Google Scholar
[36]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, 2018: 3–19. https://doi.org/10.1007/978-3-030-01234-2_1. Google Scholar
[37]	Ouyang D L, He S, Zhang G Z, et al. Efficient multi-scale attention module with cross-spatial learning[C]//ICASSP 2023–2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, 2023: 1–5. https://doi.org/10.1109/ICASSP49357.2023.10096516. Google Scholar
[38]	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: 2818–2826. https://doi.org/10.1109/CVPR.2016.308. Google Scholar
[39]	Wu C C, Hung Y C. A study on electroluminescent phenomenon for solar cells test[C]//2018 1st IEEE International Conference on Knowledge Innovation and Invention (ICKII), Jeju, Korea (South), 2018: 172–175. https://doi.org/10.1109/ICKII.2018.8569147. Google Scholar

Overview

Overview

The methods employed for identifying surface defects on solar cell panels encompass traditional machine learning and deep learning. Traditional machine learning methods have advantages in defect recognition and well-established algorithms for detecting surface defects on solar cell panels. However, these methods encounter challenges, including extensive parameter tuning, issues with model robustness, suboptimal generalization performance, and reliance on engineers' subjective experience for defect discrimination in solar cell defect detection. Moreover, they need help adapting to prolonged manual labor. In contrast, deep learning methods face challenges from the high similarity of defect features on solar cell panels and the complexity of background features. Issues such as insufficient extraction of fine-grained defect features and feature loss during network deepening may arise, resulting in decreased detection accuracy. The surface defects on solar cell panels show significant intra-class and minimal inter-class differences, combined with a complex background. Therefore, achieving high-precision automatic detection of surface defects on solar cell panels becomes challenging. We utilize advanced techniques in deep learning and computer vision to address this issue. We propose a method named Convolutional-Vision Transformer Networks (CViT-Net), specifically designed to efficiently integrate local and global features for accurate defect detection in solar cell panels. The model initially utilizes a Ghost Focus (G-C2F) module to extract local features related to defects in solar cell panels. Subsequently, a coordinate attention mechanism is introduced to emphasize defect features and attenuate background features. Finally, we construct a Ghost Vision (G-ViT) module to integrate local and global features of defects in solar cell panels. To address various demands for detection accuracy and model parameterization, we introduce the CViT-Net-S structure with a parameter count of 5.6 M and the CViT-Net-L structure with a parameter count of 21.9 M, serving diverse practical applications in low-resource and high-resource environments, respectively. Experimental results illustrate the remarkable performance of our model in classifying and detecting defects in solar cell panels. Compared to lightweight models like MobileVit, MobileNetV3, and GhostNet, our CViT-Net-S model achieves accuracy improvements of 1.4%, 2.3%, and 1.3%, respectively, for defect classification in solar cell panels and mAP50 enhancements of 2.7%, 0.3%, and 0.8%, respectively, in defect detection. Compared to RecNet50 and RegNet, the CNN-ViT-L model demonstrates classification accuracy enhancements of 0.72% and 0.7% and mAP50 improvements of 3.9% and 1.3%, respectively. When compared to advanced object detection models like YOLOv6, YOLOv7, and YOLOv8, CViT-Net-S and CViT-Net-L, serving as backbone networks, continue to demonstrate robust detection performance in terms of mAP and mAP50 metrics. These results underscore the algorithm's significant practical value in the surface defect detection field of solar cell panels. In future work, we plan to extend the CViT-Net model for application in defect classification detection for other physical entities to meet diverse defect recognition needs.