Lightweight YOLOv5 sonar image object detection algorithm and implementation based on ZYNQ

Zhao Dongdong; Xie Dunhan; Chen Peng; Liang Ronghua; Shen Yi; Guo Xinxin

doi:10.12086/oee.2024.230284

Article navigation > Opto-Electronic Engineering > 2024 Vol. 51 > No. 1 > 230284

Next Article Previous Article

Zhao D D, Xie D H, Chen P, et al. Lightweight YOLOv5 sonar image object detection algorithm and implementation based on ZYNQ[J]. Opto-Electron Eng, 2024, 51(1): 230284. doi: 10.12086/oee.2024.230284

Citation:

Zhao D D, Xie D H, Chen P, et al. Lightweight YOLOv5 sonar image object detection algorithm and implementation based on ZYNQ[J]. Opto-Electron Eng, 2024, 51(1): 230284. doi: 10.12086/oee.2024.230284

Lightweight YOLOv5 sonar image object detection algorithm and implementation based on ZYNQ

1.
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, Zhejiang 310023, China
2.
Institute of Deep-sea Science and Engineering, Chinese Academy of Sciences, Sanya, Hainan 572000, China

Fund Project: Project supported by National Natural Science Foundation of China (62371421, 62001418, 62005245, U1909203), and Zhejiang Provincial Natural Science Foundation of China (LQ21F010011)

More Information

^*Corresponding author: chenpeng@zjut.edu.cn

Received Date 21 November 2023

Revised Date 10 January 2024

Accepted Date 10 January 2024

Published Date 25 January 2024

Abstract

Abstract

To address the problems of blurring and insufficient sample size in sonar images, an improved sonar image target detection algorithm is proposed based on YOLOv5. The algorithm uses geometric filtering, vertical flipping, and other methods to enhance the sonar image dataset. The fusion attention mechanism module is added to make the algorithm better focus on the features of small targets in sonar images. At the same time, in response to the problem that most target detection algorithms currently run on the cloud and cannot achieve real-time sonar image detection, this paper uses lightweight network replacement and NCNN edge porting technology. It adopts the GSConv module in the neck network to successfully transplant the algorithm to the ZYNQ platform, realizing real-time detection of sonar images on the embedded end. After experiments, the algorithm proposed in this paper reduced the parameter quantity by 56%, increasing map50 and map50-95 by 2.2% and 2.5%, respectively. The algorithm’s performance has significantly improved, proving that the method proposed has certain feasibility and effectiveness in lightweight sonar image target detection tasks.
- underwater target detection /
- YOLO /
- ZYNQ /
- sonar image /
- deep learning /
- lightweight

FullText(HTML)

References

[1]	郭戈, 王兴凯, 徐慧朴. 基于声呐图像的水下目标检测、识别与跟踪研究综述[J]. 控制与决策, 2018, 33(5): 906−922. doi: 10.13195/j.kzyjc.2017.1678 CrossRef Google Scholar Guo G, Wang X K, Xu H P. Review on underwater target detection, recognition and tracking based on sonar image[J]. Control Decis, 2018, 33(5): 906−922. doi: 10.13195/j.kzyjc.2017.1678 CrossRef Google Scholar
[2]	Petrich J, Brown M F, Pentzer J L, et al. Side scan sonar based self-localization for small autonomous underwater vehicles[J]. Ocean Eng, 2018, 161: 221−226. doi: 10.1016/j.oceaneng.2018.04.095 CrossRef Google Scholar
[3]	Huang S W, Chen E, Guo J. Efficient seafloor classification and submarine cable route design using an autonomous underwater vehicle[J]. IEEE J Oceanic Eng, 2018, 43(1): 7−18. doi: 10.1109/JOE.2017.2686558 CrossRef Google Scholar
[4]	Park C, Kim Y, Lee H, et al. Development of a 2 MHz sonar sensor for inspection of bridge substructures[J]. Sensors (Basel), 2018, 18(4): 1222. doi: 10.3390/s18041222 CrossRef Google Scholar
[5]	陈朋, 蔡烜伟, 赵冬冬, 等. 基于自适应BM3D的侧扫声纳图像散斑降噪[J]. 光电工程, 2020, 47(7): 190580. doi: 10.12086/oee.2020.190580 CrossRef Google Scholar Chen P, Cai X W, Zhao D D, et al. Despeckling for side-scan sonar images based on adaptive block-matching and 3D filtering[J]. Opto-Electron Eng, 2020, 47(7): 190580. doi: 10.12086/oee.2020.190580 CrossRef Google Scholar
[6]	Abu A, Diamant R. A statistically-based method for the detection of underwater objects in sonar imagery[J]. IEEE Sensors J, 2019, 19(16): 6858−6871. doi: 10.1109/JSEN.2019.2912325 CrossRef Google Scholar
[7]	Negahdaripour S. Application of forward-scan sonar stereo for 3-D scene reconstruction[J]. IEEE J Oceanic Eng, 2020, 45(2): 547−562. doi: 10.1109/JOE.2018.2875574 CrossRef Google Scholar
[8]	Shang Z G, Zhao C H, Wan J. Application of multi-resolution analysis in sonar image denoising[J]. J Sys Eng Electron, 2008, 19(6): 1082−1089. doi: 10.1016/S1004-4132(08)60201-7 CrossRef Google Scholar
[9]	Jin Y, Ku B, Ahn J, et al. Nonhomogeneous noise removal from side-scan sonar images using structural sparsity[J]. IEEE Geosci Remote Sens Lett, 2019, 16(8): 1215−1219. doi: 10.1109/LGRS.2019.2895843 CrossRef Google Scholar
[10]	Wang Z, Zhang S W, Huang W Z, et al. Sonar image target detection based on adaptive global feature enhancement network[J]. IEEE Sensors J, 2022, 22(2): 1509−1530. doi: 10.1109/JSEN.2021.3131645 CrossRef Google Scholar
[11]	Mignotte M, Collet C, Pérez P, et al. Markov random field and fuzzy logic modeling in sonar imagery: application to the classification of underwater floor[J]. Comput Vis Image Underst, 2000, 79(1): 4−24. doi: 10.1006/cviu.2000.0844 CrossRef Google Scholar
[12]	Köhntopp D, Lehmann B, Kraus D, et al. Classification and localization of naval mines with superellipse active contours[J]. IEEE J Ocean Eng, 2019, 44(3): 767−782. doi: 10.1109/JOE.2018.2835218 CrossRef Google Scholar
[13]	Valdenegro-Toro M. End-to-end object detection and recognition in forward-looking sonar images with convolutional neural networks[C]//2016 IEEE/OES Autonomous Underwater Vehicles (AUV), 2016: 144–150. https://doi.org/10.1109/AUV.2016.7778662 Google Scholar
[14]	Williams D P. Underwater target classification in synthetic aperture sonar imagery using deep convolutional neural networks[C]//2016 23rd International Conference on Pattern Recognition (ICPR), 2016: 2497–2502. https://doi.org/10.1109/ICPR.2016.7900011. Google Scholar
[15]	Kim J, Yu S C. Convolutional neural network-based real-time ROV detection using forward-looking sonar image[C]//2016 IEEE/OES Autonomous Underwater Vehicles (AUV), 2016: 396–400. https://doi.org/10.1109/AUV.2016.7778702. Google Scholar
[16]	刘桂雄, 刘思洋, 吴俊芳, 等. 基于深度学习的机器视觉目标检测算法及在票据检测中应用[J]. 中国测试, 2019, 45(5): 1−9. doi: 10.11857/j.issn.1674-5124.2019030130 CrossRef Google Scholar Liu G X, Liu S Y, Wu J F, et al. Machine vision object detection algorithm based on deep learning and application in banknote detection[J]. China Measur Test Technol, 2019, 45(5): 1−9. doi: 10.11857/j.issn.1674-5124.2019030130 CrossRef Google Scholar
[17]	Harikrishnan J, Sudarsan A, Sadashiv A, et al. Vision-face recognition attendance monitoring system for surveillance using deep learning technology and computer vision[C]//2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), 2019: 1–5. https://doi.org/10.1109/ViTECoN.2019.8899418. Google Scholar
[18]	李宝奇, 黄海宁, 刘纪元, 等. 基于改进SSD的合成孔径声呐图像水下多尺度目标轻量化检测模型[J]. 电子与信息学报, 2021, 43(10): 2854−2862. doi: 10.11999/JEIT201042 CrossRef Google Scholar Li B Q, Huang H N, Liu J Y, et al. Synthetic aperture sonar underwater multi-scale target efficient detection model based on improved single shot detector[J]. J Electron Inf Technol, 2021, 43(10): 2854−2862. doi: 10.11999/JEIT201042 CrossRef Google Scholar
[19]	陈龙, 张建林, 彭昊, 等. 多尺度注意力与领域自适应的小样本图像识别[J]. 光电工程, 2023, 50(4): 220232. doi: 10.12086/oee.2023.220232 CrossRef Google Scholar Chen L, Zhang J L, Peng H, et al. Few-shot image classification via multi-scale attention and domain adaptation[J]. Opto-Electron Eng, 2023, 50(4): 220232. doi: 10.12086/oee.2023.220232 CrossRef Google Scholar
[20]	葛锡云, 魏柠阳, 周宏坤, 等. 基于侧扫声呐的水下小目标检测技术研究[J]. 数字海洋与水下攻防, 2023, 6(2): 155−161. doi: 10.19838/j.issn.2096-5753.2023.02.004 CrossRef Google Scholar Ge X Y, Wei N Y, Zhou H K, et al. Research on small underwater target detection technology based on side-scan sonar[J]. Digit Ocean Underwater Warf, 2023, 6(2): 155−161. doi: 10.19838/j.issn.2096-5753.2023.02.004 CrossRef Google Scholar
[21]	Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: optimal speed and accuracy of object detection[Z]. arXiv: 2004.10934, 2020. https://doi.org/10.48550/arXiv.2004.10934. Google Scholar
[22]	Singh A, Kalaichelvi V, Karthikeyan R. Performance analysis of object detection algorithms for robotic welding applications in planar environment[J]. Int J Comput Integr Manuf, 2023, 36(7): 1083−1108. doi: 10.1080/0951192X.2022.2162601 CrossRef Google Scholar
[23]	陈旭, 彭冬亮, 谷雨. 基于改进YOLOv5s的无人机图像实时目标检测[J]. 光电工程, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372 CrossRef Google Scholar Chen X, Peng D L, Gu Y. Real-time object detection for UAV images based on improved YOLOv5s[J]. Opto-Electron Eng, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372 CrossRef Google Scholar
[24]	Ge Z, Liu S T, Wang F, et al. YOLOX: exceeding YOLO series in 2021[Z]. arXiv: 2107.08430, 2021. https://doi.org/10.48550/arXiv.2107.08430. Google Scholar
[25]	Ying B Y, Xu Y C, Zhang S, et al. Weed detection in images of carrot fields based on improved YOLO v4[J]. Traitem Signal, 2021, 38(2): 341−348. doi: 10.18280/ts.380211 CrossRef Google Scholar
[26]	黄磊, 杨媛, 杨成煜, 等. FS-YOLOv5: 轻量化红外目标检测方法[J]. 计算机工程与应用, 2023, 59(9): 215−224. doi: 10.3778/j.issn.1002-8331.2210-0487 CrossRef Google Scholar Huang L, Yang Y, Yang C Y, et al. FS-YOLOv5: lightweight infrared rode target detection method[J]. Comput Eng Appl, 2023, 59(9): 215−224. doi: 10.3778/j.issn.1002-8331.2210-0487 CrossRef Google Scholar
[27]	Li H L, Li J, Wei H B, et al. Slim-neck by GSConv: a better design paradigm of detector architectures for autonomous vehicles[Z]. arXiv: 2206.02424, 2022. https://doi.org/10.48550/arXiv.2206.02424. Google Scholar
[28]	奉志强, 谢志军, 包正伟, 等. 基于改进YOLOv5的无人机实时密集小目标检测算法[J]. 航空学报, 2023, 44(7): 327106. doi: 10.7527/s1000-6893.2022.27106 CrossRef Google Scholar Feng Z Q, Xie Z J, Bao Z W, et al. Real-time dense small object detection algorithm for UAV based on improved YOLOv5[J]. Acta Aeronaut Astronaut Sin, 2023, 44(7): 327106. doi: 10.7527/s1000-6893.2022.27106 CrossRef Google Scholar
[29]	Zhu X K, Lyu S C, Wang X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops, 2021: 2778–2788. https://doi.org/10.1109/ICCVW54120.2021.00312. Google Scholar
[30]	Niu Z Y, Zhong G Q, Yu H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48−62. doi: 10.1016/j.neucom.2021.03.091 CrossRef Google Scholar
[31]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 7132–7141. https://doi.org/10.1109/CVPR.2018.00745. Google Scholar
[32]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 3–19. https://doi.org/10.1007/978-3-030-01234-2_1. Google Scholar
[33]	Yang L X, Zhang R Y, Li L D, et al. SimAM: a simple, parameter-free attention module for convolutional neural networks[C]//Proceedings of the 38th International Conference on Machine Learning, 2021: 11863–11874. Google Scholar
[34]	Zheng Z H, Wang P, Liu W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020: 12993–13000. https://doi.org/10.1609/aaai.v34i07.6999. Google Scholar
[35]	袁明阳, 宋亚林, 张潮, 等. 基于GA-RetinaNet的水下目标检测[J]. 计算机系统应用, 2023, 32(6): 80−90. doi: 10.15888/j.cnki.csa.009134 CrossRef Google Scholar Yuan M Y, Song Y L, Zhang C, et al. Underwater object detection based on GA-RetinaNet[J]. Comput Syst Appl, 2023, 32(6): 80−90. doi: 10.15888/j.cnki.csa.009134 CrossRef Google Scholar
[36]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017: 2999–3007. https://doi.org/10.1109/ICCV.2017.324. Google Scholar

Overview

Overview

In the 21st century, mankind has entered a period of large-scale development and utilization of the ocean. With the strategic mission of becoming a maritime power, we need to improve our ability to develop and utilize marine resources, which requires a large amount of accurate marine environment data to carry out tasks such as underwater target detection and seabed resource detection. Underwater target detection technology is crucial for underwater target positioning and underwater resource exploration. Underwater target detection can be achieved through different imaging technologies, but sonar is currently the most commonly used detection method because it can operate reliably in low visibility conditions. Due to the complexity of underwater acoustic channels, as well as the loss and scattering of sound waves themselves, the image quality and resolution obtained by imaging sonar are low, accompanied by a large amount of speckle noise and unclear edges, which also seriously affects the follow-up of sonar images. Currently, most sonar image target detection algorithms run on the cloud and cannot achieve real-time sonar image detection. This paper proposes a lightweight YOLOv5 sonar image target detection algorithm based on ZYNQ to achieve real-time detection of small target images on the embedded side of sonar equipment. First, geometric filtering, vertical flipping, and other methods are used to perform data enhancement on the sonar image dataset, adding a fusion attention mechanism module allows the algorithm to better focus on the characteristics of small targets in sonar images. At the same time, in order to solve the problem that most target detection algorithms currently run in the cloud and cannot achieve real-time sonar image detection, this paper uses replacement lightweight networks and NCNN edge-end transplantation technology, and uses the GSConv module in the neck network to convert the algorithm successfully ported to ZYNQ platform. The sonar image detection system is independently designed. The PL side uses the wave velocity formation algorithm to generate images, and the PS side realizes the embedded side real-time detection of sonar images. After experiments, the algorithm proposed in this article reduced the calculation amount by 56%, while map50 and map50-95 increased by 2.2% and 2.5%, respectively. The performance of the improved algorithm has been significantly improved, proving that the method proposed in this article has certain feasibility and effectiveness in lightweight sonar image target detection tasks.