The detection method for grab of portal crane based on deep learning

Zhang Wenming; Liu Xiangyang; Li Haibin; Li Yaqian

doi:10.12086/oee.2021.200062

Article navigation > Opto-Electronic Engineering > 2021 Vol. 48 > No. 1 > 200062

Next Article Previous Article

Zhang W M, Liu X Y, Li H B, et al. The detection method for grab of portal crane based on deep learning[J]. Opto-Electron Eng, 2021, 48(1): 200062. doi: 10.12086/oee.2021.200062

Citation:

Zhang W M, Liu X Y, Li H B, et al. The detection method for grab of portal crane based on deep learning[J]. Opto-Electron Eng, 2021, 48(1): 200062. doi: 10.12086/oee.2021.200062

The detection method for grab of portal crane based on deep learning

1.
School of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei 066004, China
2.
Key Laboratory of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao, Hebei 066004, China

Fund Project: Natural Science Foundation of Hebei Province (F2019203195)

More Information

^*Corresponding author: Liu Xiangyang, E-mail: 2041203253@qq.com

Received Date 25 February 2020

Revised Date 09 May 2020

Published Date 15 January 2021

Abstract

Abstract

In order to solve the problems of low work efficiency and safety caused by the inability of human eyes to accurately determine the position of the grab during the loading and unloading of dry bulk cargo by portal crane, a method of grab detection based on deep learning is proposed for the first time. The improved deep convolution neural network (YOLOv3-tiny) is used to train and test on the data set of grab, and then to learn its internal feature representation. The experimental results show that the detection method based on deep learning can achieve a detection speed of 45 frames per second and a recall rate of 95.78%. It can meet the real-time and accuracy of detection, and improve the safety and efficiency of work in the industrial field.
- grab detection /
- deep learning /
- YOLOv3-tiny /
- SPP /
- inverted residual group /
- dilated convolution

FullText(HTML)

References

[1]	陈英明. 中国港口现状及未来走势[J]. 中国水运, 2019(6): 7. Google Scholar Chen Y M. Current situation and future trend of Chinese ports[J]. China Water Transp, 2019(6): 7. Google Scholar
[2]	邢小健. 对国内港口散货装卸行业的一些思考[J]. 起重运输机械, 2019(10): 1. Google Scholar Xing X J. Some thoughts on bulk cargo handling industry of domestic port[J]. Hoist Con Mach, 2019(10): 1. Google Scholar
[3]	季本山. 现代门机抓斗控制程序的设计[J]. 南通航运职业技术学院学报, 2012, 11(4): 76–79. Google Scholar Ji B S. A study on the design of grab bucket control program[J]. J Nantong Vocational Tech Shipping Coll, 2012, 11(4): 76–79. Google Scholar
[4]	姚筑宇. 基于深度学习的目标检测研究与应用[D]. 北京: 北京邮电大学, 2019. Google Scholar Yao Z Y. Research on the application of object detection technology based on deep learning algorithm[D]. Beijing: Beijing University of Posts and Telecommunications, 2019. Google Scholar
[5]	Manana M, Tu C L, Owolawi P A. A survey on vehicle detection based on convolution neural networks[C]//Proceedings of the 3rd IEEE International Conference on Computer and Communications, 2017: 1751–1755. Google Scholar
[6]	戴伟聪, 金龙旭, 李国宁, 等. 遥感图像中飞机的改进YOLOv3实时检测算法[J]. 光电工程, 2018, 45(12): 180350. doi: 10.12086/oee.2018.180350 CrossRef Google Scholar Dai W C, Jin L X, Li G N, et al. Real-time airplane detection algorithm in remote-sensing images based on improved YOLOv3[J]. Opto-Electron Eng, 2018, 45(12): 180350. doi: 10.12086/oee.2018.180350 CrossRef Google Scholar
[7]	Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 580–587. Google Scholar
[8]	Girshick R. Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 1440–1448. Google Scholar
[9]	Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 91–99. Google Scholar
[10]	Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779–788. Google Scholar
[11]	Redmon J, Farhadi A. YOLOv3: an incremental improvement[Z]. arXiv: 1804.02767, 2018. Google Scholar
[12]	He K M, Zhang X Y, Ren S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans Pattern Anal Mach Intell, 2014, 37(9): 1904–1916. Google Scholar
[13]	Howard A G, Zhu M L, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[Z]. arXiv: 1704.04861, 2017. Google Scholar
[14]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 936–944. Google Scholar
[15]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778. Google Scholar
[16]	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[Z]. arXiv: 1502.03167, 2015. Google Scholar
[17]	LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Comput, 1989, 1(4): 541–551. Google Scholar
[18]	Maas A L, Hannum A Y, Ng A Y. Rectifier nonlinearities improve neural network acoustic models[C]//Proceedings of the 30th International Conference on Machine Learning, 2013. Google Scholar
[19]	Sandler M, Howard A, Zhu M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 4510–4520. Google Scholar
[20]	Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[Z]. arXiv: 1511.07122, 2015. Google Scholar

Overview

Overview

Overview: In recent years, with the vigorous development of the port industry, the port throughput is increasing, and the demand for loading and unloading dry bulk cargo is also increasing. At present, the method adopted is mainly man-made operation. The driver sits in the cab of the gantry crane, and observes whether the grab reaches the proper position to grab or release the dry bulk by naked eyes, and judges when to lower or raise the steel wire rope on the grab. Then there will be the following problems: first, because the human eyes are far away from the goods, the wire rope is easy to be over released when the driver releases the grab. A few seconds are wasted in one operation cycle, and a lot of time is wasted and a lot of idle work is produced in multiple operation cycles. Second, the driver's long-term operation will lead to eyestrain, which will lead to misjudgment and over the release. It is not conducive to the development of the enterprise, because, in addition to time-consuming and labor-consuming, it will increase the input cost of the company. So how to accurately detect the position of grab and make it more efficient to load and unload cargo has become an urgent problem for the port industry. In order to solve the problems of low work efficiency and safety caused by the inability of human eyes to accurately determine the position of the grab during the loading and unloading of dry bulk cargo by portal crane, a method of grab detection based on deep learning is proposed for the first time. The improved deep convolution neural network (YOLOv3-tiny) is used to train and test on the data set of grab, and then to learn its internal feature representation. The experimental results show that the detection method based on deep learning can achieve a detection speed of 45 frames per second, a recall rate of 95.78%, and a false detection rate of 0. Although the accuracy of detection is lower than Faster RCNN, the detection speed is 225 times faster than Faster RCNN. Compared with the original model YOLOv3-tiny, the detection speed of the improved network model in this paper is slightly reduced, but the detection accuracy has been greatly improved. Through the contrast test, we can see that the YOLOv3 network model is not as good as the improved network in the two indicators of mAP and FPS. Therefore, for the real-time detection task of gantry crane grab, the improved model in this paper performs better. It can meet the real-time and accuracy of detection, and improve the safety and efficiency of work in the industrial field.