Chen H S, Yao M H, Qu X Y. Pavement crack detection based on the U-shaped fully convolutional neural network[J]. Opto-Electron Eng, 2020, 47(12): 200036. doi: 10.12086/oee.2020.200036
Citation: Chen H S, Yao M H, Qu X Y. Pavement crack detection based on the U-shaped fully convolutional neural network[J]. Opto-Electron Eng, 2020, 47(12): 200036. doi: 10.12086/oee.2020.200036

Pavement crack detection based on the U-shaped fully convolutional neural network

    Fund Project: Supported by National Natural Science Foundation of China (61871350) and Zhejiang Provincial National Science Foundation of China (GG19E050005)
More Information
  • Crack detection is one of the most important works in the system of pavement management. Cracks do not have a certain shape and the appearance of cracks usually changes drastically in different lighting conditions, making it hard to be detected by the algorithm with imagery analytics. To address these issues, we propose an effective U-shaped fully convolutional neural network called UCrackNet. First, a dropout layer is added into the skip connection to achieve better generalization. Second, pooling indices is used to reduce the shift and distortion during the up-sampling process. Third, four atrous convolutions with different dilation rates are densely connected in the bridge block, so that the receptive field of the network could cover each pixel of the whole image. In addition, multi-level fusion is introduced in the output stage to achieve better performance. Evaluations on the two public CrackTree206 and AIMCrack datasets demonstrate that the proposed method achieves high accuracy results and good generalization ability.
  • 加载中
  • [1] http://www.zgjtb.com/2019-10/08/content_230254.htm.

    Google Scholar

    [2] Schnebele E, Tanyu B F, Cervone G, et al. Review of remote sensing methodologies for pavement management and assessment[J]. European Transport Research Review, 2015, 7(2): 7. doi: 10.1007/s12544-015-0156-6

    CrossRef Google Scholar

    [3] 张德津, 李清泉.公路路面快速检测技术发展综述[J].测绘地理信息, 2015, 40(1): 1-8. doi: 10.14188/j.2095-6045.2015.01.001

    CrossRef Google Scholar

    Zhang D J, Li Q Q. A review of pavement high speed detection technology[J]. Journal of Geomatics, 2015, 40(1): 1-8. doi: 10.14188/j.2095-6045.2015.01.001

    CrossRef Google Scholar

    [4] Shi Y, Cui L M, Qi Z Q, et al. Automatic road crack detection using random structured forests[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12): 3434-3445. doi: 10.1109/TITS.2016.2552248

    CrossRef Google Scholar

    [5] 徐威, 唐振民, 吕建勇.基于图像显著性的路面裂缝检测[J].中国图象图形学报, 2013, 18(1): 69-77. doi: 10.11834/jig.20130109

    CrossRef Google Scholar

    Xu W, Tang Z M, Lv J Y. Pavement crack detection based on image saliency[J]. Journal of Image and Graphics, 2013, 18(1): 69-77. doi: 10.11834/jig.20130109

    CrossRef Google Scholar

    [6] Zhang L, Yang F, Zhang Y D, et al. Road crack detection using deep convolutional neural network[C]//International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 2016: 3708-3712.

    Google Scholar

    [7] Cha Y J, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361-378. doi: 10.1111/mice.12263

    CrossRef Google Scholar

    [8] Maeda H, Sekimoto Y, Seto T, et al. Road damage detection and classification using deep neural networks with smartphone images[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1127-1141. doi: 10.1111/mice.12387

    CrossRef Google Scholar

    [9] Carr T A, Jenkins M D, Iglesias M I, et al. Road crack detection using a single stage detector based deep neural network[C]//2018 IEEE Workshop on Environmental, Energy and Structural Monitoring Systems, Salerno, Italy, 2018.

    Google Scholar

    [10] Bang S, Park S, Kim H, et al. Encoder-decoder network for pixel-level road crack detection in black-box images[J]. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(8): 713-727. doi: 10.1111/mice.12440

    CrossRef Google Scholar

    [11] Yang X C, Li H, Yu Y T, et al. Automatic pixel-level crack detection and measurement using fully convolutional network[J]. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090-1109. doi: 10.1111/mice.12412

    CrossRef Google Scholar

    [12] Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision, Glasgow, United Kingdom, 2018: 833-851.

    Google Scholar

    [13] Yang F, Zhang L, Yu S J, et al. Feature pyramid and hierarchical boosting network for pavement crack detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1525-1535. doi: 10.1109/TITS.2019.2910595

    CrossRef Google Scholar

    [14] Zou Q, Zhang Z, Li Q Q, et al. DeepCrack: learning hierarchical convolutional features for crack detection[J]. IEEE Transactions on Image Processing, 2019, 28(3): 1498-1512. doi: 10.1109/TIP.2018.2878966

    CrossRef Google Scholar

    [15] Mei Q P, Gül M, Azim M R, et al. Densely connected deep neural network considering connectivity of pixels for automatic crack detection[J]. Automation in Construction, 2020, 110: 103018. doi: 10.1016/j.autcon.2019.103018

    CrossRef Google Scholar

    [16] Fei Y, Wang K C P, Zhang A, et al. Pixel-level cracking detection on 3D asphalt pavement images through deep-learning-based CrackNet-V[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(1): 273-284. doi: 10.1109/TITS.2019.2891167

    CrossRef Google Scholar

    [17] Zhang A, Wang K C P, Li B X, et al. Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network[J]. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(10): 805-819. doi: 10.1111/mice.12297

    CrossRef Google Scholar

    [18] Ronneberger O, Fischer P, Brox T, et al. U-Net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 2015: 234-241.

    Google Scholar

    [19] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]//International Conference on Learning Representations, 2015.

    Google Scholar

    [20] Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615

    CrossRef Google Scholar

    [21] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[C]//International Conference on Learning Representations, 2016.

    Google Scholar

    [22] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184

    CrossRef Google Scholar

    [23] Wang P Q, Chen P F, Yuan Y, et al. Understanding convolution for semantic segmentation[C]//2018 IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, NV, USA, 2018: 1451-1460.

    Google Scholar

    [24] Yang M K, Yu K, Zhang C, et al. DenseASPP for semantic segmentation in street scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 3684-3692.

    Google Scholar

    [25] Luo W J, Li Y J, Urtasun R, et al. Understanding the effective receptive field in deep convolutional neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, 2016: 4905-4913.

    Google Scholar

    [26] Xie S, Tu Z. Holistically-Nested Edge Detection[J]. International Journal of Computer Vision, 2015, 125(1-3): 3-18. doi: 10.1007/s11263-017-1004-z

    CrossRef Google Scholar

    [27] Zou Q, Cao Y, Li Q Q, et al. CrackTree: Automatic crack detection from pavement images[J]. Pattern Recognition Letters, 2012, 33(3): 227-238. doi: 10.1016/j.patrec.2011.11.004

    CrossRef Google Scholar

    [28] Kingma D P, Ba L J. Adam: A method for stochastic optimization[C]//International Conference on Learning Representations, Ithaca, NY, 2015.

    Google Scholar

    [29] Chaurasia A, Culurciello E. LinkNet: Exploiting encoder representations for efficient semantic segmentation[C]//2007 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 2017: 1-4.

    Google Scholar

    [30] Zhang Z X, Liu Q J, Wang Y H. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753. doi: 10.1109/LGRS.2018.2802944

    CrossRef Google Scholar

    [31] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016: 770-778.

    Google Scholar

  • Overview: Cracks are one of the most common categories of pavement distress. Early locating and repairing the cracks can not only reduce the cost of pavement maintenance but also decrease the probability of road accidents happening. Precise measurement of the crack is an essential step toward identifying the road condition and determining rehabilitation strategies. Nevertheless, cracks do not have a certain shape and the appearance of cracks usually changes drastically in different lighting conditions, making it hard to be detected by the algorithm with imagery analytics. Therefore, fully automated and comprehensive crack detection is still challenging. In this paper, we focus on pixel-level crack detection in 2D vision and propose an effective U-shaped fully convolutional neural network called UCrackNet, which is the enhanced version of U-Net. It consists of three main components: an encoder, a bridge block, and a decoder. The backbone of the encoder is the pre-trained VGG-16 that extracts spatial features from the pavement image. The last convolutional layer at each scale in the encoder has a skip connection to connect the corresponding layer in the decoder to preserve and reuse feature maps at different pooling stages. To minimize the possibility of overfitting and achieve better generalization ability, we add a dropout layer into each skip connection. The bridge block is a bridge path between the encoder and the decoder. Motivated by DenseASPP, four densely connected atrous convolutional layers with different dilation rates are employed in the bridge block, so that it generates features with a larger receptive field to effectively capture multi-scale information. The decoder has four convolutional blocks, and in each block, up-sampling with indices is used to reduce the shift and distortion during the up-sampling operation. Furthermore, multi-level fusion is introduced in the output stage to utilize multiscale and multilevel information of objects. The idea of multi-level fusion is inspired by the success of the HED network architecture, which showed that it is capable of fully exploiting the rich feature hierarchies from convolutional neural network (CNNs). Specifically, the feature maps of each stage are first up-sampled to the size of the output image, then a 1×1 convolutional is used to fuse these maps to get the final prediction map. Qualitative evaluations on the two public CrackTree206 and AIMCrack datasets demonstrate that the proposed method achieves superior performance compared with CrackForest, LinkNet, ResUNet, U-Net, and DeepCrack. The qualitative results show that our method produces high-quality crack maps, which are closer to the ground-truth and have lower noise compared with the other methods.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(8)

Tables(4)

Article Metrics

Article views(5751) PDF downloads(1399) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint