Ye Zihao, Sun Rui, Wang Huihui. Lane recognition method based on fully convolution neural network and conditional random fields[J]. Opto-Electronic Engineering, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274
Citation: Ye Zihao, Sun Rui, Wang Huihui. Lane recognition method based on fully convolution neural network and conditional random fields[J]. Opto-Electronic Engineering, 2019, 46(2): 180274. doi: 10.12086/oee.2019.180274

Lane recognition method based on fully convolution neural network and conditional random fields

    Fund Project: Supported by National Natural Science Foundation of China (61471154) and Anhui Science and Technology Research Project (170d0802181)
More Information
  • Aiming at the poor adaptability of traditional lane recognition method in complex pavement, this paper proposes a lane recognition method based on full convolutional neural network and conditional random field, according to image segmentation technology. The method can make the neural network model identify the lanes by training a large amount of data, and then make the segmentation of the lanes' coverage and the lane edges more perfect through the conditional random field. At the same time, in order to solve the high requirement of real-time detection in expressway, a fully convolution neural network is designed in this paper. The network structure is simple with only 130000 parameters and three improvements are made as follows: BN algorithm is used to improve network generalization ability and convergence rate; LeakyReLU activation function is used to replace the commonly used relu or sigmoid activation function, and using Nadam as the network optimizer makes the network have better robustness; Conditional random field is used as the back-end processing solution insufficient lane segmentation and further to increase lane coverage. Finally, in order to solve the problem of complex road environment in urban road testing, this paper uses the back-end processing of FCN-16s network model and conditional random field to realize the recognition of complex urban roads. Experiments show that the network model designed in this paper is more real-time and sufficient for lane identification in the face of high-speed expressways and simple lanes. In the complex environment of urban road, FCN-16s model plus conditional random field can identify lane more accurately and get good results on KITTI road test benchmarks.
  • 加载中
  • [1] 李明, 黄华, 夏建刚.基于Hough变换的车道检测改进算法研究[J].计算机工程与设计, 2012, 33(4):1638-1642. doi: 10.3969/j.issn.1000-7024.2012.04.075

    CrossRef Google Scholar

    Li M, Huang H, Xia J G. Research of robust lane detection algorithm based on Hough transform[J]. Computer Engineering And Design, 2012, 33(4):1638-1642. doi: 10.3969/j.issn.1000-7024.2012.04.075

    CrossRef Google Scholar

    [2] Sharifi M, Fathy M, Mahmoudi M T. A classified and comparative study of edge detection algorithms[C]//Proceedings of International Conference on Information Technology: Coding and Computing, Las Vegas, NV, USA, 2002: 117.

    Google Scholar

    [3] Jung H G, Lee Y H, Kang H J, et al. Sensor fusion-based lane detection for LKS+ACC system[J]. International Journal of Automotive Technology, 2009, 10(2): 219-228. doi: 10.1007/s12239-009-0026-0

    CrossRef Google Scholar

    [4] McCall J C, Trivedi M M. An integrated, robust approach to lane marking detection and lane tracking[C]//IEEE Intelligent Vehicles Symposium, Parma, Italy, 2004: 533-537.

    Google Scholar

    [5] 郭磊, 李克强, 王建强, 等.应用方向可调滤波器的车道线识别方法[J].机械工程学报, 2008, 44(8): 214-218, 226. doi: 10.3321/j.issn:0577-6686.2008.08.038

    CrossRef Google Scholar

    [6] Vitor G B, Lima D A, Victorino A C, et al. A 2D/3D vision based approach applied to road detection in urban environments[C]//2013 IEEE Intelligent Vehicles Symposium, Gold Coast, QLD, Australia, 2013: 952-957.

    Google Scholar

    [7] Mendes C C T, Frémont V, Wolf D F. Exploiting fully convolutional neural networks for fast road detection[C]//2016 IEEE International Conference on Robotics and Automation, Stockholm, Sweden, 2016.

    Google Scholar

    [8] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683

    CrossRef Google Scholar

    [9] Wang B, Frémont V, Rodríguez S A. Color-based road detection and its evaluation on the KITTI road benchmark[C]//2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, 2014: 31-36.

    Google Scholar

    [10] Mohan R. Deep deconvolutional networks for scene parsing[EB/OL]. https://available:arxiv.org/abs/1411.4101.

    Google Scholar

    [11] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[EB/OL]. https: //available: arxiv.org/abs/1409.1556.

    Google Scholar

    [12] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, 2012: 1097-1105.

    Google Scholar

    [13] Szegedy C, Liu W, Jia Y Q, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 1-9.

    Google Scholar

    [14] Brust C A, Sickert S, Simon M, et al. Convolutional patch networks with spatial prior for road detection and urban scene understanding[C]//VISAPP 2015-10th International Conference on Computer Vision Theory and Applicayions, Berlin, Germany, 2015: 510-517.

    Google Scholar

    [15] Mendes C C T, Frémont V, Wolf D F. Exploiting fully convolutional neural networks for fast road detection[C]//2016 IEEE International Conference on Robotics and Automation, Stockholm, Sweden, 2016.

    Google Scholar

    [16] Dinesh Reddy N, Prateek Singhal, Madhava Krishna K. Semantic Motion Segmentation Using Dense CRF Formulation[C]//Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP, Bangalore, India.2014: a56.

    Google Scholar

    [17] Fritsch J, Kühnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms[C]//16th International IEEE Conference on Intelligent Transportation Systems, The Hague, Netherlands, 2013: 1693-1700.

    Google Scholar

    [18] Dan C C, Giusti A, Gambardella L M, et al. Deep neural networks segment neuronal membranes in electron microscopy images[J]. Advances in Neural Information Processing Systems, 2012, 25: 2852-2860.

    Google Scholar

    [19] Dai J F, He K M, Sun J. Convolutional feature masking for joint object and stuff segmentation[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2014: 3992-4000.

    Google Scholar

    [20] Huval B, Wang T, Tandon S, et al. An empirical evaluation of deep learning on highway driving[EB/OL].https://arxiv.org/pdf/1504.01716.pdf.

    Google Scholar

    [21] Krähenbühl P, Koltun V. Efficient inference in fully connected CRFs with Gaussian edge potentials[C]//Proceedings of the 24th Neural Information Processing Systems, Granada, Spain, 2011: 109-117.

    Google Scholar

    [22] 孙锐, 方蔚, 高隽.暗通道和测度学习的雾天行人再识别[J].光电工程, 2016, 43(12): 142-146. doi: 10.3969/j.issn.1003-501X.2016.12.022

    CrossRef Google Scholar

    Sun R, Fang W, Gao J. Person re-identification in foggy weather based on dark channel prior and metric learning[J]. Opto-Electronic Engineering, 2016, 43(12): 142-146. doi: 10.3969/j.issn.1003-501X.2016.12.022

    CrossRef Google Scholar

  • Overview: In recent years, with the rapid development of smart cars, autonomous driving has attracted great attention from industry and academia. Lane recognition is a fundamental work for achieving automatic driving. Specifically, accurate lane recognition not only allows the vehicle to travel on the correct road but also alerts the control system with other information such as lane markings, pedestrians and other anomalous events. The traditional lane detection method generally adopts the steps of preprocessing, edge detection, Hough transform, lane matching, lane segmentation, etc. These methods interact with each other, and it is difficult to achieve global optimization and real-time optimization simultaneously. Furthermore, the lane change adaptation is obviously insufficient. Deep neural network is a powerful visual analysis tool. Compared with the traditional shallow computing structure, its main advantage lies in its self-learning ability according to input data and its end-to-end unified structure. Aiming at the poor adaptability of traditional lane recognition method in complex pavement, this paper proposes a lane recognition method based on full convolutional neural network and conditional random field, according to image segmentation technology. The method can make the neural network model identify the lanes by training a large amount of data, and then make the segmentation of the lanes' coverage and the lane edges more perfect through the conditional random field. At the same time, in order to solve the high requirement of real-time detection in expressway, a fully convolution neural network is designed in this paper. The network structure is simple with only 130000 parameters and three improvements are made as follows: BN algorithm is used to improve network generalization Ability and convergence rate; LeakyReLU activation function is used to replace the commonly used relu or sigmoid activation function, and using Nadam as the network optimizer makes the network have better robustness. Conditional random field is used as the back-end processing solution insufficient lane segmentation and further to increase lane coverage. Finally, in order to solve the problem of complex road environment in urban road testing, this paper uses the back-end processing of FCN-16s network model and conditional random field to realize the recognition of complex urban roads. Experiments show that the network model designed in this paper is more real-time and sufficient for lane identification in the face of high-speed expressways and simple lanes. In the complex environment of urban road, FCN-16s model plus conditional random field can identify lane more accurately and get good result on KITTI road test benchmarks.

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(10)

Tables(4)

Article Metrics

Article views(7091) PDF downloads(3239) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint