Nighttime sea fog recognition based on remote sensing satellite and deep neural decision tree

Li Tao; Jin Wei; Fu Randi; Li Gang; Yin Caoqian

doi:10.12086/oee.2022.220007

Article navigation > Opto-Electronic Engineering > 2022 Vol. 49 > No. 9 > 220007

Next Article Previous Article

Li T, Jin W, Fu R D, et al. Nighttime sea fog recognition based on remote sensing satellite and deep neural decision tree[J]. Opto-Electron Eng, 2022, 49(9): 220007. doi: 10.12086/oee.2022.220007

Citation:

Li T, Jin W, Fu R D, et al. Nighttime sea fog recognition based on remote sensing satellite and deep neural decision tree[J]. Opto-Electron Eng, 2022, 49(9): 220007. doi: 10.12086/oee.2022.220007

Nighttime sea fog recognition based on remote sensing satellite and deep neural decision tree

Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, Zhejiang 315211, China

Fund Project: National Natural Science Foundation of China (42071323) and Public Welfare Science and Technology Project of Ningbo (202002N3104).

More Information

^*Corresponding author: xyjw1969@126.com

Received Date 01 March 2022

Revised Date 22 May 2022

Accepted Date 27 May 2022

Published Date 25 September 2022

Abstract

Abstract

Remote sensing satellites have the characteristics of wide coverage and continuous observation, and are widely used in research related to the sea fog identification. Firstly, the Cloud-Aerosol LiDAR with Orthogonal Polarization (CALIOP), which is capable of penetrating clouds and obtaining atmospheric profiles, was used to annotate medium and high cloud, low cloud, sea fog, and clear sky sea surface samples. Then, bright temperature features and texture features were extracted from each type of sample in combination with multi-channel data from the Himawari-8 satellite. Finally, according to the needs of sea fog monitoring, the inference decision tree for sea fog monitoring was abstracted and a deep neural decision tree model was built accordingly, which could achieve high accuracy for nighttime sea fog monitoring while having strong interpretability. The continuous observation data of Himawari-8 on the night of June 5, 2020 was selected to test the sea fog. The monitoring results can clearly show the dynamic development process of the sea fog events. At the same time, the sea fog monitoring method in this paper has an average probability of detection (POD) of 87.32%, an average false alarm ratio (FAR) of 13.19%, and an average critical success index (CSI) of 77.36%, which provides a new method for disaster prevention and mitigation of heavy fog at sea.
- Himawari-8 /
- CALIOP /
- deep neural decision tree /
- nighttime sea fog recognition

FullText(HTML)

References

[1]	肖艳芳, 张杰, 崔廷伟, 等. 海雾卫星遥感监测研究进展[J]. 海洋科学, 2017, 41(12): 146−154. doi: 10.11759/hykx20170523001 CrossRef Google Scholar Xiao Y F, Zhang J, Cui T W, et al. Review of sea fog detection from satellite remote sensing data[J]. Marine Sciences, 2017, 41(12): 146−154. doi: 10.11759/hykx20170523001 CrossRef Google Scholar
[2]	Shin D, Kim J H. A new application of unsupervised learning to nighttime sea fog detection[J]. Asia Pac J Atmos Sci, 2018, 54(4): 527−544. doi: 10.1007/s13143-018-0050-y CrossRef Google Scholar
[3]	Kim D, Park M S, Park Y J, et al. Geostationary Ocean Color Imager (GOCI) marine fog detection in combination with Himawari-8 based on the decision tree[J]. Remote Sens, 2020, 12(1): 149. doi: 10.3390/rs12010149 CrossRef Google Scholar
[4]	郝姝馨, 郝增周, 黄海清, 等. 基于Himawari-8数据的夜间海雾识别[J]. 海洋学报, 2021, 43(11): 166−180. doi: 10.12284/hyxb2021158 CrossRef Google Scholar Hao S X, Hao Z Z, Huang H Q, et al. Nighttime sea fog recognition based on Himawari-8 data[J]. Haiyang Xuebao, 2021, 43(11): 166−180. doi: 10.12284/hyxb2021158 CrossRef Google Scholar
[5]	Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. UNet++: a nested U-Net architecture for medical image segmentation[M]//Stoyanov D, Taylor Z, Carneiro G, et al. Deep learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2018: 3–11. Google Scholar
[6]	Xuan R R, Li T, Wang Y T, et al. Prenatal prediction and typing of placental invasion using MRI deep and radiomic features[J]. BioMed Eng OnLine, 2021, 20(1): 56. doi: 10.1186/s12938-021-00893-5 CrossRef Google Scholar
[7]	Zha S J, Jin W, He C F, et al. Detecting of overshooting cloud tops via Himawari-8 imagery using dual channel multiscale deep network[J]. IEEE J Sel Top Appl Earth Obs Remote Sens, 2021, 14: 1654−1664. doi: 10.1109/JSTARS.2020.3044618 CrossRef Google Scholar
[8]	Kim J H, Ryu S, Jeong J, et al. Impact of satellite sounding data on virtual visible imagery generation using conditional generative adversarial network[J]. IEEE J Sel Top Appl Earth Obs Remote Sens, 2020, 13: 4532−4541. doi: 10.1109/JSTARS.2020.3013598 CrossRef Google Scholar
[9]	Kim K, Kim J H, Moon Y J, et al. Nighttime reflectance generation in the visible band of satellites[J]. Remote Sens, 2019, 11(18): 2087. doi: 10.3390/rs11182087 CrossRef Google Scholar
[10]	Jeon H K, Kim S, Edwin J, et al. Sea fog identification from GOCI images using CNN transfer learning models[J]. Electronics, 2020, 9(2): 311. doi: 10.3390/electronics9020311 CrossRef Google Scholar
[11]	Xu M Q, Wu M, Guo J, et al. Sea fog detection based on unsupervised domain adaptation[J]. Chin J Aeron, 2022, 35(4): 415−425. doi: 10.1016/j.cja.2021.06.019 CrossRef Google Scholar
[12]	Huang Y X, Wu M, Guo J, et al. A correlation context-driven method for sea fog detection in meteorological satellite imagery[J]. IEEE Geosci Remote Sens Lett, 2022, 19: 1003105. doi: 10.1109/LGRS.2021.3095731 CrossRef Google Scholar
[13]	刘树霄, 衣立, 张苏平, 等. 基于全卷积神经网络方法的日间黄海海雾卫星反演研究[J]. 海洋湖沼通报, 2019(6): 13−22. doi: 10.13984/j.cnki.cn37-1141.2019.06.002 CrossRef Google Scholar Liu S X, Yi L, Zhang S P, et al. A study of daytime sea fog retrieval over the yellow sea based on fully convolutional networks[J]. Trans Oceanol Limnol, 2019(6): 13−22. doi: 10.13984/j.cnki.cn37-1141.2019.06.002 CrossRef Google Scholar
[14]	黄彬, 吴铭, 孙舒悦, 等. 基于深度学习的卫星多通道图像融合的海雾监测处理方法[J]. 气象科技, 2021, 49(6): 823−829,850. Google Scholar Huang B, Wu M, Sun S Y, et al. Sea fog monitoring method based on deep learning satellite multi-channel image fusion[J]. Meteorol Sci Technol, 2021, 49(6): 823−829,850. Google Scholar
[15]	Wan J H, Su J, Liu S W, et al. The research on the spectral characteristics of sea fog based on CALIOP and MODIS data[J]. Int Arch Photogramm Remote Sens Spatial Inf Sci, 2018, XLII-3: 1667−1671. doi: 10.5194/isprs-archives-XLII-3-1667-2018 CrossRef Google Scholar
[16]	Wu D, Lu B, Zhang T C, et al. A method of detecting sea fogs using CALIOP data and its application to improve MODIS-based sea fog detection[J]. J Quant Spectr Radiat Transfer, 2015, 153: 88−94. doi: 10.1016/j.jqsrt.2014.09.021 CrossRef Google Scholar
[17]	司光, 符冉迪, 何彩芬, 等. 结合遥感卫星及深度神经网络的白天海雾识别[J]. 光电子·激光, 2020, 31(10): 1074−1082. doi: 10.16136/j.joel.2020.10.0172 CrossRef Google Scholar Si G, Fu R D, He C F, et al. Daytime sea fog recognition based on remote sensing satellite and deep neural network[J]. J Optoelectron·Laser, 2020, 31(10): 1074−1082. doi: 10.16136/j.joel.2020.10.0172 CrossRef Google Scholar
[18]	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning, 2015: 448–456. Google Scholar
[19]	Wan A, Dunlap L, Ho D, et al. NBDT: neural-backed decision trees[Z]. arXiv: 2004.00221, 2020. https://arxiv.org/abs/2004.00221. Google Scholar
[20]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016. Google Scholar

Overview

Overview

Sea fog is a dangerous weather phenomenon that seriously affects maritime traffic and other operations at sea. Remote sensing satellites have the characteristics of wide coverage and continuous observation, and are widely used in research related to the sea fog identification. Traditional sea fog monitoring algorithms usually establish a single channel or multi-channel model to gradually separate the clear sky sea surface, medium and high clouds, and low clouds with the help of the differences of reflectivity or brightness-temperature distribution of clouds in different channel satellite cloud images, so as to finally achieve the purpose of identifying sea fog. Although this method has the advantages of being simple, efficient, and highly interpretable, its sea fog identification accuracy is usually low, and is susceptible to seasonal and regional influences. As the main method of deep learning, convolutional neural network has the advantages of strong feature learning ability and high prediction accuracy. It is widely used in cloud image related fields. Although many studies have transferred convolutional neural network to sea fog monitoring task, they are limited to daytime sea fog monitoring. It is more difficult to label sea fog because of the lack of visible wavelength data at night compared to daytime monitoring scenarios. In addition, convolutional neural network is "black box" in nature, i.e. it is difficult to explain their inference process in a reasonable way.

In order to make the recognition of sea fog with high accuracy and reasonable interpretability, the cloud-aerosol LiDAR with orthogonal polarization (CALIOP), which is capable of penetrating clouds and obtaining atmospheric profiles, was first used to annotate medium and high cloud, low cloud, sea fog, and clear sky sea surface samples. Then, bright temperature features and texture features were extracted for each type of sample in combination with multi-channel data from the Himawari-8 satellite. Finally, according to the needs of sea fog monitoring, the inference decision tree for sea fog monitoring was abstracted and a deep neural decision tree model was built accordingly, which achieves high accuracy for nighttime sea fog monitoring while having strong interpretability. The continuous observation data of Himawari-8 on the night of June 5, 2020 was selected to test the sea fog. The monitoring results can clearly show the dynamic development process of the sea fog events. At the same time, the proposed sea fog monitoring method has an average probability of detection (POD) of 87.32%, an average false alarm ratio (FAR) of 13.19%, and an average critical success index (CSI) of 77.36%, which provides a new method for disaster prevention and mitigation of heavy fog at sea.