扫描电子显微镜图像漂移的实时矫正方法

徐伟, 谷森, 储成智, 等. 扫描电子显微镜图像漂移的实时矫正方法[J]. 光电工程, 2018, 45(12): 180198. doi: 10.12086/oee.2018.180198
引用本文: 徐伟, 谷森, 储成智, 等. 扫描电子显微镜图像漂移的实时矫正方法[J]. 光电工程, 2018, 45(12): 180198. doi: 10.12086/oee.2018.180198
Xu Wei, Gu Sen, Chu Chengzhi, et al. Real-time correction of image drift in scanning electron microscope[J]. Opto-Electronic Engineering, 2018, 45(12): 180198. doi: 10.12086/oee.2018.180198
Citation: Xu Wei, Gu Sen, Chu Chengzhi, et al. Real-time correction of image drift in scanning electron microscope[J]. Opto-Electronic Engineering, 2018, 45(12): 180198. doi: 10.12086/oee.2018.180198

扫描电子显微镜图像漂移的实时矫正方法

  • 基金项目:
    国家自然科学基金资助项目(61774107)
详细信息
    作者简介:
    通讯作者: 汝长海(1976-),男,博士后,教授,主要从事微纳自动化的研究。E-mail: rzh@suda.edu.cn
  • 中图分类号: O436.3:TP391

Real-time correction of image drift in scanning electron microscope

  • Fund Project: Supported by National Natural Science Foundation of China (61774107)
More Information
  • 为解决扫描电子显微镜(SEM)由于电子束漂移、电磁干扰等原因导致的图像漂移问题,提出基于ORB结合PROSAC的图像漂移矫正算法。首先采用ORB算法对基准图像和实时图像进行特征检测,然后利用汉明距离与交叉匹配实现特征的初匹配,再结合RANSAC的优化算法PROSAC计算帧间的单应矩阵,利用单应矩阵映射剔除外点后重新迭代计算出最终的精确单应矩阵,最后利用单应矩阵的透视变换实现SEM图像漂移实时矫正。通过实验证明,该算法不仅精度高,而且能够满足SEM实时处理的要求。

  • Overview: With the development of nanoscience, new nanomaterials and their excellent properties have been continuously discovered and recognized, showing a broad application prospect. Compared with the traditional optical microscope, the scanning electron microscope (SEM) has been widely used in the characterization of nanomaterials, micro deformation measurement with the advantages of its simple preparation, wide range of adjustable magnification, high resolution and great depth of field. In addition, the use of SEM images as visual sensors has merit in the development of automated nanomanipulations, such as automatic detection of IC chips, automatic pick-up of nanowires, and automatic measurement of nanowire impedance characteristics by four-point probes, making humans liberated from tedious nano-manipulations, greatly improving work efficiency. However, the drift of images caused by electron beam drift and electromagnetic interference at high magnification will affect the size measurement and characterization of nanomaterials in SEM.

    At present, domestic and foreign scholars have actively studied the correction method of SEM image drift, but there are still many obvious shortcomings. Cizmar et al. aligned the images correctly to form a single image by using larges of image frames, but which can't meet the requirements of the real-time in SEM. In order to solve the problem of real time, Naresh took the ORB combing the RANSAC to correct the drift image, but it does not eliminate the external points. The increase of the external points lead to the rapid increase of the sampling times of RANSAC, and then affects the efficiency of the algorithm and the accuracy of the measurement. Although AFM compensation algorithm has been able to solve the problem of image drift, it is not suitable for SEM because of the different principles. An image shift correction algorithm based on ORB (oriented FAST and rotated BRIEF) combing the PROSAC (progressive sample consensus) is proposed in this paper. Firstly, the ORB algorithm is used to detect the feature between the reference image and real-time image. Then the initial matching of the feature is implemented by using the Hamming distance and cross-matching. Moreover, the RANSAN (random sample consensus) optimization algorithm PROSAC is used to calculate the homography matrix between frames and the final exact homography matrix is re-iterated after eliminating exterior point. Finally, the SEM image drift is corrected in real time using the perspective transformation of the homography matrix. The experiments show that the proposed algorithm is high precision and satisfies the requirement of SEM real-time processing. This method can meet the requirements of the drift correction in SEM under different magnifications, and provide help for SEM measurement, characterization and other applications.

  • 加载中
  • 图 1  扫描电镜工作原理图

    Figure 1.  The working principle of SEM

    图 2  不同时刻SEM图像

    Figure 2.  SEM images at different times

    图 3  SEM图像漂移矫正流程图

    Figure 3.  The flow chart of SEM image drift correction

    图 4  FEI在30000倍率下漂移矫正。(a)基准图;(b)实时图;(c)矫正图

    Figure 4.  FEI drift correction at 30000× magnification. (a) Reference image; (b) Real-time image; (c) Corrective image

    图 5  FEI在35000倍率下漂移矫正。(a)基准图;(b)实时图;(c)矫正图

    Figure 5.  FEI drift correction at 35000× magnification. (a) Reference image; (b) Real-time image; (c) Corrective image

    图 6  SU8010在30000倍率下漂移矫正。(a)基准图;(b)实时图;(c)矫正图

    Figure 6.  SU8010 drift correction at 30000× magnification. (a) Reference image; (b) Real-time image; (c) Corrective image

    图 7  SU8010在35000倍率下漂移矫正。(a)基准图;(b)实时图;(c)矫正图

    Figure 7.  SU8010 drift correction at 35000× magnification. (a) Reference image; (b) Real-time image; (c) Corrective image

    表 1  FEI在30000倍率下单应矩阵参数

    Table 1.  Homography parameters computed at 30000× magnification in FEI

    方法 H11 H12 H21 H22 H13 H23
    ORB+RANSAC 0.989 0.002 -0.003 0.993 -2.418 -3.269
    ORB+D_RANSAC 1.007 -0.002 0.002 1.003 1.574 1.306
    本文算法 0.999 -0.001 0.003 1.001 1.478 1.345
    下载: 导出CSV

    表 2  FEI在35000倍率下单应矩阵参数

    Table 2.  Homography parameters computed at 35000× magnification

    方法 H11 H12 H21 H22 H13 H23
    ORB+RANSAC 0.999 -0.002 0.001 0.996 3.937 -1.413
    ORB+D_RANSAC 0.999 -0.001 -0.001 0.100 -2.558 2.962
    本文算法 0.989 -0.002 -0.001 0.101 -2.345 2.768
    下载: 导出CSV

    表 3  SU8010在30000倍率下单应矩阵参数

    Table 3.  Homography parameters computed at 30000× magnification in SU8010

    方法 H11 H12 H21 H22 H13 H23
    ORB+RANSAC 1.001 -0.001 -0.001 0.999 1.187 -4.069
    ORB+D_RANSAC 1.000 -0.002 -0.001 1.001 -1.355 -5.768
    本文算法 0.999 -0.002 -0.001 1.001 -1.344 -6.023
    下载: 导出CSV

    表 4  SU8010在35000倍率下单应矩阵参数

    Table 4.  Homography parameters computed at 35000× magnification in SU8010

    方法 H11 H12 H21 H22 H13 H23
    ORB+RANSAC 0.988 -0.003 -0.003 0.989 -1.446 -1.558
    ORB+D_RANSAC 0.994 -0.003 -0.002 0.992 -7.654 -1.369
    本文算法 0.993 -0.003 -0.003 0.993 -7.345 -1.568
    下载: 导出CSV

    表 5  FEI漂移矫正平均绝对误差

    Table 5.  The MAE of FEI drift correction

    方法 30000× 35000×
    ORB+RANSAC 12.57 20.98
    ORB+D_RANSAC 5.47 7.28
    本文算法 8.68 13.38
    下载: 导出CSV

    表 6  FEI漂移矫正帧率

    Table 6.  The frame rate of FEI drift correction (f/s)

    方法 30000× 35000×
    ORB+RANSAC 13.38 14.28
    ORB+D_RANSAC 11.57 9.54
    本文算法 21.80 19.91
    下载: 导出CSV

    表 7  SU8010漂移矫正平均绝对误差

    Table 7.  The MAE of SU8010 drift correction

    方法 30000× 35000×
    ORB+RANSAC 23.97 23.19
    ORB+D_RANSAC 19.88 21.89
    本文算法 20.95 21.95
    下载: 导出CSV

    表 8  SU8010漂移矫正帧率

    Table 8.  The frame rate of SU8010 drift correction (f/s)

    方法 30000× 35000×
    ORB+RANSAC 14.07 13.45
    ORB+D_RANSAC 13.55 12.27
    本文算法 21.91 20.47
    下载: 导出CSV
  • [1]

    曹耀宇, 谢飞, 张鹏达, 等.双光束超分辨激光直写纳米加工技术[J].光电工程, 2017, 44(12): 1135-1145. 10.3969/j.issn.1003-501X.2017.12.001

    Cao Y Y, Xie F, Zhang P D, et al. Dual-beam super-resolution direct laser writing nanofabrication technology[J]. Opto-Electronic Engineering, 2017, 44(12): 1135-1145. 10.3969/j.issn.1003-501X.2017.12.001

    [2]

    Gong Z, Chen B K, Sun Y, et al. Robotic Probing of Nanostructures inside Scanning Electron Microscopy[J]. IEEE Transactions on Robotics, 2014, 30(3): 758-765. doi: 10.1109/TRO.2014.2298551

    [3]

    Ye X T, Zhang Y, Ru C H, et al. Automated Pick-Place of Silicon Nanowires[J]. IEEE Transactions on Automation Science and Engineering, 2013, 10(3): 554-561. doi: 10.1109/TASE.2013.2244082

    [4]

    Ru C H, Zhang Y, Sun Y, et al. Automated Four-Point probe Measurement of Nanowires Inside a Scanning Electron Microscope[J]. IEEE Transactions on Nanotechnology, 2011, 10(4): 674-681. http://ieeexplore.ieee.org/document/5546976/

    [5]

    Gong Z, Chen B K, Sun Y, et al. Fluorescence and SEM correlative microscopy for nanomanipulation of subcellular structures[J]. Light: Science and Application, 2014, 3(11): e224. doi: 10.1038/lsa.2014.105

    [6]

    Cizmar P, Vladar A E, Postek M T. Real-Time Scanning Charged-Particle Microscope Image Composition with Correction of Drift[J]. Microscopy and Microanalysis, 2011, 17(2): 302-308. doi: 10.1017/S1431927610094250

    [7]

    Marturi N, Demele S, Piat N. Fast Image Drift Compensation in Scanning Electron Microscope using Image Registration[C]// Proceedings of 2013 IEEE International Conference on Automation Science and Engineering, 2013: 807-812.

    [8]

    肖春宝, 冯大政, 冯祥卫.重抽样优化的快速随机抽样一致性算法[J].计算机辅助设计与图形学学报, 2016, 28(4): 607-613. http://d.old.wanfangdata.com.cn/Periodical/jsjfzsjytxxxb201604010

    Xiao C B, Feng D Z, Feng X W. Fast RANSAC algorithm with resample optimization[J]. Journal of Computer-Aided Design and Computer Graphics, 2016, 28(4): 607-613. http://d.old.wanfangdata.com.cn/Periodical/jsjfzsjytxxxb201604010

    [9]

    Sutton M A, Li N, Li X D. Metrology in a scanning electron microscope: theoretical developments and experimental validation[J]. Measurement Science and Technology, 2006, 17(10): 2613-2622. doi: 10.1088/0957-0233/17/10/012

    [10]

    Malti A C, Dembele S, Piat N, et al. Magnification-continuous static calibration model of a scanning- electron microscope[J]. Journal of Electronic Imaging, 2012, 21(3): 033020. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=61a324377bfff3b2b158c2396d1708dc

    [11]

    Marturi N. Vision and visual servoing for nanomanipulation and nanocharacterization in scanning electron microscope[D]: France: Univerite De France-comte, 2013.

    [12]

    Liu W, Zhao W J, Li C, et al. Detecting small moving target based on the improved ORB feature matching[J]. Opto-Electronic Engineering, 2015, 42(10): 13-20. 10.3969/j.issn.1003-501X.2015.10.003

  • 加载中

(7)

(8)

计量
  • 文章访问数:  11233
  • PDF下载数:  3229
  • 施引文献:  0
出版历程
收稿日期:  2018-04-17
修回日期:  2018-05-16
刊出日期:  2018-12-01

目录

/

返回文章
返回