Micro-vascular enhancement algorithm assists clinical diagnosis of conventional white light endoscopy
According the cancer report of China 2018, there were 4.29 million new cases of malignant tumors in China. More than 10,000 people were diagnosed with cancer every day, which means 7 people were diagnosed with cancer per minute. At present, the diagnosis of gastric cancer, colorectal cancer and esophageal cancer which are in the top five of morbidity and mortality are mainly based on conventional white light endoscopes, which cannot clearly reveal important tissue and vascular features and easily lead to misdiagnosis and missed diagnosis. Therefore, various new image enhancement technologies have been introduced into white light endoscopes. For example, through specific wavelengths of optical filters, Olympus’ NBI (Narrow Band Imaging) endoscope system makes the three wave band ranges of the blue, the green and the red narrow, which differently penetrate the macusa and make the blood vessel display clearly, but its hardware is complex and the bands are fixed. Fujifilm’s FICE (Fuji Intelligent Color Enhancement) technology extracts three images of a specific wavelength by a spectral estimation algorithm, and synthesizes to produce a color enhanced image in order to enhance the contrast between the micro-vascular and tissue of the mucosal surface layer. Karl Stroz's SPIES ( Storz Professional Image Enhancement System) offers four image enhancement technologies – Clara , Chroma, Spectra A and Spectra B, which enhance the dark areas in images, sharp images and increase the image contrast. However, the current image enhancement technologies in the white light endoscopes are all from the high-end foreign products, and these technologies cannot be directly used because of the patent barriers.
The research team led by Associate Professor Wang Liqiang and Associate Professor Yuan Bo of the State Key Laboratory of Modern Optical Instruments of Zhejiang University is dedicated to the research of high-definition electronic endoscopic imaging technology, and propose a vascular enhancement algorithm based on the spectral transformation. According to the absorption characteristics of capillaries in the different mucosal depth, the G and B channel of the image are enhanced and the R channel is reduced to make a remarkable color contrast between the tissue background and the blood vessels. Moreover, by using guided filtering, the brightness layer and the detail layer of each channel are separated and the detail layer containing the blood vessel features is enhanced, which further enhances the contrast of the image. Using the self-developed high-definition gastroscopy to photograph the oral cavity of different people, multiple endoscopic images were obtained (Fig. 1 a), and the algorithm was used to enhance the images which are shown in Fig. 1(b). In the original images, the tissue background color is very close to the blood vessels, and the tiny blood vessels are difficult to find. The overall contrast and the information of the blood vessels are significantly improved by using the proposed enhancement technology, and the mucosal structure, tissue characteristics, and microscopic blood vessels in the mucosa are clearly revealed.
The endoscopic images of SPIES are also enhanced by the proposed algorithm and the results are compared with those from Spectra B mode of SPIES. The Spectra B mode highlights the hemoglobin color, enhances the contrast between the blood and the surrounding tissue, and improves the details of the blood and the capillary by altering the spectral response of the imaging system. The comparison results are shown in Figure 2. The enhancement effect of this paper’s method is better, which are mainly reflected in two aspects: (1) The blood vessels are more vivid red and the background tissues are more greenish white, which makes the higher contrast between the blood vessels and the tissues; (2) The minutia structures of the blood vessels are more distinguishable and the micro-vessels are easier to recognize.
Fig. 1 The processed images. (a) Original images; (b) Enhanced images
Fig.2 Comparison with other enhancement methods.
Left is the original images of Karl Storz, middle is the images of Spectra B , right is the images of our method
The research team of the advanced medical endoscopy in the State Key Laboratory of Modern Optical Instruments of Zhejiang University has 2 professors, 3 associate professors and more than 10 graduate students. In the recent 5 years, the team has undertaken 2 programs of National High Technology Research and Development Program of China (863 Program), 2 programs of National Key R&D Program during the 13th Five-Year Plan Period and 2 programs of Key R&D Program in Zhejiang Province. More than 30 papers have been published on the advanced imaging technology of endoscopes, and more than 10 China invention patents have been obtained. The team had been awarded Second Prize for Science & Technology Development of the Ministry of Education in the field of digital photoelectric image sensor research and application. The team is leading in the domestic fields of the driver and signal transmission technology for high-resolution CMOS image sensor, the design, processing and assembly of the wide-angle and micro-endoscopic objective lens, the solution for real-time video signal acquisition and image processing, spectral color control based on micro-nano structure and multi-spectral imaging, and so on. The team had developed the first miniaturized CMOS imaging module for high-definition electronic gastroscopy, colonoscopy and laparoscopy in China, which has filled the domestic gap, and has realized the engineering and industrialization of this technology in the field of medical endoscopic imaging.
Jiang Hongpeng, Zhang Kejian, Yuan Bo, et al. A vascular enhancement algorithm for endoscope image[J]. Opto-Electronic Engineering, 2019, 46(1): 180167.