臺灣博碩士論文加值系統

現今深度學習已經廣泛的運用在各種領域當中，並且已經是人們日常生活中
不可或缺的一部分，例如在工業自動化領域，我們運用深度學習讓工廠減少人力
成本支出、在金融方面，我們可以利用過往的數據來預測未來的經濟走勢及趨勢，
在醫療方面，可以找出病灶區，來幫助醫生進行診斷，等例子可以發現深度學習
的潛力。
在當前的臨床醫療診斷中，醫學圖像具有越來越重要的應用價值。常用的醫
學圖像有磁共振（Magnetic Resonance Imaging，MRI）、X 光（X-Ray）、電腦斷
層攝影(Computer Tomography，CT）。然而傳統的醫學影像處理主要依靠專家的
個人經驗，並且人工地對圖像來進行分析以及處理。然而，這種人工處理方法效
率較低，而且分析結果往往受主觀影響較大。此外，近年來醫學圖像數量急劇的
增加更是導致傳統以人工處理的方式變得嚴重耗力且耗時。因此，所以迫切需要
利用電腦來類比人類的視覺感知機制，讓電腦自動地對於圖像進行分類，以此來
方便醫生診斷。如今隨著科學技術以及硬體性能的迅速發展，運用電腦來快速的
分析、處理各類醫學圖像來輔助專家進行診斷，已經成為醫學領域的一種重要趨
勢，具有重要的研究意義與應用價值。基於電腦視覺的醫學圖像自動分析與處理
技術具有其獨特的優勢，如：利用電腦的強大計算能力可獲取快速準確的分析與
處理結果，可使醫學影像處理結果不受專家疲勞或資料過多等等影響，利用電腦
技術可促進更快的臨床資訊溝通，以方便為偏遠地區病人提供快速、準確的醫療
診斷資訊。
有鑑於此，本論文提出一個結合 ResNet 與 U-NET 的深度學習模型的
RUNet。在 RUNet 中，使用了 U-NET 做為基本模型架構，並將模型與 ResNet 中
的殘差模塊進行結合，以此來提升網路的深度，增強網路提取特徵的能力，而在
神經網路當中末端的特徵圖雖然較細緻但範圍較小，而前端的特徵圖雖然範圍大
但特徵比較粗糙，因此最好的方法是都讓他們做學習，
ii
所以這裡使用全尺寸的跳躍連接使模型能更好的利用各層提取的特徵。而在充分
結合圖像各層的資訊後，接著在底層時使用不同尺寸的捲積核，提取不同尺度的
特徵，再加以結合。而在實驗設計上，我們使用視網膜之資料集 DRIVE 和 ROSE，
並對其進行資料前處理以及資料增強，進一步提升分割效果。實驗結果顯示
RUNet 相較於其他網路模型 U-NET、ResUNet、U-NET3+、ResUNet++，在圖像
分割的準確度更好。本研究希望透過此模型能夠更精準地分割出血管，同時大幅
減少人力的成本。

Today, deep learning has been widely used in various fields and has become an
indispensable part of people's daily lives. For example, in the field of industrial
automation, we use deep learning to reduce labor costs in factories. In finance, we can
use the past. The data can be used to predict future economic trends and trends. In terms
of medical treatment, the lesion area can be found to help doctors make a diagnosis.
Examples such as the potential of deep learning can be found.
In the current clinical medical diagnosis, medical images have more and more important
application value. Commonly used medical images include Magnetic Resonance
Imaging (MRI), X-ray (X-Ray), and Computer Tomography (CT). However, the
traditional medical image processing mainly relies on the personal experience of
experts, and the images are analyzed and processed manually. However, this manual
processing method is inefficient, and the analysis results are often subject to greater
subjective influence. In addition, the rapid increase in the number of medical images in
recent years has made the traditional manual processing methods become seriously
labor-intensive and time-consuming. Therefore, it is urgent to use the computer to
analogize the human visual perception mechanism, so that the computer can
automatically classify the images, so as to facilitate the doctor's diagnosis. Nowadays,
with the rapid development of science and technology and hardware performance, the
use of computers to quickly analyze and process various medical images to assist
experts in diagnosis has become an important trend in the medical field, with important
research significance and application value. The automatic analysis and processing
technology of medical images based on computer vision has its unique advantages, such
as: using the powerful computing power of the computer to obtain fast and accurate
analysis and processing results, the medical image processing results can be free from
iv
expert fatigue or excessive data, etc. The use of computer technology can facilitate
faster communication of clinical information, so as to facilitate the provision of rapid
and accurate medical diagnosis information for patients in remote areas.
In view of this, this paper proposes a RUNet that combines the deep learning model of
ResNet and U-NET. In RUNet, U-NET is used as the basic model architecture, and the
model is combined with the residual module in ResNet to improve the depth of the
network and enhance the ability of the network to extract features. The feature map at
the end is more detailed but the scope is small, and the feature map at the front end is
large but the features are relatively rough, so the best way is to let them all learn,
Therefore, the use of full-size skip connections here enables the model to better utilize
the features extracted by each layer. After fully combining the information of each layer
of the image, convolution kernels of different sizes are used at the bottom layer to
extract features of different scales, and then combine them. In the experimental design,
we use the retinal datasets DRIVE and ROSE, and perform data preprocessing and data
enhancement to further improve the segmentation effect. The experimental results show
that RUNet has better image segmentation accuracy than other network models U-NET,
ResNet, and U-NET3+. This study hopes to use this model to segment blood vessels
more accurately, while greatly reducing labor costs.

摘要 i
Abstract iii
目錄 v
表目錄 vii
圖目錄 viii
1、 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 2
1.3 論文架構 3
2、 文獻回顧 4
2.1 醫學影像 4
2.1.1 眼科光學斷層掃描 4
2.2 圖像語意分割(Semantic Segmentation) 5
2.2.1 醫學影像分割 6
2.3 卷積神經網路（Convolutional Neural Networks） 6
2.4 殘差網路(Resideul Network) [7] 10
2.5 Fully Convolutional Networks（FCN）[8] 11
2.6 U-NET[9] 12
2.7 ResUNet[2] 14
2.8 ResUNet++[10] 16
2.9 UNet3+[11] 17
3、 研究方法 18
3.1 資料集介紹 18
3.1.1 DRIVE (Digital Retinal Images for Vessel Extraction) 18
3.1.2 ROSE: A Retinal OCT-Angiography Vessel SEgmentation Dataset 19
3.2 資料前處理 20
3.2.1灰階化 21
3.2.2雙邊濾波器(Bilateral Filter )[16] 21
3.2.3直方圖均衡化[17] 21
3.2.4 伽瑪校正(Gamma Correction)[18] 22
3.3 資料增強[19] 23
3.4 系統架構 24
3.5 模型架構 25
3.5.1 Resblock 26
3.5.2 全尺度跳躍連接 29
3.5.3 Inception block [23] 30
4、 研究結果與分析 31
4.1 評估指標 31
4.2 各模型之評估結果 33
4.2.1 未做前處理之DRIVE資料集 33
4.2.2 DRIVE資料集做完前處理後之評估結果 37
4.2.3 ROSE資料集之評估結果 41
模型成效分析 46
4.3 46
5、 結論與未來展望 47
5.1 結論 47
5.2 未來展望 48
6、 參考文獻 49

[1]D. L.Pham, C.Xu, & J. L.Prince, (2000). Current methods in medical image segmentation. Annual review of biomedical engineering, 2(1), 315-337.
[2]P.Xiuqin, Q.Zhang, H.Zhang, & S.Li, (2019). A fundus retinal vessels segmentation scheme based on the improved deep learning U-Net model. IEEE Access, 7, 122634-122643.
[3]D.Huang, E. A.Swanson, C. P.Lin, J. S. Schuman, W. G.Stinson, W.Chang,... & J. G.Fujimoto, (1991). Optical coherence tomography. science, 254(5035), 1178-1181.
[4]H.Zhang, K.Dana, J.Shi, Z.Zhang, X.Wang, A.Tyagi, & A.Agrawal, (2018). Context encoding for semantic segmentation. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 7151-7160).
[5]R.Shetty, B.Schiele, & M. Fritz, (2019). Not Using the Car to See the Sidewalk--Quantifying and Controlling the Effects of Context in Classification and Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8218-8226).
[6]Y.LeCun, L.Bottou, Y.Bengio, & P.Haffner, (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
[7]K.,He, Zhang, S.X.Ren, & J.Sun, (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[8]J.Long, E.Shelhamer, T. & Darrell, (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431-3440).
[9]O.Ronneberger, P.Fischer, & T.Brox, (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
[10]D.Jha, P. H.Smedsrud, M. A.Riegler, D.Johansen, T.De Lange, P.Halvorsen, & H. D.Johansen, (2019, December). Resunet++: An advanced architecture for medical image segmentation. In 2019 IEEE International Symposium on Multimedia (ISM) (pp. 225-2255). IEEE.
[11]H.Huang, L.Lin, R.Tong, H. Hu, Q.Zhang, Y.Iwamoto, ... & J.Wu (2020, May). Unet 3+: A full-scale connected unet for medical image segmentation. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1055-1059). IEEE.
[12]J. Staal, M. D. Abramoff, and M. Niemeijer, ‘‘Ridge-based vessel segmentation in color images of the retina,’’ IEEE Trans. Med. Imag., vol. 23, no. 4,pp. 501–509, Aug. 2004.
[13]Y.Ma, ,H.Hao, J. Xie, H. Fu, J. Zhang, J.Yang, ... & Y. Zhao, (2020). ROSE: a retinal OCT-angiography vessel segmentation dataset and new model. IEEE transactions on medical imaging, 40(3), 928-939.
[14]M.Li, Y.Zhang, Z.Ji, K. Xie, S.Yuan, Q.Liu, & Q. Chen, (2020). Ipn-v2 and octa-500: Methodology and dataset for retinal image segmentation. arXiv preprint arXiv:2012.07261.
[15]C.Tomasi, & R. Manduchi, (1998, January). Bilateral filtering for gray and color images. In Sixth international conference on computer vision (IEEE Cat. No. 98CH36271) (pp. 839-846). IEEE.
[16]A. M. Reza, (2004). Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. Journal of VLSI signal processing systems for signal, image and video technology, 38(1), 35-44.
[17]S.Rahman, M. M. Rahman, M.Abdullah-Al-Wadud, G. D.Al-Quaderi, & M. Shoyaib, (2016). An adaptive gamma correction for image enhancement. EURASIP Journal on Image and Video Processing, 2016(1), 1-13.
[18]C.Shorten, & T. M.Khoshgoftaar, (2019). A survey on image data augmentation for deep learning. Journal of big data, 6(1), 1-48.
[19]C.Nwankpa, W.Ijomah, A.Gachagan, & S. Marshall, (2018). Activation functions: Comparison of trends in practice and research for deep learning. arXiv preprint arXiv:1811.03378.
[20]A.F.Agarap(2018). Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375..
[21]A.Krizhevsky, I.Sutskever, & G. E.Hinton, (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25.
[22]C.Szegedy, W.Liu,Y. Jia, P.Sermanet, S.Reed, D.Anguelov, ... & A. Rabinovich, (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).
[23]X.Li, H.Chen, X.Qi, Q.Dou, C.W.Fu, & P. A. Heng, (2018). H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE transactions on medical imaging, 37(12), 2663-2674.
[24]Y.Weng, T.Zhou, Y. Li, & X.Qiu, (2019). Nas-unet: Neural architecture search for medical image segmentation. IEEE Access, 7, 44247-44257.
[25]Z.Zeng, W. Xie, , Y.Zhang, , & Y. Lu, (2019). RIC-Unet: An improved neural network based on Unet for nuclei segmentation in histology images. Ieee Access, 7, 21420-21428.
[26]K.Naveed, F.Abdullah, H. A.Madni, M. A. Khan, T. M.Khan, ,& S. S.Naqvi, (2021). Towards automated eye diagnosis: an improved retinal vessel segmentation framework using ensemble block matching 3D filter. Diagnostics, 11(1), 114.
[27]K.Naveed, F.Abdullah, H. A.Madni, M. A.Khan, T. M. Khan, & S. S.Naqvi, (2021). Towards automated eye diagnosis: an improved retinal vessel segmentation framework using ensemble block matching 3D filter. Diagnostics, 11(1), 114.
[28]M.Chala, B. Nsiri, M. H.El yousfi Alaoui, A.Soulaymani, A. Mokhtari, & B.Benaji, (2021). An automatic retinal vessel segmentation approach based on Convolutional Neural Networks. Expert Systems with Applications, 184, 115459.
[29]M. R. K.Mookiah, S.Hogg, T.MacGillivray, & E. Trucco, (2021). On the quantitative effects of compression of retinal fundus images on morphometric vascular measurements in VAMPIRE. Computer Methods and Programs in Biomedicine, 202, 105969.
[30]K.Balasubramanian, & N. P.Ananthamoorthy, (2021). Robust retinal blood vessel segmentation using convolutional neural network and support vector machine. Journal of Ambient Intelligence and Humanized Computing, 12(3), 3559-3569.
[31]S. M.Boubakar Khalifa Albargathe, E.Kamberli, F.Kandemirli, & J. Rahebi, (2021). Blood vessel segmentation and extraction using H-minima method based on image processing techniques. Multimedia Tools and Applications, 80(2), 2565-2582.
[32]K.Mardani, & K.Maghooli, (2021). Enhancing retinal blood vessel segmentation in medical images using combined segmentation modes extracted by DBSCAN and morphological reconstruction. Biomedical Signal Processing and Control, 69, 102837.
[33]U.Dikkala, M. K.Joseph, & M.Alagirisamy, (2021, February). A comprehensive analysis of morphological process dependent retinal blood vessel segmentation. In 2021 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) (pp. 510-516). IEEE.
[34]C. Y.Cheung, D.Xu, C. Y.Cheng, C.Sabanayagam, Y. C.Tham, M.Yu, ... & T. Y. Wong, (2021). A deep-learning system for the assessment of cardiovascular disease risk via the measurement of retinal-vessel calibre. Nature biomedical engineering, 5(6), 498-508.
[35]Y.Zhou, H.Yu, & H.Shi, (2021, September). Study group learning: Improving retinal vessel segmentation trained with noisy labels. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 57-67). Springer, Cham.
[36]O.Ramos-Soto, E.Rodríguez-Esparza, S. E. Balderas-Mata, D.Oliva, A. E.Hassanien, R. K.Meleppat, & R. J.Zawadzki, (2021). An efficient retinal blood vessel segmentation in eye fundus images by using optimized top-hat and homomorphic filtering. Computer Methods and Programs in Biomedicine, 201, 105949.
[37]B.Toptaş, & D. Hanbay, (2021). Retinal blood vessel segmentation using pixel-based feature vector. Biomedical Signal Processing and Control, 70, 103053
[38]A. A.Abdulsahib, M. A. Mahmoud, M. A.Mohammed, H. H.Rasheed, S. A.Mostafa, & M. S. Maashi, (2021). Comprehensive review of retinal blood vessel segmentation and classification techniques: intelligent solutions for green computing in medical images, current challenges, open issues, and knowledge gaps in fundus medical images. Network Modeling Analysis in Health Informatics and Bioinformatics, 10(1), 1-32..
[39]N. A.El‐Hag, A.Sedik, W.El‐Shafai, H. M.El‐Hoseny, A. A.Khalaf, A. S.El‐Fishawy, ... & G. M.El‐Banby, (2021). Classification of retinal images based on convolutional neural network. Microscopy Research and Technique, 84(3), 394-414.
[40]M.Yeung, E. Sala, C. B.Schönlieb, & L.Rundo, (2022). Unified focal loss: Generalising dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Computerized Medical Imaging and Graphics, 95, 102026.
[41]M.Yeung, E.Sala, C. B.Schönlieb, & L.Rundo, (2021). Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy. Computers in Biology and Medicine, 137, 104815.
[42]M. S.Moustafa, S. A.Mohamed, S. Ahmed, & A. H.Nasr, (2021). Hyperspectral change detection based on modification of UNet neural networks. Journal of Applied Remote Sensing, 15(2), 028505.
[43]M. S.Moustafa, S. A.Mohamed, S.Ahmed, & A. H.Nasr, (2021). Hyperspectral change detection based on modification of UNet neural networks. Journal of Applied Remote Sensing, 15(2), 028505.
[44]T.Sugino, T.Kawase, S.Onogi, T.Kin, N.Saito, & Y.Nakajima, (2021, August). Loss weightings for improving imbalanced brain structure segmentation using fully convolutional networks. In Healthcare (Vol. 9, No. 8, p. 938). Multidisciplinary Digital Publishing Institute.
[45]Z.Wu, ,C.Shen, & A.Van Den Hengel, (2019). Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 90, 119-133.
[46]C.Szegedy, S.Ioffe, V.Vanhoucke, & A. A. Alemi, (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.
[47]H.Lin, & S. Jegelka, (2018). Resnet with one-neuron hidden layers is a universal approximator. Advances in neural information processing systems, 31.
[48]M.Farooq, & A.Hafeez, (2020). Covid-resnet: A deep learning framework for screening of covid19 from radiographs. arXiv preprint arXiv:2003.14395.
[49]Z.Chen, Z.Xie, W.Zhang, & X.Xu, (2017, August). ResNet and Model Fusion for Automatic Spoofing Detection. In Interspeech (pp. 102-106).
[50]L.Wen, X. Li, & L. Gao, (2020). A transfer convolutional neural network for fault diagnosis based on ResNet-50. Neu ral Computing and Applications, 32(10), 6111-6124.
[51]V.Badrinarayanan, A.Kendall, & R.Cipolla, (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12), 2481-2495.
[52]A.Kendall, V.Badrinarayanan, & R.Cipolla, (2015). Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. arXiv preprint arXiv:1511.02680.
[53]S.Almotairi, G.Kareem, M.Aouf, B.Almutairi, & M. A. M.Salem, (2020). Liver tumor segmentation in CT scans using modified SegNet. Sensors, 20(5), 1516.
[54]A.Saood, & I.Hatem, (2021). COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet. BMC Medical Imaging, 21(1), 1-10.
[55]J.Dai, Y.Li, K.He, & J.Sun, (2016). R-fcn: Object detection via region-based fully convolutional networks. Advances in neural information processing systems, 29.
[56]P.Siano, C. Cecati, H.Yu, & J.Kolbusz, (2012). Real time operation of smart grids via FCN networks and optimal power flow. IEEE Transactions on Industrial Informatics, 8(4), 944-952.
[57]J.Ashburner, & K. J.Friston, (2005). Unified segmentation. Neuroimage, 26(3), 839-851.
[58]J.Ma, J.Chen, M.Ng, R. Huang, Y.Li, C.Li, ... & A. L.Martel, (2021). Loss odyssey in medical image segmentation. Medical Image Analysis, 71, 102035.
[59]R.Strudel, R. Garcia, I.Laptev, & C.Schmid, (2021). Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7262-7272).
[60]S.Akbar, M.Sharif, M. U. Akram, T.Saba, T.Mahmood, & M.Kolivand, (2019). Automated techniques for blood vessels segmentation through fundus retinal images: A review. Microscopy research and technique, 82(2), 153-170.
[61]R. C.Gonzales, & R. E.Woods, (2018). Digital image processing 4th edition.
[62]C.Szegedy, S.Ioffe, V.Vanhoucke, & A. A.Alemi, (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.