臺灣博碩士論文加值系統

目的：本研究主要針對第二型糖尿病患者之視網膜眼底影像，目的是透過人工智慧技術預測患者罹患糖尿病周邊神經病變(Diabetic peripheral neuropathy, DPN)之概率，最終建立可解釋性人工智慧模型(Explainable AI, XAI)，透過該模型之判讀所勾勒出的關注區域(region of interest, ROI)，得出糖尿病患者之視網膜眼底影像與DPN於醫學中的關聯性。
材料與方法：本研究自高雄市立大同醫院(KMTTH)與高雄醫學大附設醫院(KMUH)內分泌新陳代謝科門診所建立之糖尿病照護資料庫，蒐集2013至2017年之間第二型糖尿病患者之眼底攝影影像以進行回顧性研究，本研究排除具備以下條件之糖尿病患者數據：年齡低於20歲之患者、姙娠期之婦女、因糖尿病併發生導致截肢者、患有惡性腫瘤或診斷罹癌者。經由臨床常規治療且有定期排檢眼底鏡檢查之第二型糖尿病患者，會根據神經傳導檢查(Nerve conduction velocity, NCV)判斷患者周邊神經病變之嚴重程度並進行分類，本研究定義患者在接受神經傳導檢查後的嚴重程度類別為Class 0：無DPN症狀患者；Class 1：輕度DPN症狀患者；Class 2：中、重度DPN症狀患者。患者之眼底影像資料首先去除文字資訊與背景以去識別化，並透過限制對比度的自適應直方圖均衡化(CLAHE)作為影像前處理之方法。經過前處理之影像被分割為訓練、驗證與測試集，再以 ±45° 旋轉進行資料增強，並個別透過InceptionNet、VGGNet、ResNet、ConvMixer等深度學習架構建立預測模型。本研究使用RAdam作為模型訓練時的參數優化器進而穩健化參數更新之效果，並透過受試者特性曲線下面積(Area under the ROC curve, AUC)以及陽性預測值(Positive predictive value, PPV)等指標對個別模型進行評估。在評估模型之預測準確度後，使用Grad-CAM(Gradient-weighted Class activation mapping)算法得出糖尿病患者之眼底影像中，模型所判斷與糖尿病周邊神經病變(DPN)具有關聯性之區域。
結果：本研究成功透過四種深度學習架構建立預測DPN嚴重程度之三元分類模型。所建立的四種DPN預測模型分別為PT-Attention、ConvMixer、ResNet34、PT-VGG16，四種模型在本研究中的測試集(testing set)上的準確度(accuracy)分別達到0.94、0.90、0.97、0.96；AUC值達到0.92、0.93、0.95、0.96；特異性(specificity)達到1.00、0.92、1.00、0.98；輕度與中重度DPN的總體敏感性(sensitivity)則達到0.84、0.90、0.90、0.92。在本研究中，以所使用的評估指標判斷得出所有模型均擁有良好的DPN預測能力，其中PT-VGG16模型取得了最穩健的DPN預測能力。
結論：本研究以人工智慧領域中的深度學習(Deep learning)，透過第二型糖尿病患者的視網膜眼底影像成功建立了糖尿病周邊神經病變(DPN)嚴重程度的AI輔助診斷模型，這一種快速的非侵入式DPN檢測方法，僅需要透過患者於眼底鏡檢測後所得出的視網膜眼底影像，便能夠判斷出該名患者是否罹患有DPN以及其嚴重程度，該方法為視網膜眼底影像增添了一種額外的貢獻，也增加了醫師能及早診斷出患者是否患有糖尿病周邊神經病變(DPN)的可能性。本研究亦透過基於Grad-CAM的影像ROI映射方法，將有助於模型以所訓練的權重加權對重視的部位(pattern)進行標註，呈現出AI模型的可解釋性，最終成功勾勒出在糖尿病患者的視網膜眼底影像中，與DPN相關聯之區域的ROI影像。

Purpose: This study focused on the retinal fundus images of type 2 diabetes patients. The purpose was to predict the probability of diabetic peripheral neuropathy (DPN) in patients through artificial intelligence technology, build an explainable AI model, and finally to derive the association between retinal fundus images and DPN in medicine in diabetic patients through the region of interest (ROI) outlined by the interpretation of the prediction model.
Materials and methods: This study collected fundus photography images of type II diabetes patients between 2013 and 2017 from the Diabetes Care Database established by the Endocrine and Metabolism Department Outpatient Clinic of Kaohsiung Municipal Ta-Tung Hospital (KMTTH) and Kaohsiung Municipal United Hospital (KMUH) to conduct a retrospective study. This study excluded data of diabetes patients with the following conditions: patients under 20 years of age, women during pregnancy, those with amputation due to diabetes complications, and those suffering from malignant tumors or diagnosed with cancer. Patients with type II diabetes who were treated clinically and had regular ophthalmoscopes were classified according to the severity of peripheral neuropathy by nerve conduction velocity (NCV). In this study, patients were classified as one of the following: Class 0 = patients with no DPN symptoms; Class 1 = patients with mild DPN symptoms; and Class 2 = patients with moderate or severe DPN symptoms, according to their severity level after NCV. The patients’ fundus image data were first de-identified by removing textual information and background and were pre-processed by contrast-limited adaptive histogram equalization (CLAHE). The pre-processed images were segmented into training, validation, and testing sets, then the data were enhanced by ±45° rotation. The prediction models were built by deep learning architecture such as InceptionNet, VGGNet, ResNet, and ConvMixer. In this study, RAdam was used as a parameter optimizer for model training to stabilize the effect of parameter updating, and individual models were evaluated by the indicators accuracy (ACC), area under the ROC curve (AUC), and positive predictive value (PPV). After evaluating the prediction accuracy of the model, the gradient-weighted class activation mapping(Grad-CAM) method was used to derive the areas in the fundus images of diabetic patients that the model determined were associated with diabetic peripheral neuropathy (DPN).
Results: In this study, retinal fundus images obtained from type II diabetic patients after routine ophthalmoscopy were used as a dataset, and a three-dimensional classification model to predict the severity of DPN was successfully developed using four deep learning architectures. The four DPN prediction models established were PT-Attention, ConvMixer, ResNet34, and PT-VGG16. The accuracy of the four models on the testing set in this study reached 0.94, 0.90, 0.97, and 0.96, respectively. In this study, all models have good DPN predictive ability as shown by the evaluation indexes used. The PT-VGG16 model achieved the most robust DPN prediction ability.
Conclusions: This study successfully developed an AI-assisted diagnosis model of diabetic peripheral neuropathy (DPN) severity from retinal fundus images of patients with type II diabetes using deep learning. It contributes to retinal fundus imaging and increases the likelihood that physicians will be able to diagnose diabetic peripheral neuropathy (DPN) early. We also used the Grad-CAM-based image ROI mapping method to annotate the patterns, which are the important parts with the trained weights, presenting the explainability of the AI model, and finally successfully showed the ROI images of the area associated with DPN in the retinal fundus images of diabetic patients.

摘要 ................................................................................................................. I
ABSTRACT ................................................................................................... III
致謝 ............................................................................................................... VI
目錄 .............................................................................................................. VII
表目錄 ............................................................................................................ IX
圖目錄 ............................................................................................................. X
符號縮寫 ........................................................................................................ XI
符號說明 ....................................................................................................... XII
第一章 緒論 ................................................................................................... 1
1.1 動機 ......................................................................................................... 1
1.2 目的 ......................................................................................................... 3
1.3 相關文獻探討 ........................................................................................... 4
第二章 材料與方法 ......................................................................................... 5
2.1 患者特性資料 ........................................................................................... 7
2.1.1 患者之特徵因子資料分布圖.................................................................... 10
2.1.2 患者進行眼底鏡攝影檢查之設備............................................................. 11
2.2 診斷糖尿病周邊神經病變之技術 ............................................................... 11
2.3 糖尿病與周邊神經病變併發症 ................................................................... 12
2.4 視網膜眼底影像之預處理 .......................................................................... 13
2.5 資料集之準備與劃分 .................................................................................. 15
2.5.1 資料增強.................................................................................................. 15
2.5.2 資料集之劃分........................................................................................... 15
2.6 神經網路之建構與訓練 ............................................................................... 16
2.6.1 批次訓練資料之調整................................................................................. 16
2.6.2 RAdam optimizer ..................................................................................... 17
2.6.3 Dynamic data generator .......................................................................... 17
2.6.4 PT-Attention ............................................................................................. 18
2.6.5 ConvMixer ................................................................................................ 19
2.6.6 ResNet34 ................................................................................................. 20
2.6.7 PT-VGG16 ................................................................................................ 21
2.7 糖尿病周邊神經病變模型之效力評估指標 .................................................... 22
2.7.1 Cross entropy loss function ...................................................................... 22
2.7.2 Categorical accuracy ................................................................................ 23
2.7.3 Sensitivity ................................................................................................. 23
2.7.4 Specificity ................................................................................................. 24
2.7.5 Positive predictive value ........................................................................... 24
2.7.6 F1 score .................................................................................................... 24
2.7.7 Area under the ROC Curve ....................................................................... 25
2.8 糖尿病周邊神經病變模型之預測關注區域可視化方法 .................................. 26
2.8.1 Class Activation Mapping ......................................................................... 26
2.8.2 Gradient-weighted Class Activation Mapping ........................................... 27
2.8.3 Layer-CAM ................................................................................................ 29
第三章 結果 ........................................................................................................ 31
3.1 預測糖尿病周邊神經病變之分類模型之預測效力評估結果 ........................... 31
3.1.1 糖尿病周邊神經病變三元分類模型之混淆矩陣與ROC曲線 ....................... 32
3.2 糖尿病周邊神經病變併發症在眼底影像之映射區可視化 ............................... 34
第四章 討論 ......................................................................................................... 39
4.1 相關文獻探討 ................................................................................................ 40
4.2 DPN診斷模型之侷限性 ................................................................................. 42
4.2.1 神經或皮膚的病理切片檢查........................................................................ 43
4.2.2 角膜共軛焦顯微鏡檢查............................................................................... 43
第五章 結論 ........................................................................................................ 44

[1] A. J. Boulton, "Management of diabetic peripheral neuropathy," Clinical diabetes, vol. 23, no. 1, pp. 9-15, 2005.
[2] P. R. Vas, S. Sharma, and G. Rayman, "Distal sensorimotor neuropathy: improvements in diagnosis," The review of diabetic studies: RDS, vol. 12, no. 1-2, p. 29, 2015.
[3] S. Tesfaye et al., "Diabetic neuropathies: update on definitions, diagnostic criteria, estimation of severity, and treatments," Diabetes care, vol. 33, no. 10, pp. 2285-2293, 2010.
[4] S. Tesfaye et al., "Painful diabetic peripheral neuropathy: consensus recommendations on diagnosis, assessment and management," Diabetes/metabolism research and reviews, vol. 27, no. 7, pp. 629-638, 2011.
[5] Z. Li et al., "Deep learning methods for lung cancer segmentation in whole-slide histopathology images—the acdc@ lunghp challenge 2019," Journal of Biomedical and Health Informatics (JBHI), vol. 25, no. 2, pp. 429-440, 2020.
[6] G. Wang et al., "A deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images," Nature biomedical engineering, vol. 5, no. 6, pp. 509-521, 2021.
[7] A. Shahidi et al., "Retinal nerve fibre layer thinning associated with diabetic peripheral neuropathy," Diabetic Medicine, vol. 29, no. 7, pp. e106-e111, 2012.
[8] K. Sinharay, U. K. Paul, A. K. Bhattacharyya, and S. K. Pal, "Prevalence of diabetic foot ulcers in newly diagnosed diabetes mellitus patients," Journal of the Indian Medical Association, vol. 110, no. 9, pp. 608-611, 2012.
[9] M. H. Megallaa, A. A. Ismail, M. H. Zeitoun, and M. S. Khalifa, "Association of diabetic foot ulcers with chronic vascular diabetic complications in patients with type 2 diabetes," Diabetes & Metabolic Syndrome: Clinical Research & Reviews, vol. 13, no. 2, pp. 1287-1292, 2019.
[10] N. Tsiknakis et al., "Deep learning for diabetic retinopathy detection and classification based on fundus images: A review," Computers in Biology and Medicine, vol. 135, p. 104599, 2021.
[11] N. Nurrahmadayeni, S. Efendi, and M. Zarlis, "Analysis of deep learning methods in diabetic retinopathy disease identification based on retinal fundus image," International Journal of Nonlinear Analysis and Applications, vol. 13, no. 1, pp. 1639-1647, 2022.
[12] S. Dixit and A. Maiya, "Diabetic peripheral neuropathy and its evaluation in a clinical scenario: a review," Journal of postgraduate medicine, vol. 60, no. 1, p. 33, 2014.
[13] J. C. Won and T. S. Park, "Recent advances in diagnostic strategies for diabetic peripheral neuropathy," Endocrinology and metabolism, vol. 31, no. 2, pp. 230-238, 2016.
[14] J. England et al., "Distal symmetric polyneuropathy: a definition for clinical research: report of the American Academy of Neurology, the American Association of Electrodiagnostic Medicine, and the American Academy of Physical Medicine and Rehabilitation," Neurology, vol. 64, no. 2, pp. 199-207, 2005.
[15] K. Van Acker et al., "Prevalence and impact on quality of life of peripheral neuropathy with or without neuropathic pain in type 1 and type 2 diabetic patients attending hospital outpatients clinics," Diabetes & metabolism, vol. 35, no. 3, pp. 206-213, 2009.
[16] S. Tesfaye, A. J. Boulton, and A. H. Dickenson, "Mechanisms and management of diabetic painful distal symmetrical polyneuropathy," Diabetes care, vol. 36, no. 9, pp. 2456-2465, 2013.
[17] R. Pop-Busui et al., "Diabetic neuropathy: a position statement by the American Diabetes Association," Diabetes care, vol. 40, no. 1, pp. 136-154, 2017.
[18] C. Swathi, B. Anoop, D. A. S. Dhas, and S. P. Sanker, "Comparison of different image preprocessing methods used for retinal fundus images," in Conference on Emerging Devices and Smart Systems (ICEDSS), 2017: IEEE.
[19] N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. P. Tang, "On large-batch training for deep learning: Generalization gap and sharp minima," in International Conference on Learning Representations (ICLR), 2017.
[20] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in International Conference on Learning Representations (ICLR), 2015.
[21] L. Liu et al., "On the variance of the adaptive learning rate and beyond," in International Conference on Learning Representations (ICLR), 2020.
[22] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Conference on Computer Vision and Pattern Recognition (CVPR), 2016: IEEE.
[23] A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, pp. 5998--6008, 2017.
[24] A. Dosovitskiy et al., "An image is worth 16x16 words: Transformers for image recognition at scale," in International Conference on Learning Representations (ICLR), 2021.
[25] A. Trockman and J. Z. Kolter, "Patches Are All You Need?," in International Conference on Learning Representations (ICLR), 2022.
[26] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Conference on Computer Vision and Pattern Recognition (CVPR), 2016: IEEE.
[27] K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks," in European Conference on Computer Vision (ECCV), 2016: Springer.
[28] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in International Conference on Learning Representations (ICLR), 2015.
[29] K. Weiss, T. M. Khoshgoftaar, and D. Wang, "A survey of transfer learning," Journal of Big data, vol. 3, no. 1, pp. 1-40, 2016.
[30] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Conference on Computer Vision and Pattern Recognition (CVPR), 2016: IEEE.
[31] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization," in International Conference on Computer Vision (ICCV), 2017: IEEE.
[32] P.-T. Jiang, C.-B. Zhang, Q. Hou, M.-M. Cheng, and Y. Wei, "Layercam: Exploring hierarchical class activation maps for localization," IEEE Transactions on Image Processing, vol. 30, pp. 5875-5888, 2021.
[33] D. R. Cervera, L. Smith, L. Diaz-Santana, M. Kumar, R. Raman, and S. Sivaprasad, "Identifying Peripheral Neuropathy in Colour Fundus Photographs Based on Deep Learning," Diagnostics, vol. 11, no. 11, p. 1943, 2021.
[34] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size," in International Conference on Learning Representations (ICLR), 2017.
[35] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Conference on Computer Vision and Pattern Recognition (CVPR), 2017: IEEE.
[36] J. Benson, T. Estrada, M. Burge, and P. Soliz, "Diabetic peripheral neuropathy risk assessment using digital fundus photographs and machine learning," in Engineering in Medicine and Biology Society (EMBC), 2020: IEEE.
[37] S. Divisova et al., "Prediabetes/early diabetes-associated neuropathy predominantly involves sensory small fibres," Journal of the Peripheral Nervous System, vol. 17, no. 3, pp. 341-350, 2012.
[38] A. I. Vinik, M.-L. Nevoret, C. Casellini, and H. Parson, "Diabetic neuropathy," Endocrinology and Metabolism Clinics, vol. 42, no. 4, pp. 747-787, 2013.
[39] A. Breiner, L. E. Lovblom, B. A. Perkins, and V. Bril, "Does the prevailing hypothesis that small-fiber dysfunction precedes large-fiber dysfunction apply to type 1 diabetic patients?," Diabetes care, vol. 37, no. 5, pp. 1418-1424, 2014.
[40] N. Papanas and D. Ziegler, "Corneal confocal microscopy: a new technique for early detection of diabetic neuropathy," Current diabetes reports, vol. 13, no. 4, pp. 488-499, 2013.
[41] X. Chen et al., "Small nerve fiber quantification in the diagnosis of diabetic sensorimotor polyneuropathy: comparing corneal confocal microscopy with intraepidermal nerve fiber density," Diabetes care, vol. 38, no. 6, pp. 1138-1144, 2015.