臺灣博碩士論文加值系統

目的 : 由於癌症患者接受放射療法時所引起的放射性皮膚炎會依據時間及個體差異導致其嚴重程度有所不同，因此本研究以乳癌患部影像為例使用機器學習預測一周後皮膚炎級別，並使用生成對抗網路模擬患部皮膚炎影像，協助相關醫護人員決策患者放射治療計畫、護理方法與衛教指導從而減輕放射治療之副作用。

材料與方法 : 本研究資料集之機構審查委員會(Institutional Review Board, IRB)編號為EMRP29104N及EMRP110089，該試驗於2016年至2017年期間招募62位接受放射療法的乳癌患者，並獲取了他們25日至30日的療程期間去識別化之患部影像及醫療紀錄。本研究使用基礎影像處理方法補正影像亮度、飽和度及色溫，並手動截取患部區塊以減少對象雜訊。影像特徵提取方法首先採用RGB、HSV、LAB及yCrCb四種色彩空間作為影像色彩信息，接著灰度共生矩陣(Gray-Level Co-occurrence Matrix, GLCM)被用作提取影像紋理，而每張影像之攝影日期則作為時間特徵，經拼接後的特徵藉由8種機器學習方法建置模型，並用於分類及預測影像之皮膚炎級別，透過比較與評估以上方法之模型成效用以提供患者皮膚炎之分級機率；CycleGAN與pix2pix方法則利用患者輕微與嚴重之影像資料作為風格遷移(Style Transfer)之依據，從而模擬該患者嚴重皮膚炎之參考影像，最後透過整合預測與影像生成模型來開發輔助診斷系統。

結果 : 實驗影像資料經由影像亮度、飽和度及色溫補正，其色差結果顯示補正後的影像色差顯著降低，且手動截取患部影像之RGB分析結果雜訊明顯減少。8種機器學習方法於分類及預測結果採用準確率(Accuracy, Acc)與曲線下面積(Area Under the Curve, AUC)評估，評估結果採用不同資料分布共執行30次取其平均值，其中分級模型用於驗證提取的特徵是否有助於預測皮膚炎級別，最佳分級模型為LGBoost，Acc與AUC分別為0.84與0.899，表示所提取之色彩及紋理與皮膚炎級別有關；而最佳預測模型為CatBoost，其Acc與AUC平均分別為0.816及0.874，與無時間特徵的模型相比，CatBoost的Acc與AUC上升了0.102與0.171，表示添加時間特徵有助於未來一週皮膚炎影像預測之問題。CycleGAN與pix2pix之影像生成結果相比，pix2pix所產生之影像無法保持同一位患者的輪廓特徵，進而使影像出現破損情形；CycleGAN所生成影像則更貼近輸入影像的輪廓特徵，並能模擬嚴重程度的放射性皮膚炎影像，因此CycleGAN影像生成效果更適用於本研究之目的。

結論：影像特徵加入時間特徵並使用CatBoost方法具有放射性皮膚炎級別的最佳預測準確度，所預測結果可協助醫護人員診斷並決策放療計畫與護理措施。而CycleGAN相比pix2pix具有更好的影像生成品質，其生成之嚴重皮膚炎影像可能有助於提高患者衛教順從度。

Purpose: Because the severity of radiation dermatitis caused by cancer patients, who are receiving radiotherapy will vary with time and individual differences, this study uses breast cancer patients as an example through using machine learning and style transfer technology to predict and simulate the level of dermatitis. So as to help the medical staff finding the patients with possible worsening dermatitis as soon as possible to reduce the side effects of radiotherapy.

Materials and methods: The Institutional Review Board (IRB) numbers of the data set for this study are EMRP29104N and EMRP110089. The trial recruited 62 breast cancer patients who received radiotherapy from 2016 to 2017, and obtained de-identified images and medical records of the affected area during the course of treatment from 25 to 30 days. This study uses basic image processing methods to correct the brightness, saturation and color temperature of the image, and manually intercept the affected area to reduce object noise. Image feature extraction uses four color spaces, RGB, HSV, LAB and yCrCb, as image color information. Gray-Level Co-occurrence Matrix (GLCM) extracts image-related texture features, and the shooting date of each image is used as time features. The combined features are used to build a model by using 8 machine learning methods, and they are still used to classify and predict the level of dermatitis in the image. Compare and evaluate the effectiveness of the above methods to provide the grading probability of patients with dermatitis. The CycleGAN and pix2pix methods use the patient’s mild to severe image data as the basis for style transfer to simulate the reference image of the patient’s severe dermatitis, and finally develop an auxiliary diagnosis system by integrating the prediction and image generation model.

Results: The experimental image data is corrected by the method of image brightness, saturation, and color temperature. The color difference results are displayed in a two-dimensional scatter chart. The color difference result is smaller than the corrected image color difference result displayed in the two-dimensional scatter chart and the noise of the RGB analysis result of manually intercepting the image of the affected area is significantly reduced. The classification and prediction results of 8 machine learning methods are evaluated using Accuracy (Accuracy, Acc) and Area Under the Curve (AUC). The evaluation results are evaluated using different data distributions for a total of 30 times and the average is taken. The classification and prediction results of 8 machine learning methods are evaluated using Accuracy (Accuracy, Acc) and Area Under the Curve (AUC). The evaluation results are evaluated using different data distributions for a total of 30 times and the average is taken. The analysis shows that the best prediction model is CatBoost. Its Acc and AUC averages are 0.816 and 0.874. Compared with the model without time features, CatBoost's Acc and AUC have increased by 0.102 and 0.171, which means that time features help predict the future Image problem of a week. The image generation results of CycleGAN and pix2pix found that the images produced by pix2pix may cause the loss of the patient’s identity, because the image can be distorted by using pix2pix. However, CycleGAN can preserve the identity, so that the contour features of the generated images remain the same for the patient. Therefore, CycleGAN has a better image generation effect.

Conclusion: The image features are added with time features and the CatBoost method is used to have the best prediction accuracy of radiation dermatitis level. The predicted results can assist medical staff in the diagnosis and decision-making of radiotherapy plans and nursing measures. CycleGAN has better image generation quality than pix2pix, and the generated images of severe dermatitis may help improve patient compliance with health education.

摘 要 i
Abstract iii
致謝 vi
目錄 vii
表目錄 ix
圖目錄 x
符號縮寫 xi
第一章 緒論 1
1.1 動機 1
1.2 目的 3
1.3 相關文獻探討 4
第二章 材料與方法 6
2.1 前言 6
2.2 研究資料與分析 8
2.3 影像品質分析與探討 10
2.3.1 影像亮度及飽和度失真 10
2.3.2 影像色溫失真 11
2.3.3 輪廓與攝影範圍分析 12
2.4 色差補正 13
2.4.1 色溫補正 16
2.4.2 影像患部擷取 18
2.5 特徵提取(Feature extraction) 19
2.5.1 色彩特徵提取 20
2.5.2 紋理特徵提取 21
2.5.3 時間特徵提取 25
2.6 機器學習分類及預測方法 26
2.7 患部影像生成模型 30
2.7.1 pix2pix模型簡介 31
2.7.2 CycleGAN模型簡介 33
2.8 程式及開發環境介紹 35
第三章 結果 36
3.1 前言 36
3.2 色彩補正之結果 37
3.3 患部擷取之示例結果 38
3.4 影像前處理之個案示例 39
3.5 皮膚炎分級之結果 40
3.6 皮膚炎下週級別預測之結果 41
3.7 pix2pix影像生成結果 42
3.8 CycleGAN影像生成結果 43
3.9 放射性皮膚炎使用者輔助診斷介面 44
3.9.1使用者介面設計 45
3.9.2使用者步驟 48
第四章 討論 51
4.1 前言 51
4.2 研究貢獻 52
4.3 影像色彩補正結果討論 53
4.4 時間特徵於分類及預測之影響 54
4.5 Cycle GAN迭代評估 55
4.6 pix2pix與CycleGAN皮膚炎生成結果差異 56
第五章 結論 57
參考文獻 58
自傳 62

[1] R. Baskar, K. A. Lee, R. Yeo, and K.-W. Yeoh, "Cancer and radiation therapy: current advances and future directions," International journal of medical sciences, vol. 9, p. 193, 2012.
[2] S. P. Jackson and J. Bartek, "The DNA-damage response in human biology and disease," Nature, vol. 461, pp. 1071-1078, 2009.
[3] T. J. Whelan, J.-P. Pignol, M. N. Levine, J. A. Julian, R. MacKenzie, S. Parpia, et al., "Long-term results of hypofractionated radiation therapy for breast cancer," New England Journal of Medicine, vol. 362, pp. 513-520, 2010.
[4] A. C. Begg, F. A. Stewart, and C. Vens, "Strategies to improve radiotherapy with targeted drugs," Nature Reviews Cancer, vol. 11, pp. 239-253, 2011.
[5] B. J. Erickson, P. Korfiatis, Z. Akkus, and T. L. Kline, "Machine learning for medical imaging," Radiographics, vol. 37, pp. 505-515, 2017.
[6] M. L. Giger, "Machine learning in medical imaging," Journal of the American College of Radiology, vol. 15, pp. 512-520, 2018.
[7] J. Ferlay, "GLOBOCAN 2008 v1. 2, Cancer incidence and mortality world-wide: IARC Cancer Base No. 10," iarc, 2010.
[8] C. L. Chaffer and R. A. Weinberg, "A perspective on cancer cell metastasis," science, vol. 331, pp. 1559-1564, 2011.
[9] J. Sia, R. Szmyd, E. Hau, and H. E. Gee, "Molecular mechanisms of radiation-induced cancer cell death: A primer," Frontiers in cell and developmental biology, vol. 8, p. 41, 2020.
[10] G. C. Barnett, C. M. West, A. M. Dunning, R. M. Elliott, C. E. Coles, P. D. Pharoah, et al., "Normal tissue reactions to radiotherapy: towards tailoring treatment dose by genotype," Nature Reviews Cancer, vol. 9, pp. 134-142, 2009.
[11] G. Ross, "Induction of cell death by radiotherapy," Endocrine-related cancer, vol. 6, pp. 41-44, 1999.
[12] S. H. Giordano, Y.-F. Kuo, J. L. Freeman, T. A. Buchholz, G. N. Hortobagyi, and J. S. Goodwin, "Risk of cardiac death after adjuvant radiotherapy for breast cancer," Journal of the National Cancer Institute, vol. 97, pp. 419-424, 2005.
[13] A. M. Rose-Ped, L. A. Bellm, J. B. Epstein, A. Trotti, C. Gwede, and H. J. Fuchs, "Complications of radiation therapy for head and neck cancers: the patient’s perspective," Cancer nursing, vol. 25, pp. 461-467, 2002.
[14] C. E. Stubbe and M. Valero, "Complementary strategies for the management of radiation therapy side effects," Journal of the advanced practitioner in oncology, vol. 4, p. 219, 2013.
[15] S. M. Bentzen, "Preventing or reducing late side effects of radiation therapy: radiobiology meets molecular pathology," Nature Reviews Cancer, vol. 6, pp. 702-713, 2006.
[16] M. S. Gujral, P. M. Patnaik, R. Kaul, H. K. Parikh, C. Conradt, C. P. Tamhankar, et al., "Efficacy of hydrolytic enzymes in preventing radiation therapy-induced side effects in patients with head and neck cancers," Cancer chemotherapy and pharmacology, vol. 47, pp. S23-S28, 2001.
[17] S. Kumar, E. Juresic, M. Barton, and J. Shafiq, "Management of skin toxicity during radiation therapy: a review of the evidence," Journal of medical imaging and radiation oncology, vol. 54, pp. 264-279, 2010.
[18] A. Yala, T. Schuster, R. Miles, R. Barzilay, and C. Lehman, "A deep learning model to triage screening mammograms: a simulation study," Radiology, vol. 293, pp. 38-46, 2019.
[19] E. J. Hwang, J. G. Nam, W. H. Lim, S. J. Park, Y. S. Jeong, J. H. Kang, et al., "Deep learning for chest radiograph diagnosis in the emergency department," Radiology, vol. 293, pp. 573-580, 2019.
[20] M. R. Arbabshirani, B. K. Fornwalt, G. J. Mongelluzzo, J. D. Suever, B. D. Geise, A. A. Patel, et al., "Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration," NPJ digital medicine, vol. 1, pp. 1-7, 2018.
[21] Y. Ding, J. H. Sohn, M. G. Kawczynski, H. Trivedi, R. Harnish, N. W. Jenkins, et al., "A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain," Radiology, vol. 290, pp. 456-464, 2019.
[22] A. Parakh, H. Lee, J. H. Lee, B. H. Eisner, D. V. Sahani, and S. Do, "Urinary stone detection on CT images using deep convolutional neural networks: evaluation of model performance and generalization," Radiology: Artificial Intelligence, vol. 1, p. e180066, 2019.
[23] D. B. Larson, M. C. Chen, M. P. Lungren, S. S. Halabi, N. V. Stence, and C. P. Langlotz, "Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs," Radiology, vol. 287, pp. 313-322, 2018.
[24] P. Schelb, S. Kohl, J. P. Radtke, M. Wiesenfarth, P. Kickingereder, S. Bickelhaupt, et al., "Classification of cancer at prostate MRI: deep learning versus clinical PI-RADS assessment," Radiology, vol. 293, pp. 607-617, 2019.
[25] S. Soffer, A. Ben-Cohen, O. Shimon, M. M. Amitai, H. Greenspan, and E. Klang, "Convolutional neural networks for radiologic images: a radiologist’s guide," Radiology, vol. 290, pp. 590-606, 2019.
[26] C. Ricciardi, K. J. Edmunds, M. Recenti, S. Sigurdsson, V. Gudnason, U. Carraro, et al., "Assessing cardiovascular risks from a mid-thigh CT image: a tree-based machine learning approach using radiodensitometric distributions," Scientific reports, vol. 10, pp. 1-13, 2020.
[27] T.-T. Zhai, L. V. van Dijk, B.-T. Huang, Z.-X. Lin, C. O. Ribeiro, C. L. Brouwer, et al., "Improving the prediction of overall survival for head and neck cancer patients using image biomarkers in combination with clinical parameters," Radiotherapy and Oncology, vol. 124, pp. 256-262, 2017.
[28] M. Tajdari, A. Pawar, H. Li, F. Tajdari, A. Maqsood, E. Cleary, et al., "Image-based modelling for Adolescent Idiopathic Scoliosis: Mechanistic machine learning analysis and prediction," Computer methods in applied mechanics and engineering, vol. 374, p. 113590, 2021.
[29] M. J. Willemink, W. A. Koszek, C. Hardell, J. Wu, D. Fleischmann, H. Harvey, et al., "Preparing medical imaging data for machine learning," Radiology, vol. 295, pp. 4-15, 2020.
[30] R. C. Gonzalez and R. E. Woods, "Digital image processing," ed: Prentice hall Upper Saddle River, NJ, 2002.
[31] T. Cover and P. Hart, "Nearest neighbor pattern classification," IEEE transactions on information theory, vol. 13, pp. 21-27, 1967.
[32] L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, Classification and regression trees: CRC press, 1984.
[33] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, pp. 273-297, 1995.
[34] L. Breiman, "Random forests," Machine learning, vol. 45, pp. 5-32, 2001.
[35] J. H. Friedman, "Greedy function approximation: a gradient boosting machine," Annals of statistics, pp. 1189-1232, 2001.
[36] T. Chen and C. Guestrin, "Xgboost: A scalable tree boosting system," in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785-794.
[37] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, et al., "Lightgbm: A highly efficient gradient boosting decision tree," Advances in neural information processing systems, vol. 30, pp. 3146-3154, 2017.
[38] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, "CatBoost: unbiased boosting with categorical features," arXiv preprint arXiv:1706.09516, 2017.
[39] T. Maruyama, N. Hayashi, Y. Sato, S. Hyuga, Y. Wakayama, H. Watanabe, et al., "Comparison of medical image classification accuracy among three machine learning methods," Journal of X-ray Science and Technology, vol. 26, pp. 885-893, 2018.
[40] S. S. Yadav and S. M. Jadhav, "Deep convolutional neural network based medical image classification for disease diagnosis," Journal of Big Data, vol. 6, pp. 1-18, 2019.
[41] D. F. Bauer, T. Russ, B. I. Waldkirch, W. P. Segars, L. R. Schad, F. G. Zöllner, et al., "Generation of Multimodal Ground Truth Datasets for Abdominal Medical Image Registration Using CycleGAN," arXiv preprint arXiv:2012.01582, 2020.
[42] N. K. Singh and K. Raza, "Medical Image Generation Using Generative Adversarial Networks: A Review," Health Informatics: A Computational Perspective in Healthcare, pp. 77-96, 2021.