臺灣博碩士論文加值系統

本文探討了Alexnet、VGGnet、Inception V1、Resnet、Resnet V2、Inception V2、Inception V3、Inception V4、Inception Resnet V1和Inception Resnet V2使用到的技術，並藉由研究各代卷積神經網路技術來設計三個適用在醫療影像大腸癌組織分類上的卷積神經網路。第一，我們基於Alexnet並參考Inception V1中的Inception Module B來設計出InceptionV1Alexnet。第二，我們基於Alexnet並參考Resnet34 V1的殘差學習來設計出Residual34 V1Alexnet。第三，我們基於Alexnet並參考InceptionResnet V1中的InceptionResidualModule A來設計出InceptionResidualV1MAAlexnet。大腸癌患者的組織學切片是從Kaggle數據網站下載，數據具有8種不同的組織類別，包括腫瘤上皮、簡單基質（均質成分、腫瘤基質、腫瘤外基質和平滑肌）、複雜基質（單個腫瘤細胞或少量免疫細胞）、免疫細胞（免疫細胞和粘膜下淋巴濾泡）、雜物（壞死出血和粘液）、正常的黏膜腺、脂肪組織和背景（無組織）。實驗結果顯示本論文提出的InceptionResidualV1MAAlexnet應用在大腸癌組織分類的準確率為97.86%，高於Alexnet模型的95.99%和Alexnet預訓練模型的97.59%。

This thesis explores the techniques used by Alexnet, VGGnet, Inception V1, Resnet, Resnet V2, Inception V2, Inception V3, Inception V4, Inception Resnet V1, and Inception Resnet V2, and three convolutional neural networks suitable for the classification of medical imaging of colorectal cancer tissues were designed by studying various convolutional neural network techniques. First, we designed InceptionV1Alexnet based on Alexnet and referred to Inception Module B in Inception V1. Second, we designed Residual34V1Alexnet based on Alexnet and referred to residual learning of Resnet34 V1. Third, we designed InceptionResidualV1MAAlexnet based on Alexnet and referred to InceptionResidualModule A in InceptionResnet V1. Histology tiles from patients with colorectal cancer were downloaded from the Kaggle data site, which has eight different classes of tissues, tumour epithelium, simple stroma (homogeneous composition, complex stroma (containing single tumour cells and/or few immune cells), immune cells (immune cell mass and submucosal lymphoid follicles), debris (including necrosis, hemorrhage and mucus), normal mucosal glands, adipose tissue and background (no tissue). The experimental results show that the accuracy rate of InceptionResidualV1MAAlexnet applied in colorectal cancer tissue classification is 97.86%, which is higher than 95.99% of Alexnet model and 97.59% of Alexnet pre-trained model.

摘要....................................................i
Abstract...............................................ii
誌謝...................................................iii
目錄...................................................iv
表目錄..................................................v
圖目錄..................................................vi
第一章 研究背景與動機....................................1
第二章 卷積神經網路概論..................................3
第三章 卷積神經網路研究與設計.............................8
3.1 Alexnet卷積神經網路技術研究....................8
3.2 VGGnet卷積神經網路技術研究.....................9
3.3 Inception V1卷積神經網路技術研究…..............10
3.4 Resnet卷積神經網路技術研究.....................11
3.5 Inception V2卷積神經網路技術研究...............14
3.6 Inception V3卷積神經網路技術研究...............15
3.7 Resnet V2卷積神經網路技術研究..................17
3.8 InceptionResnet V2卷積神經網路技術研究.........20
3.9 卷積神經網路設計...............................28
3.9.1 InceptionV1Alexnet..........................28
3.9.2 Residual34VlAlexnet.........................29
3.9.3 InceptionResidualV1MAAlexnet................29
第四章 卷積神經網路實作與實驗結果討論......................31
第五章 結論..............................................46
參考文獻.................................................47
Extended Abstract.......................................59

[1] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
[2] Ripley, B. D., & Hjort, N. L. (1996). Pattern recognition and neural networks. Cambridge university press.
[3] Hochreiter, S. (1991). Untersuchungen zu dynamischen neuronalen Netzen. Diploma, Technische Universität München, 91(1).
[4] Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[5] Kather, J. N., Weis, C. A., Bianconi, F., Melchers, S. M., Schad, L. R., Gaiser, T., ... & Zöllner, F. G. (2016). Multi-class texture analysis in colorectal cancer histology. Scientific reports, 6, 27988.
[6] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
[7] Zeiler, M. D., & Fergus, R. (2014, September). Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833). Springer, Cham.
[8] Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015). Training very deep networks. In Advances in neural information processing systems (pp. 2377-2385).
[9] J. Pathol. Inform. (2017), p. 8 Tompson, J., Jain, A, LeCun, Y. & Bregler, C. Joint training of convolutional network and a graphical model for human pose estimation. In Proc. Advances in Neural Information Processing SyStems 27 1799–1807 (2014)
[10] Zeiler, M. D., & Fergus, R. (2014, September). Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833). Springer, Cham.
[11] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).
[12] Venables, W. N., & Ripley, B. D. (2013). Modern applied statistics with S-PLUS. Springer Science & Business Media.
[13] Ripley, B. D., & Hjort, N. L. (1996). Pattern recognition and neural networks. Cambridge university press.Christian Szegedy et al,Going Deeper with Convolutions ,In ICLR, 2015.
[14] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[15] He, K., Zhang, X., Ren, S., & Sun, J. (2016, October). Identity mappings in deep residual networks. In European conference on computer vision (pp. 630-645). Springer, Cham.
[16] Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.
[17] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826).
[18] Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-First AAAI Conference on Artificial Intelligence.
[19] Liu, W., Wen, Y., Yu, Z., & Yang, M. (2016, June). Large-margin softmax loss for convolutional neural networks. In ICML (Vol. 2, No. 3, p. 7).
[20] He, K., & Sun, J. (2015). Convolutional neural networks at constrained time cost. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5353-5360).
[21] Krähenbühl, P., & Koltun, V. (2011). Efficient inference in fully connected crfs with gaussian edge potentials. In Advances in neural information processing systems (pp. 109-117).
[22] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
[23] Kather, J. N., Weis, C. A., Bianconi, F., Melchers, S. M., Schad, L. R., Gaiser, T., ... & Zöllner, F. G. (2016). Multi-class texture analysis in colorectal cancer histology. Scientific reports, 6, 27988.
[24] Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford university press.
[25] Lee, C. Y., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015, February). Deeply-supervised nets. In Artificial intelligence and statistics (pp. 562-570).
[26] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105)..
[27] Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015). Training very deep networks. In Advances in neural information processing systems (pp. 2377-2385).
[28] Bell, R. M., & Koren, Y. (2007). Lessons from the Netflix prize challenge. SiGKDD Explorations, 9(2), 75-79.