臺灣博碩士論文加值系統

本論文提出基於卷積神經網路之智慧型手勢識別技術(Design of intelligent gesture recognition based on convolutional neural network, IGRCNN)提升FIGARIRS(Face identification and gesture answering recognition for interactive response system)系統辨識率，IGRCNN技術主要是解決FIGARIRS所面臨的問題。因FIGARIRS遇到手部影像太小企圖放大後來辨識處理，但產生影像模糊問題。另外，手臉影像重疊或背景複雜，也導致FIGARIRS降低辨識率。因此本論文提出IGRCNN技術，利用影像超解析度(Super Resolution)、影像實例分割(Image Instance Segmentation)及顏色去背技術進行影像的預處理，強化手部影像的清晰度提升FIGARIRS系統的辨識率。實驗證明先以影像超解析度技術將影像進行預處理的效能最佳。

This paper proposes the design of intelligent gesture recognition based on convolutional neural networks (IGRCNN) to improve the gesture recognition rate of FIGARIRS (Face identification and gesture answering recognition for interactive response system) system. The IGRCNN technique focuses on solving the problems the FIGARIRS suffers. First, a small hand images require to enlarge the images when recognizing them. However, enlarged version of images are getting blur. In addition, hand and face are overlapping in an image. Moreover, complex backgrounds exist in an image. These three cases reduce the gesture recognition rate of the FIGARIRS. Therefore, the IGRCNN technique take super resolution based on autoencoder, image instance segmentation, and image matting based on color to promote the image quality before the gesture recognition of the FIGARIRS. Experiments have proved that the gesture recognition adopting the image super-resolution technique is superior to other methods.

摘要 ...i
Abstract ...ii
誌謝 ...iii
目錄 ...iv
表目錄 ...vi
圖目錄 ...vii
第一章 研究動機目的 ...1
1.1 研究動機 ...1
1.2 研究目的 ...4
1.3 研究流程 ...6
第二章 相關文獻回顧 ...7
2.1 顏色空間相關文獻 ...7
2.2 影像超解析度相關文獻 ...9
2.3 基於影像實例分割相關文獻 ...12
第三章 背景回顧 ...14
3.1 AutoEncoder ...14
3.2 Peak Signal-to-Noise Ratio(PSNR) ...14
3.3 Fast R-CNN ...16
3.4 Mask R-CNN ...17
3.5 ROI Polling ...20
3.6 ROI Align ...21
第四章 手勢辨識技術 ...23
4.1 影像預處理 ...23
4.1.1 模糊、清晰對應影像製作 ...23
4.1.2 影像Label ...25
4.1.3 基於影像實例分割去除背景訓練前準備 ...26
4.2 基於顏色去除背景 ...28
4.3 基於影像實例分割去除背景 ...30
4.3.1 Mask R-CNN架構流程 ...30
4.3.2 Mask R-CNN訓練 ...32
4.4 SuperResolution影像超解析度 ...34
4.4.1 超解析度架構流程 ...34
4.4.2 Autoencoder訓練 ...36
4.4.3 Autoencoder架構 ...37
4.4.4 超解析度效能評估 ...39
第五章 實驗結果 ...40
5.1 系統需求 ...40
5.2 影像資料庫 ...42
5.3 基於影像實例分割去除背景模型實驗結果 ...43
5.4 影像超解析度模型實驗結果 ...45
5.5 辨識率結果 ...47
5.6 結果與討論 ...50
第六章 結論與未來研究 ...51
6.1. 結論 ...51
6.2. 未來研究 ...51
參考文獻 ...52
Extended Abstract ...56
Abstract ...56
Introduction ...57
Methods ...57
Results & Discussions ...60
References ...61

[1]Tapingkae, P., Panjaburee, P., Hwang, G., & Srisawasdi, N. (2020). Effects of a formative assessment-based contextual gaming approach on students’ digital citizenship behaviours, learning motivations, and perceptions. Computers & Education, 159, 103998. doi: 10.1016/j.compedu.2020.103998
[2]XIN-YU HOU（2020）。Based on Convolutional Neural Networks Design of Intilligent Object Recognition Technique to Develop Interactive Response Portfolio System〔
Unpublished〕。National Formosa University。
[3]Imura, T., Fukumi, M., Akamatsu, N., & Nakaura, K. (2004). Face Search by Neural Network Based Skin Color Threshold Method. Lecture Notes In Computer Science, 840-846. doi: 10.1007/978-3-540-30132-5_113
[4]Hernández-Hernández, J., Hernández-Hernández, M., Feliciano-Morales, S., Álvarez-Hilario, V., & Herrera-Miranda, I. (2017). Search for Optimum Color Space for the Recognition of Oranges in Agricultural Fields. Communications In Computer And Information Science, 296-307. doi: 10.1007/978-3-319-67283-0_22
[5]Supe, K., Jaiswal, K., Khan, A., Katkar, V., & Nirmal, P. (2017). Image Classification Using Discrete Block Truncation Coding. Advances In Intelligent Systems And Computing, 393-402. doi: 10.1007/978-981-10-2750-5_41
[6]Ganeswara Rao, M., Panakala, R., & Mallikarjuna Prasad, A. (2018). A New VLSI Architecture for Skin Tone Detection in an Uncontrolled Background. Lecture Notes In Electrical Engineering, 847-854. doi: 10.1007/978-981-10-7329-8_87
[7]Tang, J., Yang, G., Sun, Y., Xin, J., & He, D. (2019). Salient object detection of dairy goats in farm image based on background and foreground priors. Neurocomputing, 332, 270-282. doi: 10.1016/j.neucom.2018.12.052
[8]Yu, X., Fernando, B., Ghanem, B., Porikli, F., & Hartley, R. (2018). Face Super-Resolution Guided by Facial Component Heatmaps. Computer Vision – ECCV 2018, 219-235. doi: 10.1007/978-3-030-01240-3_14
[9]Dittimi, T., & Suen, C. (2020). Single Image Super-Resolution for Medical Image Applications. Pattern Recognition And Artificial Intelligence, 660-666. doi: 10.1007/978-3-030-59830-3_57
[10]Hyun, S., & Heo, J. (2020). VarSR: Variational Super-Resolution Network for Very Low Resolution Images. Computer Vision – ECCV 2020, 431-447. doi: 10.1007/978-3-030-58592-1_26
[11]Sharma, M., Chaudhury, S., & Lall, B. (2017). Space-Time Super-Resolution Using Deep Learning Based Framework. Lecture Notes In Computer Science, 582-590. doi: 10.1007/978-3-319-69900-4_74
[12]Basty, N., & Grau, V. (2018). Super Resolution of Cardiac Cine MRI Sequences Using Deep Learning. Image Analysis For Moving Organ, Breast, And Thoracic Images, 23-31. doi: 10.1007/978-3-030-00946-5_3
[13]Niu, Z., Liu, L., Zhang, K., Dong, J., Yang, Y., & Mao, X. (2018). Single Image Super-Resolution via Perceptual Loss Guided by Denoising Auto-Encoder. Lecture Notes In Computer Science, 126-136. doi: 10.1007/978-3-319-97304-3_10
[14]Pan, Z., & Shen, H. (2018). Multispectral Image Super-Resolution Using Structure-Guided RGB Image Fusion. Pattern Recognition And Computer Vision, 155-167. doi: 10.1007/978-3-030-03398-9_14
[15]Quan, L., & Kim, Y. (2019). Iterative Application of Autoencoders for Video Inpainting and Fingerprint Denoising. Inpainting And Denoising Challenges, 63-76. doi: 10.1007/978-3-030-25614-2_5
[16]Patel, N., Shinde, S., & Poly, F. (2020). Automated Damage Detection in Operational Vehicles Using Mask R-CNN. Algorithms For Intelligent Systems, 563-571. doi: 10.1007/978-981-15-3242-9_54
[17]Wang, X., Xie, T., & Chen, L. (2019). Urban Village Identification from City-Wide Satellite Images Leveraging Mask R-CNN. Advances In Intelligent Systems And Computing, 166-172. doi: 10.1007/978-3-030-29933-0_14
[18]Patil, A. (2020). Car Damage Recognition Using the Expectation Maximization Algorithm and Mask R-CNN. Information And Communication Technology For Intelligent Systems, 607-616. doi: 10.1007/978-981-15-7062-9_61
[19]Li, Z., Miao, D., Liang, H., Zhang, H., Liu, J., & He, Z. (2019). Efficient and Accurate Iris Detection and Segmentation Based on Multi-scale Optimized Mask R-CNN. Lecture Notes In Computer Science, 715-726. doi: 10.1007/978-3-030-34110-7_60
[20]de F. Souza, L., Holanda, G., Alves, S., dos S. Silva, F., & Filho, P. (2020). Automatic Lung Segmentation in CT Images Using Mask R-CNN for Mapping the Feature Extraction in Supervised Methods of Machine Learning. Advances In Intelligent Systems And Computing, 140-149. doi: 10.1007/978-3-030-49342-4_14
[21]Cheng, T., Wang, X., Huang, L., & Liu, W. (2020). Boundary-Preserving Mask R-CNN. Computer Vision – ECCV 2020, 660-676. doi: 10.1007/978-3-030-58568-6_39
[22]Liu, Z., Yang, C., Huang, J., Liu, S., Zhuo, Y., & Lu, X. (2021). Deep learning framework based on integration of S-Mask R-CNN and Inception-v3 for ultrasound image-aided diagnosis of prostate cancer. Future Generation Computer Systems, 114, 358-367. doi: 10.1016/j.future.2020.08.015
[23]AutoEncoder (一)-認識與理解：
https://medium.com/%E7%A8%8B%E5%BC%8F%E5%B7%A5%E4%BD%9C%E7%B4%A1/autoencoder-%E4%B8%80-%E8%AA%8D%E8%AD%98%E8%88%87%E7%90%86%E8%A7%A3-725854ab25e8
[24]【影像處理】峰值信噪比 Peak Single-to-Noise Ratio：
https://jason-chen-1992.weebly.com/home/-peak-single-to-noise-ratio
[25]峰值訊噪比：
https://zh.wikipedia.org/wiki/%E5%B3%B0%E5%80%BC%E4%BF%A1%E5%99%AA%E6%AF%94
[26]RCNN,Fast RCNN,Faster RCNN比較歸納總結：
https://blog.csdn.net/xiaoye5606/article/details/71191429
[27]Mask RCNN學習筆記：
https://www.twblogs.net/a/5b7a8e7a2b7177392c964b74
[28]例項分割模型Mask R-CNN詳解：從R-CNN，Fast R-CNN，Faster R-CNN再到Mask R-CNN：
https://www.itread01.com/content/1547791045.html
[29]詳解 ROI Align 的基本原理和實現細節：
https://codertw.com/%E7%A8%8B%E5%BC%8F%E8%AA%9E%E8%A8%80/590370/