臺灣博碩士論文加值系統

近年來的人臉辨識系統已經有著優秀的辨識率，能夠應用在門禁系統、打卡系統或是手機上的驗證解鎖，但在比對證件上的頭貼與生活照相似度的例子屈指可數，且證件資料取得不易，因為證件照數據集涉及隱私的關係多為私有，加上證件上的照片可能隨著證件的樣式不同，導致不同的模糊程度或是人臉特徵過於平滑，雖然證件照與生活照之比對同一般人臉辨識，一樣要經過人臉偵測、人臉校正、做適當的前處理後再套入模型中訓練與測試，但若使用直接使用一般的人臉辨識方法有很大的機會無法辨識，原因在於類神經網路所訓練的人臉圖像皆為生活照的格式，證件照的人臉格式並不適用在這些模型上。本文著重在未使用證件照訓練模型的情況下，如何使用前處理方法提升證件照與生活照的辨識率，以及使用先進的FaceNet進行模型訓練，並比較了在少量的資料下對網路模型進行轉移學習，優化了證件照與生活照之間的辨識率。

In recent years, face recognition systems have achieved excellent performance when applied to access control systems, time and attendance systems, and for user verification for unlocking mobile phones. However, few studies have evaluated the comparative effectiveness of using either ID（Identity） photos or daily-life photos for such purposes. Moreover, it is difficult to obtain credible official photo data because privacy regulations governing their use constrain access to such official photo records. Additionally, the photos on credentials, such as travel documents and bank access cards, may vary according to the standards applied by the credential issuer, resulting in different levels of blurring or facial features that have been overly smoothed. The process for comparing daily-life photos to ID photos is the same as that for general face recognition. Each must undergo face detection, image correction, and preprocessing steps. Deep learning is then applied to train the model for testing. However, if a standard face recognition technique is used on an ID photo, it is often not identified because the facial images used to train the neural network are taken from a collection of daily-life photos. The identification of ID photos often fails in such models. This article focuses on the use of pre-processing techniques to improve the recognition rate of ID photos and daily-life photos without the training model using ID photos. This approach uses the advanced FaceNet for model training. We observed that Transfer Learning with a small amount of data optimized the network model and the comparative recognition of ID photos and daily-life photos improved.

Abstract i
Contents iii
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Related Works 8
2.1 General Biometrics Technologies . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.1 Fingerprint Recognition . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 Iris Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.3 Face Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Face Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Matching Document ID Photo and Daily-Life Face Image . . . . . . . . . . . 10
2.4 Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . 12
iii
2.5 Transfer Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.6 FaceNet and ResNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Methodology 16
3.1 Proposed Method and System Architecture . . . . . . . . . . . . . . . . . . . 16
3.2 Face Detection and Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3 Introduction of Face Image Preprocessing . . . . . . . . . . . . . . . . . . . 19
3.3.1 Sharp Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.2 Retinex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.3 ACE (Automatic Color Equalization) Algorithm . . . . . . . . . . . . 23
3.3.4 CLAHE Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.4 Deep Learning and Face Similarity Measure . . . . . . . . . . . . . . . . . . 26
4 Experimental Results 31
4.1 Experiment Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2 Face Verification Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5 Conclusions and Future Work 43
iv
List of Figures
2.1 The 68 feature points of the human face . . . . . . . . . . . . . . . . . . . . 10
2.2 The architechure of basic CNN . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Compute of convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Compute of pooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1 Flowchart of the proposed document photo and daily-life photo comparison
system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 The 68 feature points of the human face . . . . . . . . . . . . . . . . . . . . 18
3.3 Image (a) is a skewed face image ；Image (b) is the result of face correction
and cropping through dlib C ++ . . . . . . . . . . . . . . . . . . . . . . . . 18
3.4 Image sharpening results, where the left side of each pair is the original image,
and the right side is the sharpening result: (a) self-simulated ID photo (b)
student ID (c) ID card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.5 Results of MSRCR, where the left side of each pair is the original image, and
the right side is the result of MSRCR : (a) self-simulated ID photo (b) student
ID (c) ID card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
v
3.6 Results of MSRCP, where the left side of each pair is the original image, and
the right side is the result of MSRCP: (a) self-simulated ID photo (b) student
ID (c) ID card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.7 Results of ACE, where the left side of each pair is the original image, and the
right side is the result of ACE: (a) self-simulated ID photo (b) student ID (c)
ID card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.8 Limiting the slope of the CDF is equivalent to limiting the amplitude of the
Histogram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.9 The four values are bilinearly interpolated of CLAHE . . . . . . . . . . . . . 25
3.10 Results of CLAHE, where the left side of each pair is the original image, and
the right side is the result of CLAHE: (a) self-simulated ID photo (b) student
ID (c) ID card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.11 FaceNet structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.12 Inception module structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.13 Tripletloss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1 Positive Data example of the IDs photo. The first row from left to right is
Simulation_ID, Student_ID, and ID. The second and third rows are daily-life
photos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Negative Data examples of the daily-life photo. . . . . . . . . . . . . . . . . 34
vi
List of Tables
4.1 Number of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2 Experimental environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.3 Comparsions of iterations and face alignment method . . . . . . . . . . . . . 37
4.4 Comparsions of pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.5 Comparsions of Our method . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.6 Comparsions of ID_Pair and Student_ID_Pair . . . . . . . . . . . . . . . . 42
4.7 Comparison of wearing glasses or not. . . . . . . . . . . . . . . . . . . . . . . 42

[1] Lars Chittka and Adrian Dyer, “Cognition: Your face looks familiar,” Nature, vol. 481, no. 7380, pp. 154, 2012.
[2] Chi Ho Chan, Muhammad Atif Tahir, Josef Kittler, and Matti Pietikäinen, “Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 5, pp. 1164–1177, 2012.
[3] Jie Chen, Shiguang Shan, Chu He, Guoying Zhao, Matti Pietikainen, Xilin Chen, and Wen Gao, “Wld: A robust local image descriptor,” IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1705–1720, 2009.
[4] Austin Blanton, Kristen C Allen, Timothy Miller, Nathan D Kalka, and Anil K Jain, “A comparison of human and automated face verification accuracy on unconstrained image sets,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016, pp. 161–168.
[5] Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs, “Frontal to profile face verification in the wild,” in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2016, pp. 1–9.
[6] Dursun Delen, Asil Oztekin, and Leman Tomak, “An analytic approach to better
understanding and management of coronary surgeries,” Decision Support Systems, vol. 52, no. 3, pp. 698–705, 2012.
[7] Matthew Turk and Alex Pentland, “Eigenfaces for recognition,” Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991.
[8] Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701–1708.
[9] Yi Sun, Ding Liang, Xiaogang Wang, and Xiaoou Tang, “Deepid3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, 2015.
[10] Keping Ye and Fangyu Hu, “Research on cross-age face verification based on artificial neural network under examination environment,” in 2017 10th International Symposium on Computational Intelligence and Design (ISCID). IEEE, 2017, vol. 1, pp. 430–433.
[11] Guilherme Folego, Marcus A Angeloni, José Augusto Stuchi, Alan Godoy, and Anderson Rocha, “Cross-domain face verification: Matching id document and self-portrait photographs,” arXiv preprint arXiv:1611.05755, 2016.
[12] John G Daugman, “High confidence visual recognition of persons by a test of statistical independence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1148–1161, 1993.
[13] Yoav Freund, Robert E Schapire, et al., “Experiments with a new boosting algorithm,” in International Conference on Machine Learning. Citeseer, 1996, vol. 96, pp. 148–156.
[14] Paul Viola, Michael Jones, et al., “Rapid object detection using a boosted cascade of simple features,” Conference on Computer Vision and Pattern Recognition, vol. 1, no. 511-518, pp. 3, 2001.
[15] Paul Viola and Michael J Jones, “Robust real-time face detection,” International
Journal of Computer Vision, vol. 57, no. 2, pp. 137–154, 2004.
[16] Rainer Lienhart and Jochen Maydt, “An extended set of haar-like features for rapid object detection,” in Proceedings. International Conference on Image Processing. IEEE, 2002, vol. 1, pp. I–I.
[17] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, et al., “Deep face recognition.,” in British Machine Vision Conference, 2015, vol. 1, p. 6.
[18] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao, “A discriminative feature learning approach for deep face recognition,” in European Conference on Computer Vision. Springer, 2016, pp. 499–515.
[19] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song,
“Sphereface: Deep hypersphere embedding for face recognition,” in Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 212–220.
[20] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu, “Cosface: Large margin cosine loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5265–5274.
[21] Jiankang Deng, Yuxiang Zhou, and Stefanos Zafeiriou, “Marginal loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 60–68.
[22] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 4690–4699.
[23] Xiaoou Tang and Xiaogang Wang, “Face photo recognition using sketch,” in Proceedings. International Conference on Image Processing. IEEE, 2002, vol. 1, pp. I–I.
[24] Brendan F Klare and Anil K Jain, “Heterogeneous face recognition using kernel prototype similarities,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1410–1422, 2012.
[25] Shengcai Liao, Dong Yi, Zhen Lei, Rui Qin, and Stan Z Li, “Heterogeneous face recognition from local structures of normalized appearance,” in International Conference on Biometrics. Springer, 2009, pp. 209–218.
[26] Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma, “A nonlinear approach for face sketch synthesis and recognition,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, vol. 1, pp. 1005–1010.
[27] Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang, “Deep learning face representation by joint identification-verification,” in Advances in Neural Information Processing Systems, 2014, pp. 1988–1996.
[28] Xiangyu Zhu, Hao Liu, Zhen Lei, Hailin Shi, Fan Yang, Dong Yi, Guojun Qi, and Stan Z Li, “Large-scale bisample learning on id versus spot face recognition,” International Journal of Computer Vision, vol. 127, no. 6-7, pp. 684–700, 2019.
[29] Yichun Shi and Anil K Jain, “Docface: Matching id document photos to selfies,”
in 2018 IEEE 9th International Conference on Biometrics Theory, Applications and
Systems (BTAS). IEEE, 2018, pp. 1–8.
[30] Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541–551, 1989.
[31] Mingsheng Long, Jianmin Wang, Guiguang Ding, Dou Shen, and Qiang Yang, “Transfer learning with graph co-regularization,” IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 7, pp. 1805–1818, 2013.
[32] Tatiana Tommasi, Francesco Orabona, and Barbara Caputo, “Learning categories from few examples with multi model knowledge transfer,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 5, pp. 928–941, 2013.
[33] Himanshu S Bhatt, Richa Singh, Mayank Vatsa, and Nalini K Ratha, “Improving crossresolution face matching using ensemble-based co-transfer learning,” IEEE Transactions on Image Processing, vol. 23, no. 12, pp. 5654–5669, 2014.
[34] Florian Schroff, Dmitry Kalenichenko, and James Philbin, “Facenet: A unified embedding for face recognition and clustering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815–823.
[35] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.
[36] Sergey Ioffe and Christian Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
[37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
[38] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016.
[39] Edwin H Land and John J McCann, “Lightness and retinex theory,” Josa, vol. 61, no. 1, pp. 1–11, 1971.
[40] Daniel J Jobson, Zia-ur Rahman, and Glenn A Woodell, “A multiscale retinex for
bridging the gap between color images and the human observation of scenes,” IEEE
Transactions on Image Processing, vol. 6, no. 7, pp. 965–976, 1997.
[41] Alessandro Rizzi, Carlo Gatta, and Daniele Marini, “From retinex to automatic color equalization: issues in developing a new algorithm for unsupervised color equalization,” Journal of Electronic Imaging, vol. 13, no. 1, pp. 75–85, 2004.
[42] Karel Zuiderveld, “Contrast limited adaptive histogram equalization,” in Graphics gems IV. Academic Press Professional, Inc., 1994, pp. 474–485.
[43] Klemen Grm, Simon Dobrisek, and Vitomir Struc, “Deep pair-wise similarity learning for face recognition,” in 2016 4th International Conference on Biometrics and Forensics (IWBF). IEEE, 2016, pp. 1–6.