臺灣博碩士論文加值系統

近年來製造業紛紛轉型智能化，常見的異常偵測為機構馬達振動監控、電流監控、溫度監控，以及工廠端的各種影像辨識。一般來說，常會運用人工智慧（Artificial Intelligence，AI）技術，包含機器學習與深度學習，用於作數據分析、物件偵測、影像辨識及自然語言處理等。工業4.0強調的「虛實整合」就是透過物聯網及雲端技術，收集設備機構的資訊及分析大數據，結合人工智慧，讓生產流程更靈活更有效率。智慧工廠聲紋監控較為少見，有鑑於設備馬達在接近損壞前，都會發出異音，本論文提出應用聲紋辨識來監控設備異音。
聲紋辨識大部分應用於人類語音聲紋辨識及音樂聲紋辨識，方式多數以頻譜圖（spectrogram）作影像分析，人工智慧的模型訓練時間較長。本論文音頻正樣本為錄製主機風扇正常運轉的聲音，共5000筆，用加噪技術作音頻合併負樣本為主機硬碟異常警示音、主機風扇被排線干擾異音、主機記憶體異常警示音，共5000筆。將總樣本數10000筆音頻正負樣本轉換為六種取樣頻率的音頻，以梅爾頻率倒譜係數（Mel-Frequency Cepstral Coefficient，MFCC）及色度特徵（Chroma Feature，Chroma）擷取聲紋特徵數據。再將音頻特徵數據作平均及重複提取，藉以提升F1-Score。訓練樣本8000筆，測試樣本2000筆，以支援向量機（Support Vector Machine，SVM）、隨機森林樹（Random Forest，RF）、雙向長短期記憶（Bidirectional Long Short-Term Memory，Bi-LSTM）與卷積神經網路（Convolutional Neural Network，CNN）來建立聲紋分類器模型。
實驗結果顯示當音頻的取樣頻率越高，測試模型分類的F1-Score越高，聲紋特徵數據重複提取與頻段平均，能提高F1-Score；MFCC在音頻取樣頻率為44100Hz，CNN的F1-Score為100%；Chroma在音頻取樣頻率為44100Hz，RF與Bi-LSTM的F1-Score為100%。
本論文提出的樹莓派結合陣列式麥克風錄製音頻，不須更動設備機構，就能監控設備運轉時馬達及減速機異音。此聲紋辨識系統能輔助有經驗的維修人員，即時監控辨識設備機構運行時有無異常聲音，達到預警維修的功能。

In recent years, the manufacturing industry has transformed into intelligence. Common abnormal detections include mechanism motor vibration monitoring, current monitoring, temperature monitoring, and various image recognition on the factory. In general, artificial intelligence (AI) technologies, including machine learning and deep learning, are often used for data analysis, object detection, image recognition, and natural language processing, etc. The "virtual and real integration" emphasized by Industry 4.0 is to collect information from equipment organizations and analyze big data through the Internet of Things and cloud technology, combined with artificial intelligence, to make the production process more efficient.
Voiceprint monitoring in intelligent factories is relatively rare. Because equipment motors will emit abnormal sounds before they are nearly damaged, this paper proposes to use voiceprint recognition to monitor abnormal sounds of equipment.
Voiceprint recognition is mainly used in human voice voiceprint recognition and music voiceprint recognition. Most methods use spectrograms for image analysis. Artificial intelligence models take a long time to train. The positive audio samples of this paper are the sound of the regular operation of the host fan, a total of 5000 audio, and the noise technology is used to make the audio. The negative samples are the abnormal warning sound of the host hard disk, the abnormal sound of the host fan being disturbed by the cable, and the abnormal warning sound of the host memory, A total of 5000 audio. Convert the total number of 10,000 audio positive and negative samples into audio with six sampling frequencies, and extract voiceprint feature data using Mel-Frequency Cepstral Coefficient (MFCC) and Chroma Feature (Chroma). Then the audio feature data is averaged and repeatedly extracted to improve F1-Score. There are eight thousand training samples, 2000 test samples. The Support Vector Machine (SVM), Random Forest (RF), Bidirectional Long Short-Term Memory (Bi-LSTM), and Convolutional Neural Network (CNN) methods are used to build the voiceprint classifier model.
The experimental results show that the higher the audio sampling frequency, the higher the F1-Score of the test model classification. The repeated extraction of voiceprint feature data and the frequency band average can improve the F1-Score. The audio sampling frequency of MFCC is 44100 Hz, and the F1-Score of CNN The Score is 100%. The audio sampling frequency of Chroma is 44100Hz, and the F1-Score of RF and Bi-LSTM is 100%.
The raspberry pi proposed in this paper combines the array microphone to record audio. Without changing the equipment mechanism, it can monitor the abnormal sound of the motor and reducer when the equipment is running. This voiceprint recognition system can assist experienced maintenance personnel in monitoring and identifying any abnormal sound during the operation of the equipment and mechanism and achieve the function of early warning and maintenance.

摘要 i
Abstract iii
誌謝 v
目錄 vi
表目錄 ix
圖目錄 xi
壹、緒論 1
1.1研究背景與動機 1
1.2研究目的 3
1.3論文架構 4
貳、文獻探討 5
2.1工業4.0與智慧製造（Industry 4.0 and Smart Manufacturing） 5
2.2 機構設備智慧監控（Intelligent Monitoring of Equipment） 7
2.2.1 物聯網（Internet of Things，IOT） 7
2.2.2 振動監控（Vibration Monitoring） 9
2.2.3 電流監控（Current Monitoring） 10
2.2.4 溫度監控（Temperature Monitoring） 10
2.2.5 影像監控 （Image Monitoring） 11
2.3 聲紋辨識（Voiceprint Recognition） 12
2.3.1 聲音技術（Voice Technology） 12
2.3.2 音樂辨識（Music recognition） 23
2.3.3 語音辨識（Speech Recognition） 23
2.4 機器學習與深度學習 24
2.4.1 支援向量機（Support Vector Machine，SVM） 25
2.4.2 隨機森林樹（Random Forest，RF） 26
2.4.3 雙向長短期記憶（Bidirectional Long Short-Term Memory，Bi-LSTM） 27
2.4.4 卷積神經網路（Convolutional Neural Network，CNN） 29
參、研究方法 31
3.1 研究架構 31
3.2 收集資料（Collect Datasets） 33
3.3 預處理（Pre-Processing） 37
3.3.1 音頻合併（Audio Merge） 38
3.3.2 取樣頻率（Sampling Frequency） 40
3.3.3 音頻轉換為特徵 41
3.3.4 特徵重複提取與頻段平均 45
3.4 訓練模型（Training Model） 49
3.4.1 支援向量機（Support Vector Machine，SVM） 50
3.4.2 隨機森林樹（Random Forest，RF） 51
3.4.3 雙向長短期記憶（Bidirectional Long Short-Term Memory，Bi-LSTM） 52
3.4.4 卷積神經網路（Convolutional Neural Network，CNN） 54
肆、實驗結果 57
4.1 實驗設備與環境 57
4.1.1 樹莓派規格 57
4.1.2 陣列式麥克風 59
4.1.3 開發環境 61
4.2 實驗結果 62
4.2.1 實驗模型評估 62
4.2.2 實驗模型比較 64
4.2.3 實驗取樣頻率分析 76
4.3 實驗討論 84
伍、結論與未來方向 86
5.1 研究貢獻 86
5.2 結論 86
5.3 未來方向 87
陸、參考文獻 89

[1] Rüßmann, M., Lorenz, M., Gerbert, P., Waldner, M., Justus, J., Engel, P., & Harnisch, M. (2015). Industry 4.0: The future of productivity and growth in manufacturing industries. Boston Consulting Group, 9(1), 54-89.
[2] Dalenogare, L. S., Benitez, G. B., Ayala, N. F., & Frank, A. G. (2018). The expected contribution of Industry 4.0 technologies for industrial performance. International Journal of Production Economics, 204, 383-394.
[3] Prihatno, A. T., Nurcahyanto, H., & Jang, Y. M. (2020, August). Smart Factory Based on IoT Platform. In KIC Summer Conf. 2020 (pp. 2-4).
[4] Huang, D. C., Lin, C. F., Chen, C. Y., & Sze, J. R. (2018, May). The Internet technology for defect detection system with deep learning method in smart factory. In 2018 4th International Conference on Information Management (ICIM) (pp. 98-102). IEEE.
[5] Rojko, A. (2017). Industry 4.0 concept: Background and overview. International Journal of Interactive Mobile Technologies, 11(5).
[6] Lasi, H., Fettke, P., Kemper, H.-G., Feld, T., & Hoffmann, M. (2014). Industry 4.0. Business & information systems engineering, 6(4), 239-242.
[7] Contreras, J. D., Garcia, J. I., & Pastrana, J. D. (2017). Developing of Industry 4.0 Applications. International Journal of Online Engineering, 13(10).
[8] Spoggi, G. (2020). Industry 4.0 and smart manufacturing: potential environmental benefits. The case of China. Università Ca'Foscari Venezia,
[9] Xu, L. D., Xu, E. L., & Li, L. (2018). Industry 4.0: state of the art and future trends. International Journal of Production Research, 56(8), 2941-2962.
[10] Frank, A. G., Dalenogare, L. S., & Ayala, N. F. (2019). Industry 4.0 technologies: Implementation patterns in manufacturing companies. International Journal of Production Economics, 210, 15-26.
[11] Shrouf, F., Ordieres, J., & Miragliotta, G. (2014, December). Smart factories in Industry 4.0: A review of the concept and of energy management approached in production based on the Internet of Things paradigm. In 2014 IEEE international conference on industrial engineering and engineering management (pp. 697-701). IEEE.
[12] Madakam, S., Lake, V., Lake, V., & Lake, V. (2015). Internet of Things (IoT): A literature review. Journal of Computer and Communications, 3(05), 164.
[13] Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future generation computer systems, 29(7), 1645-1660.
[14] Lee, I., & Lee, K. (2015). The Internet of Things (IoT): Applications, investments, and challenges for enterprises. Business Horizons, 58(4), 431-440.
[15] Jung, D., Zhang, Z., & Winslett, M. (2017, April). Vibration analysis for iot enabled predictive maintenance. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE) (pp. 1271-1282). IEEE.
[16] Ganga, D., & Ramachandran, V. (2018). IoT-based vibration analytics of electrical machines. IEEE Internet of Things Journal, 5(6), 4538-4549.
[17] Gao, S., Zhang, X., Du, C., & Ji, Q. (2019). A multichannel low-power wide-area network with high-accuracy synchronization ability for machine vibration monitoring. IEEE Internet of Things Journal, 6(3), 5040-5047.
[18] Khademi, A., Raji, F., & Sadeghi, M. (2019, April). IoT enabled vibration monitoring toward smart maintenance. In 2019 3rd International Conference on Internet of Things and Applications (IoT) (pp. 1-6). IEEE.
[19] Verma, A., Goyal, A., Kumara, S., & Kurfess, T. (2021). Edge-cloud computing performance benchmarking for IoT based machinery vibration monitoring. Manufacturing Letters, 27, 39-41.
[20] Schoen, R. R., Habetler, T. G., Kamran, F., & Bartfield, R. (1995). Motor bearing damage detection using stator current monitoring. IEEE transactions on industry applications, 31(6), 1274-1279.
[21] Blodt, M., Granjon, P., Raison, B., & Rostaing, G. (2008). Models for bearing damage detection in induction motors using stator current monitoring. IEEE transactions on industrial electronics, 55(4), 1813-1822.
[22] Saadaoui, W., & Jelassi, K. (2011, May). Induction motor bearing damage detection using stator current analysis. In 2011 International Conference on Power Engineering, Energy and Electrical Drives (pp. 1-6). IEEE.
[23] Gangsar, P., & Tiwari, R. (2017). Comparative investigation of vibration and current monitoring for prediction of mechanical and electrical faults in induction motor based on multiclass-support vector machine algorithms. Mechanical Systems and Signal Processing, 94, 464-481.
[24] Şen, M., & Kul, B. (2017, September). IoT-based wireless induction motor monitoring. In 2017 XXVI International Scientific Conference Electronics (ET) (pp. 1-5). IEEE.
[25] Faiazuddin, S., Lakshmaiah, M. V., Alam, K. T., & Ravikiran, M. (2020, November). IoT based Indoor Air Quality Monitoring system using Raspberry Pi4. In 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA) (pp. 714-719). IEEE.
[26] Islam, F. B., Nwakanma, C. I., Kim, D. S., & Lee, J. M. (2020, October). IoT-Based HVAC Monitoring System for Smart Factory. In 2020 International Conference on Information and Communication Technology Convergence (ICTC) (pp. 701-704). IEEE.
[27] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
[28] Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018). Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8697-8710).
[29] Rukmani, P., Teja, G. K., & Vinay, M. S. (2018). Industrial monitoring using image processing, IoT and analyzing the sensor values using big data. Procedia computer science, 133, 991-997.
[30] Wang, T., Yao, Y., Chen, Y., Zhang, M., Tao, F., & Snoussi, H. (2018). Auto-sorting system toward smart factory based on deep learning for image segmentation. IEEE Sensors Journal, 18(20), 8493-8501.
[31] Guanyu, G., Kai, K., Xiao, G. S., & Guanhua, G. (2012, June). Design and implementation of a high-performance client/server voiceprint recognition system. In 2012 IEEE International Conference on Information and Automation (pp. 704-708). IEEE.
[32] Li, P., Chen, M., Hu, F., & Xu, Y. (2015, May). A spectrogram-based voiceprint recognition using deep neural network. In The 27th Chinese Control and Decision Conference (2015 CCDC) (pp. 2923-2927). IEEE.
[33] Liu, J. H., Zou, Y. X., & Huang, Y. C. (2016, December). An effective voiceprint based identity authentication system for Mandarin smartphone users. In 2016 23rd International Conference on Pattern Recognition (ICPR) (pp. 1077-1082). IEEE.
[34] Xue, Y., Wang, L., Li, L., Liu, Z., & Liu, J. (2016, July). Matlab-Based Intelligent Voiceprint Recognition System. In 2016 Sixth International Conference on Instrumentation & Measurement, Computer, Communication and Control (IMCCC) (pp. 303-306). IEEE.
[35] Zhu, S., Xu, C., Wang, J., Xiao, Y., & Ma, F. (2017, May). Research and application of combined kernel SVM in dynamic voiceprint password authentication system. In 2017 IEEE 9th International Conference on Communication Software and Networks (ICCSN) (pp. 1052-1055). IEEE.
[36] Lei, L. I. (2021, April). Design of Acoustic System in Smart Classroom. In 2021 6th International Conference on Intelligent Computing and Signal Processing (ICSP) (pp. 916-920). IEEE.
[37] Nawas, K. K., Barik, M. K., & Khan, A. N. (2021). Speaker Recognition using Random Forest. In ITM Web of Conferences (Vol. 37, p. 01022). EDP Sciences.
[38] 謝維哲, & 蔡偉和. (2008). 基於梅爾頻譜質心倒頻譜係數之音樂聲紋辨識研究. 資訊科技國際期刊, 2(2), 18-31.
[39] 周智勳, & 林秉韶. (2010). 最佳化梅爾倒頻譜係數之研究及其於音樂曲風辨識之應用. Journal of Information Technology and Applications (資訊科技與應用期刊), 4(1), 53-58.
[40] Wang, A. (2003, October). An industrial strength audio search algorithm. In Ismir (Vol. 2003, pp. 7-13).
[41] Cheng, Y. (2020, October). Music Information Retrieval Technology: Fusion of Music, Artificial Intelligence and Blockchain. In 2020 3rd International Conference on Smart BlockChain (SmartBlock) (pp. 143-146). IEEE.
[42] Ghosal, A., Chakraborty, R., Chakraborty, R., Haty, S., Dhara, B. C., & Saha, S. K. (2009, November). Speech/music classification using occurrence pattern of zcr and ste. In 2009 Third International Symposium on Intelligent Information Technology Application (Vol. 3, pp. 435-438). IEEE.
[43] Lalitha, S., Mudupu, A., Nandyala, B. V., & Munagala, R. (2015, December). Speech emotion recognition using DWT. In 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC) (pp. 1-4). IEEE.
[44] Le, P. N., Ambikairajah, E., Epps, J., Sethu, V., & Choi, E. H. (2011). Investigation of spectral centroid features for cognitive load classification. Speech Communication, 53(4), 540-551.
[45] Kos, M., Kačič, Z., & Vlaj, D. (2013). Acoustic classification and segmentation using modified spectral roll-off and variance-based features. Digital Signal Processing, 23(2), 659-674.
[46] On, C. K., Pandiyan, P. M., Yaacob, S., & Saudi, A. (2006, June). Mel-frequency cepstral coefficient analysis in speech recognition. In 2006 International Conference on Computing & Informatics (pp. 1-5). IEEE.
[47] Martinez, J., Perez, H., Escamilla, E., & Suzuki, M. M. (2012, February). Speaker recognition using Mel frequency Cepstral Coefficients (MFCC) and Vector quantization (VQ) techniques. In CONIELECOMP 2012, 22nd International Conference on Electrical Communications and Computers (pp. 248-251). IEEE.
[48] Nagawade, M. S., & Ratnaparkhe, V. R. (2017, May). Musical instrument identification using MFCC. In 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT) (pp. 2198-2202). IEEE.
[49] Jiang, N., Grosche, P., Konz, V., & Müller, M. (2011, July). Analyzing chroma feature types for automated chord recognition. In Audio Engineering Society Conference: 42nd International Conference: Semantic Audio. Audio Engineering Society.
[50] Megalingam, R. K., Reddy, R. S., Jahnavi, Y., & Motheram, M. (2019, January). ROS based control of robot using voice recognition. In 2019 Third International Conference on Inventive Systems and Control (ICISC) (pp. 501-507). IEEE.
[51] Ponraj, A. S. (2020, November). Speech Recognition with Gender Identification and Speaker Diarization. In 2020 IEEE International Conference for Innovation in Technology (INOCON) (pp. 1-4). IEEE.
[52] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.
[53] Ganapathiraju, A., Hamaker, J. E., & Picone, J. (2004). Applications of support vector machines to speech recognition. IEEE transactions on signal processing, 52(8), 2348-2355.
[54] Ho, T. K. (1995, August). Random decision forests. In Proceedings of 3rd
international conference on document analysis and recognition (Vol. 1, pp. 278-282). IEEE.
[55] Phan, H., Maaß, M., Mazur, R., & Mertins, A. (2014). Random regression forests for acoustic event detection and classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(1), 20-31.
[56] Xia, X., Togneri, R., Sohel, F., & Huang, D. (2017, July). Random forest classification based acoustic event detection. In 2017 IEEE International Conference on Multimedia and Expo (ICME) (pp. 163-168). IEEE.
[57] Nawas, K. K., Barik, M. K., & Khan, A. N. (2021). Speaker Recognition using Random Forest. In ITM Web of Conferences (Vol. 37, p. 01022). EDP Sciences.
[58] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
[59] Graves, A., Jaitly, N., & Mohamed, A. R. (2013, December). Hybrid speech recognition with deep bidirectional LSTM. In 2013 IEEE workshop on automatic speech recognition and understanding (pp. 273-278). IEEE.
[60] LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4), 541-551.