臺灣博碩士論文加值系統

現在各個企業皆面臨人力資源不足與人力成本提高的問題，使許多企業加速進入自動化的行列，本研究以自動化研磨站透過 AI 建立研磨粗糙度分級檢測與人員不良姿態檢測為例，塑膠產品生產時所產生的分模線去除步驟，以往透過人力去除後再由人員進行檢測，有機率造成去除成效不好或是視覺疲勞後檢測標準不一的情況發生，進而導致產品的品質不穩定，透過建立粗糙度分級檢測使用 AI 進行分級判斷，讓產品可以在最短的時間內完成分級檢測，此外考量本研究的自動化研磨站仍須由人員進行上下料在搬運物品時可能會不自覺或是在不知道的情況下以不良的姿態來搬運，可能提高職業病發生的機率，針對上述的兩個目標因此本研究提出以六軸機械手臂進行研磨將塑膠表面的分模線先透過粗磨去除後，再進行細磨使塑膠表面的 Ra 值(算術平均粗度)降低，完成後將成品放入工業相機內進行取像並建立 Yolov7、Yolact++、Faster R-CNN 三種不同的深度學習模型架構進行比較，選出檢測速度快與準確度高的模型進行研磨後粗糙度分級檢測，並將操作與檢測過程透過網頁來呈現，使整個過程實現自動化;在姿態檢測上以Nvidia Jetson Agx Xavier 邊緣運算裝置結合 Openpose 來檢測人員的 2D 骨架關鍵點，並使用長短期記憶(Long short-term memory, LSTM)訓練姿態模型，最後將兩者結合透過鏡頭來檢測人員的姿態，如果檢測到不良的姿態將會透過 Modbus TCP 向 PLC 發送指令，讓三色燈發出聲音與亮燈提醒人員注意目前的姿態，藉此來降低職業病的發生。

Currently, numerous enterprises grapple with the challenge of inadequate human resources and escalating labor costs. This predicament has prompted many businesses to hasten their transition towards automation. This study serves as an illustration, showcasing the implementation of AI to establish a system for detecting roughness classifications, as well as identifying unfavorable personnel postures within an automated grinding station. Traditionally, the process of removing parting lines generated during plastic product manufacturing has relied on manual intervention, followed by subsequent manual inspections to detect defects. However, this approach carries the risk of suboptimal removal outcomes and inconsistent inspection standards due to visual fatigue. Consequently, this may lead to fluctuations in product quality stability. To address these objectives, this study introduces the utilization of a six-axis robotic arm for grinding purposes. Initially, it addresses the removal of parting lines from plastic surfaces through a coarse grinding process. Subsequently, a refined grinding procedure is applied to further reduce the Ra value (arithmetic mean roughness) of the plastic surface. Upon completion, the finished products are positioned within an industrial camera setup to capture images. A comprehensive analysis is then conducted, comparing three distinct deep learning model architectures: Yolov7, Yolact++, and Faster R-CNN. The model demonstrating both swift detection speed and heightened precision is selected for subsequent roughness classification inspections following the grinding process. Furthermore, the operational protocols and inspection procedures are presented through an intuitive web interface, thus achieving a comprehensive level of process automation. Regarding posture detection, the Nvidia Jetson Agx Xavier edge computing device collaborates with Openpose to determine the 2D skeletal keypoints of personnel. Subsequently, a Long Short-Term Memory (LSTM) architecture is employed to train the posture model. Ultimately, the integration of these components culminates in the application of a camera to detect personnel postures. In instances where unfavorable postures are detected, instructions are communicated to the Programmable Logic Controller (PLC) using the Modbus TCP communication protocol. To effectively mitigate the occurrence of occupational health issues, a stack light system is employed. This system not only emits audible alerts but also illuminates accordingly, serving as a visual and auditory prompt for personnel to remain vigilant about their current posture.

摘要..........................................i
Abstract......................................ii
誌謝..........................................iv
目錄..........................................v
表目錄........................................vii
圖目錄........................................viii
第一章 緒論..............................................1
1.1 研究背景..........................................1
1.2 文獻探討..........................................2
1.3 論文架構..........................................3
第二章 系統架構..........................................4
2.1 前期研究與開發.....................................4
2.2 自動化研磨與AI研磨粗糙度分級檢測....................6
2.3 人體姿態辨識系統...................................8
第三章 自動化研磨站..................................................10
3.1 自動化研磨站主要架構..........................................10
3.2 自動化研磨站加工流程..........................................11
3.3 自動化研磨站通訊整合..........................................12
3.4 硬體設備..............................................13
3.4.1 六軸機械手臂..........................................13
3.4.2 氣動研磨平台..........................................14
3.4.3 視覺檢測平台..........................................14
3.4.4 表面粗糙度測量儀......................................15
3.4.5 PLC系統控制...........................................16
第四章 AI研磨粗糙度分級檢測...................................17
4.1 工業相機焦距設定.......................................17
4.2 工業相機光圈設定.......................................20
4.3 影像上傳伺服器架設.....................................21
4.4 深度學習模型架構選用...................................21
4.4.1 Yolov7...............................................22
4.4.2 Yolact...............................................24
4.4.3 Detectron2-Faster R-CNN..............................26
4.5 模型準確度測試........................................27
第五章 人體姿態辨識模組......................................29
5.1 人體骨架模型選用......................................29
5.2 人體骨架模型測試......................................30
5.3 長短期記憶模型LSTM....................................33
5.4 建立自定義人體姿態模型.................................33
5.5 姿態偵測過程..........................................35
5.5.1 影像獲取..............................................35
5.5.2 設定Region of interest................................36
5.5.3 Openpose人體關鍵點獲取.................................36
5.5.4 通訊傳遞...............................................37
5.6 硬體介紹...............................................37
5.6.1 NVIDIA Jetson AGX Xavier..............................37
5.6.2 USB相機................................................38
5.6.3 遠端I/O................................................38
第六章 網頁系統開發..........................................39
6.1 使用者登入介面..........................................39
6.2 資料庫..........................................40
6.3 控制介面..........................................41
6.4 操控研磨工作站..........................................41
6.5 工業相機影像串流..........................................42
6.6 Yolov7影像串流..........................................42
6.7 外網影像串流..........................................43
6.8 人員姿態影像串流..........................................43
第七章 實驗結果與分析..........................................44
7.1 自動化AI研磨粗糙度分級檢測..........................................44
7.2 跨平台網頁操控介面..........................................45
7.3 檢測數據收集..........................................46
7.4 自定義訓練人體姿態模型..........................................47
7.5 外部通訊整合..........................................47
7.6 影像串流..........................................48
第八章 結論暨未來展望..........................................49
8.1 結論..........................................49
8.2 未來展望..........................................49
附錄..........................................53
Extended Abstract..........................................66

[1]第65期-台灣勞工季刊-數位科技下邁向人機協作的新趨勢，取自: https://www.mol.gov.tw/2578/21000/34504/
[2]M. d. M. Fernandes, J. A. Bigheti, R. P. Pontarolli and E. P. Godoy, "Industrial Automation as a Service: A New Application to Industry 4.0," in IEEE Latin America Transactions, vol. 19, no. 12, pp. 2046-2053, Dec. 2021, doi: 10.1109/TLA.2021.9480146.
[3]C. -S. Lin and H. -Y. Hsieh, "An Automatic Defect Detection System for Synthetic Shuttlecocks Using Transformer Model," in IEEE Access, vol. 10, pp. 37412-37421, 2022, doi: 10.1109/ACCESS.2022.3165224.
[4]C. Yu, Y. Liu, W. Zhang, X. Zhang, Y. Zhang and X. Jiang, "Foreign Objects Identification of Transmission Line Based on Improved YOLOv7," in IEEE Access, vol. 11, pp. 51997-52008, 2023, doi: 10.1109/ACCESS.2023.3277954.
[5]J. Joshi, P. Gandhi and R. Sawant, "Sign Language Certification Platform with Action Recognition using LSTM Neural Networks," 2022 International Conference on Computing, Communication, Security and Intelligent Systems (IC3SIS), Kochi, India, 2022, pp. 1-6, doi: 10.1109/IC3SIS54991.2022.9885321.
[6]陳冠銓，“自動化機械手臂研磨工作站開發與加工參數優化研究”，國立虎尾科技大學自動化工程系碩士班，2022年1月。
[7]S. Liu, Y. Wang, Q. Yu, H. Liu and Z. Peng, "CEAM-YOLOv7: Improved YOLOv7 Based on Channel Expansion and Attention Mechanism for Driver Distraction Behavior Detection," in IEEE Access, vol. 10, pp. 129116-129124, 2022, doi: 10.1109/ACCESS.2022.3228331.
[8]N. Zarei, P. Moallem and M. Shams, "Fast-Yolo-Rec: Incorporating Yolo-Base Detection and Recurrent-Base Prediction Networks for Fast Vehicle Detection in Consecutive Images," in IEEE Access, vol. 10, pp. 120592-120605, 2022, doi: 10.1109/ACCESS.2022.3221942.
[9]Y. Dai, W. Liu, H. Wang, W. Xie and K. Long, "YOLO-Former: Marrying YOLO and Transformer for Foreign Object Detection," in IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-14, 2022, Art no. 5026114, doi: 10.1109/TIM.2022.3219468.
[10]W. Lina and J. Ding, "Behavior detection method of OpenPose combined with Yolo network," 2020 International Conference on Communications, Information System and Computer Engineering (CISCE), Kuala Lumpur, Malaysia, 2020, pp. 326-330, doi: 10.1109/CISCE50729.2020.00072.
[11]R. S. Charran and R. K. Dubey, "Two-Wheeler Vehicle Traffic Violations Detection and Automated Ticketing for Indian Road Scenario," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 22002-22007, Nov. 2022, doi: 10.1109/TITS.2022.3186679.
[12]C. Wang, Z. Zhou and Z. Chen, "An Enhanced YOLOv4 Model With Self-Dependent Attentive Fusion and Component Randomized Mosaic Augmentation for Metal Surface Defect Detection," in IEEE Access, vol. 10, pp. 97758-97766, 2022, doi: 10.1109/ACCESS.2022.3203198.
[13]C. -C. Wong, M. -Y. Chien, R. -J. Chen, H. Aoyama and K. -Y. Wong, "Moving Object Prediction and Grasping System of Robot Manipulator," in IEEE Access, vol. 10, pp. 20159-20172, 2022, doi: 10.1109/ACCESS.2022.3151717.
[14]J. Zhong et al., "A Looseness Detection Method for Railway Catenary Fasteners Based on Reinforcement Learning Refined Localization," in IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-13, 2021, Art no. 3518913, doi: 10.1109/TIM.2021.3086913.
[15]K. Abe, K. -I. Tabei, K. Matsuura, K. Kobayashi and T. Ohkubo, "Relationship Between the Results of Arm Swing Data From the OpenPose-Based Gait Analysis System and MDS-UPDRS Scores," in IEEE Access, vol. 10, pp. 118992-119000, 2022, doi: 10.1109/ACCESS.2022.3220767.
[16]P. Nguyen Huu, N. Nguyen Thi and T. P. Ngoc, "Proposing Posture Recognition System Combining MobilenetV2 and LSTM for Medical Surveillance," in IEEE Access, vol. 10, pp. 1839-1849, 2022, doi: 10.1109/ACCESS.2021.3138778.
[17]M. M. E. Yurtsever and S. Eken, "BabyPose: Real-Time Decoding of Baby’s Non-Verbal Communication Using 2D Video-Based Pose Estimation," in IEEE Sensors Journal, vol. 22, no. 14, pp. 13776-13784, 15 July15, 2022, doi: 10.1109/JSEN.2022.3183502.
[18]M. A. Khatun et al., "Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor," in IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, pp. 1-16, 2022, Art no. 2700316, doi: 10.1109/JTEHM.2022.3177710.
[19]Y. -P. Huang, S. Kshetrimayum and C. -T. Chiang, "Object-Based Hybrid Deep Learning Technique for Recognition of Sequential Actions," in IEEE Access, vol. 11, pp. 67385-67399, 2023, doi: 10.1109/ACCESS.2023.3291395.
[20]C.-Y. Wang and A. Bochkovskiy and H.-Y. Mark Liao, "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," in arXiv, 2022, cs.CV 2207.02696.
[21]D. Bolya and C. Zhou and F. Xiao and Y. J. Lee, "YOLACT: Real-time Instance Segmentation," in arXiv, 2019, cs.CV 1904.02689.
[22]D. Bolya, C. Zhou, F. Xiao and Y. J. Lee, "YOLACT++ Better Real-Time Instance Segmentation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 2, pp. 1108-1121, 1 Feb. 2022, doi: 10.1109/TPAMI.2020.3014297.
[23]S. Ren and K. He and R. Girshick and J. Sun, " Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks," in arXiv, 2016, cs.CV 1506.01497.