臺灣博碩士論文加值系統

現今工業中使用螺絲起子組裝零件或設備是各個生產產業的基礎流程，然而目前機械手臂與螺絲起子整合的自動化組裝技術中，需根據預先定義的手臂位置後再自動組裝零件；故本研究提出基於多參考影像的人工智慧辨識完成機械手臂自動組裝扣件研究，此採用強大的捲積神經網路(CNN)的影像處理演算法，旨在準確引導配備螺絲起子的機械手臂放置，研究上整合了 6 自由度串聯機械手臂、網路攝影機和智慧自動螺絲起子的系統，期待為現今工業組裝製造零件提供了一種適應性更強、更有效率的方法。此外，為了最大限度地減少機械手臂的誤差，機械手臂的座標系是使用 20 點校正功能建立的，再者，研究使用 YOLOv5m 影像處理模型評估檢測精度，並根據混淆矩陣評估 F1 分數達成多參考影像的人工智慧辨識，結果顯非CNN演算法顯著增強了系統在整個組裝過程中準確分類和識別螺絲、組裝孔及其各自位置的組裝能力。

The implementation of screwdrivers in assembling components stands as a foun-dational process in various production industries. Presently, automated assembly technology integrates robotic arms alongside screwdrivers to autonomously fasten parts according to predefined positions. This research employs a robust image pro-cessing with convolution based object detection algorithm aimed at accurately guiding the placement of a robotic arm equipped with a screwdriver. The study introduces a system that integrates a six degree of freedom (DOF) serial robot arm, with a webcam, and an intelligent automatic screwdriver, providing a more adaptable and efficient ap-proach in assembling manufactured components. To minimize error, the position co-ordinate system of the robot arm is established using the “Mastering Tool Calibration” function with 20-points reference method provided in robot's controller. Furthermore, this paper assesses and discusses the detection accuracy of the custom trained YOLOv5 object detection models by evaluating the F1-scores and the confusion ma-trices of the trained models in this study. According to the trained models in this study, the highest F1-Score obtained is 100% at 0.672 confidence rate. This model has a True-Positive (TP) score of 100% and 93% for the True-Negative (TN) score. Additionally, this study also discusses and compares the assembling performance of the screwdriver robot arm system developed by relying on both “Preset Positions” and “AI Detection Positions” for the screw hole position. From trials, the AI object detection algorithm applied to the screw driving robot arm system shows a significant enhancement to the system's capabilities in accurately classifying and recognizing screws, screw holes, and their respective positions throughout the assembly process, which provides more flexibility to the system for fastening screws compared to rely-ing on the preset positions. Despite having a slightly higher occurrence of failure, by approximately 8%, in the process with the developed system in this study with, there are solutions and improvements that can be made to the system in order to increase the accuracy of the screw driving robot arm with AI object detection system.

摘要...i
Abstract...ii
Acknowledgements or Preface...iv
Table of Contents...v
List of Tables...vii
List of Figures...viii
Symbols...xi
Chapter 1 Introduction...1
1.1. Study Background and Motivation...2
1.2. Thesis Objective...3
1.3. Outline of Thesis...3
1.4. Adopted Technologies of Thesis...4
Chapter 2 Literature Review...6
2.1. Developments of Automated Screw Driving...6
2.2. Screw and Holes Detection with AI Image Processing...11
Chapter 3 System Setup and Methodology...16
3.1. System Devices and Specifications...16
3.1.1. 6 Degree of Freedom Serial Robot Arm...17
3.1.2. Intelligent Screwdriver...18
3.1.3. Webcam Camera...19
3.2. System Setup and Fixtures Design...19
3.2.1. Designing and Manufacturing Supporting-End Effector Fixtures...20
3.2.2. Assemble and Setup Hardware for Automatic Screw driving...24
3.3. Calibrate and Program Automatic Screwdriver-Robot System...30
3.3.1. Robot Calibration using Mastering Tool Method...30
3.3.2. Plan Robot Path and Fasten Screw Programs...41
3.4. Vision system and Multi-Reference Images for AI Algorithm Models...48
3.4.1. Eye-in-hand Calibration Relative Position of Tool and Camera...49
3.4.2. Multi-reference Models of Cross Stage Partial Network...51
Chapter 4 Result and Discussion...55
4.1. Detect and Identify Holes with/without Screw for Fastening...55
4.1.1. Confusion Matrix Analysis on Trained Models...55
4.1.2. Precision, Recall and F1-Score Analysis on Trained Models...58
4.1.3. Detection Result Analysis on Trained Models...61
4.2. Fastening Screw Process Analysis...63
4.3. Evaluate Success Rate of System Assembly Performance...65
4.3.1. Robot Positioning Accuracy...66
4.3.2. Assembly Performance with Preset Positions...66
4.3.3. Assembly Performance with Multi-Reference Images AI...67
Chapter 5 Conclusion...69
References...71
Extended Abstract...73

[1]L. Tang and Y.-B. Jia, “Robotic fastening with a manual screwdriver,” 2023 IEEE Interna-tional Conference on Robotics and Automation (ICRA), 2023. doi:10.1109/icra48891.2023.10161139
[2]S. Pitipong, P. Pornjit, and P. Watcharin, “An automated four-dof robot screw fastening using Visual Servo,” 2010 IEEE/SICE International Symposium on System Integration, Oct. 2010. doi:10.1109/sii.2010.5708355
[3]K. Manisha, S. K. Ananya, D. Geervani, K. S. Sanihith, and G. Swetha, “Robotic arm for un-fastening screws in automated disassembly process,” 2023 9th International Conference on Control, Automation and Robotics (ICCAR), 2023. doi:10.1109/iccar57134.2023.10151756
[4]R. Li et al., “Unfastening of hexagonal headed screws by a collaborative robot,” IEEE Trans-actions on Automation Science and Engineering, vol. 17, pp. 1–14, Jan. 2020. doi:10.1109/tase.2019.2958712
[5]Rastegarpanah, A., Ner, R., Stolkin, R., & Marturi, N. (2021). Nut unfastening by robotic sur-face exploration. Robotics, 10(3), 107. https://doi.org/10.3390/robotics10030107
[6]G. Burresi et al., “Image-based defect detection in assembly line with Machine Learning,” 2021 10th Mediterranean Conference on Embedded Computing (MECO), 2021. doi:10.1109/meco52532.2021.9460291
[7]Baykal, I. C. (2018). Inspection of screw holes on machine parts using Robot Vision. 2018 International Conference on Artificial Intelligence and Data Processing (IDAP). https://doi.org/10.1109/idap.2018.8620904
[8]Z. Deng, D. Guan, Y. Qiao, and C. Zhuang, “Tiny screw and screw hole detection for Auto-mated Maintenance Processes,” 2022 IEEE International Conference on Mechatronics and Automation (ICMA), 2022. doi:10.1109/icma54519.2022.9855990
[9]Mei, B., Zhu, W., Yan, G., & Ke, Y. (2014). A new elliptic contour extraction method for ref-erence hole detection in robotic drilling. Pattern Analysis and Applications, 18(3), 695–712. https://doi.org/10.1007/s10044-014-0394-6
[10]W. Niu, H. Wang, and C. Zhuang, “Detection for tiny screw and screw hole by semantic seg-mentation model,” 2023 9th International Conference on Mechatronics and Robotics Engi-neering (ICMRE), 2023. doi:10.1109/icmre56789.2023.10106576
[11]Ryan. (2023, March 25). PRO-FUSE 電動精密螺絲鎖付機 - Utrun-Tech Co.,ltd 友創工業股份有限公司. Utrun. https://www.utrun.com/products/pro-fuse/
[12]Products. Products | Vanguard Systems Inc. (n.d.-a). https://www.hp-vanguard.com/en/products/
[13] Lai, Y.-L. (2023). A fast task planning system for 6R articulated robots based on inverse kin-ematics. The International Journal of Advanced Manufacturing Technology, 125(11–12), 4989–5013. https://doi.org/10.1007/s00170-023-11002-2
[14] Jia, Z., Bhatia, A., Aronson, R. M., Bourne, D., & Mason, M. T. (2019). A survey of automat-ed threaded fastening. IEEE Transactions on Automation Science and Engineering, 16(1), 298–310. https://doi.org/10.1109/tase.2018.2835382
[15]Enebuse, I., Foo, M., Ibrahim, B. S., Ahmed, H., Supmak, F., & Eyobu, O. S. (2021). A com-parative review of hand-eye calibration techniques for Vision Guided Robots. IEEE Access, 9, 113143–113155. https://doi.org/10.1109/access.2021.3104514
[16]J. Solawetz, “What is Yolov5? A guide for beginners.,” Roboflow Blog, https://blog.roboflow.com/yolov5-improvements-and-evaluation/ (accessed Oct. 3, 2023).
[17]C. Imane, “Yolo V5 model architecture [explained],” OpenGenus IQ: Computing Expertise & Legacy, https://iq.opengenus.org/yolov5/ (accessed Oct. 3, 2023).
[18]S.-H. Tsang, “Brief review: Yolov5 for object detection,” Medium, https://sh-tsang.medium.com/brief-reviewyolov5-for-object-detection-84cc6c6a0e3a#:~:text=YOLOv5%20uses%20the%20methods%20of,for%20detecting%20different%20scales%20targets. (accessed Oct. 3, 2023).
[19]G. Jocher and Sergiuwaxmann, “Architecture summary,” Architecture Summary – Ultralytics YOLOv8 Docs, https://docs.ultralytics.com/yolov5/tutorials/architecture_description/ (ac-cessed Oct. 3, 2023).
[20]J. Chen et al., “A novel disassembly strategy of hexagonal screws based on robot vision and robot-tool cooperated motion,” Applied Sciences, vol. 13, no. 1, p. 251, 2022. doi:10.3390/app13010251
[21]B. Lara, K. Althoefer, and L. D. Seneviratne, “Automated Robot-based screw insertion sys-tem,” IECON ’98. Proceedings of the 24th Annual Conference of the IEEE Industrial Elec-tronics Society (Cat. No.98CH36200). doi:10.1109/iecon.1998.724109
[22]N. M. DiFilippo and M. K. Jouaneh, “A system combining force and vision sensing for auto-mated screw removal on laptops,” IEEE Transactions on Automation Science and Engineer-ing, vol. 15, no. 2, pp. 887–895, Apr. 2018. doi:10.1109/tase.2017.2679720
[23]R. Chumakov, “An artificial neural network for fault detection in the assembly of thread-forming screws,” Journal of Intelligent Manufacturing, vol. 19, no. 3, pp. 327–333, 2008. doi:10.1007/s10845-008-0085-5
[24]W.-Y. Chang, M. L. Hartono, and Nadia, “AI recognition based on multi-target images for assembling parts using robot with screwdriver and Vision System,” 2023 IEEE 5th Eurasia Conference on IOT, Communication and Engineering (ECICE), 2023. doi:10.1109/ecice59523.2023.10382994