臺灣博碩士論文加值系統

隨著智慧製造和工業自動化的快速發展，機械手臂在各行各業的應用越來越廣泛。機械手臂能夠高速執行精確且重複性的任務，因此被廣泛應用於製造業生產線中，大幅提高了生產效率，並提高了產品品質的穩定性和製造流程的一致性，然而，在實際應用中，機械手臂的精確操作需要依賴於手眼校正技術。手眼校正是機械手臂應用中的關鍵步驟，主要是確定相機與機械手臂之間的空間關係，以便精確控制手臂的運動。傳統的手眼校正方法通常需要手動操作，過程繁瑣且費時，並且容易因人為操作錯誤而導致校正結果不準確，這種方法對於現代工業中需要高效的自動化流程來說顯得不夠理想，因此，本研究開發了一種利用神經網路整合機械手臂之自動化手眼校正系統，整合了OpenCV、C#及神經網路，使用Eye-In-Hand的手眼校正方法，即相機安裝於手臂上，以確定相機與機械手臂之間的空間關係。此系統利用神經網路建立訓練模型，並採用辨識結果高達98%的Separated_Enhanced模型，來辨識棋盤格和各角位的相對位置，系統會藉由輸出最近角位的結果，讓手臂移動至該角位上方拍攝10張不同角度的照片，最後輸出拍攝結果完成手眼校正，整個過程只需按下一個按鈕即可完成，操作簡便且高效，這不僅減少了人為操作錯誤的可能性，還大幅提高了校正的速度和準確性，系統的自動化特性顯著減少了操作人員的工作負擔，使得手眼校正過程更加簡單和高效。最後，本研究探討了自動手眼校正的穩定性，分析需要拍攝多少張校正板的照片，才能讓手眼校正的結果達到穩定，並且在不同的位置校正後，觀察相機內部參數和畸變參數的標準偏差。此外，還對機械手臂和相機的姿態校正進行測試，其結果平均誤差X方向為6.67mm，Y方向為3.37mm，最終可以透過自動手眼校正的方法得知整體誤差值為5.02mm。

With the rapid development of smart manufacturing and industrial automation, robotic arms are increasingly being applied across various industries. Robotic arms can perform precise and repetitive tasks at high speeds, thus they are widely used in manufacturing production lines, significantly improving production efficiency, and enhancing the stability of product quality and consistency of manufacturing processes. However, in practical applications, the precise operation of robotic arms relies on hand-eye calibration technology. Hand-eye calibration is a crucial step in the application of robotic arms, mainly to determine the spatial relationship between the camera and the robotic arm, allowing for precise control of the arm's movements. Traditional hand-eye calibration methods usually require manual operation, which is cumbersome, time-consuming, and prone to human error, leading to inaccurate calibration results. This approach is not ideal for modern industry, which demands highly efficient automated processes. Therefore, this study developed an automated hand-eye calibration system for robotic arms using neural network integration, incorporating OpenCV, C#, and neural networks. The system employs the Eye-In-Hand calibration method, where the camera is mounted on the robotic arm, to determine the spatial relationship between the camera and the robotic arm. This system uses a neural network to build a training model and adopts the Separated_Enhanced model with a recognition accuracy of up to 98% to identify the checkerboard and the relative positions of the corners. The system outputs the nearest corner's result, allowing the arm to move above this corner and take 10 photos from different angles. The calibration process is completed by outputting the results of the photos, and the entire process requires only the press of a single button, making the operation simple and efficient. This not only reduces the possibility of human error but also significantly improves the speed and accuracy of calibration. The system's automation greatly reduces the workload of operators, making the hand-eye calibration process simpler and more efficient. Finally, this study explored the stability of automated hand-eye calibration, analyzing the number of checkerboard photos needed to achieve stable calibration results. The study also observed the standard deviation of internal camera parameters and distortion parameters after calibration at different positions. In addition, the pose calibration of the robotic arm and the camera was tested, with results showing an average error of 6.67mm in the X direction and 3.37mm in the Y direction. Ultimately, through the automated hand-eye calibration method, the overall error value was found to be 5.02mm.

摘要...............................................i
Abstract..........................................iv
目錄...............................................v
表目錄...........................................viii
圖目錄............................................ix
符號說明.........................................xii
第一章 緒論........................................1
1.1 研究背景與動機..................................1
1.2 研究目的.......................................2
1.3 論文架構.......................................2
第二章 文獻探討.....................................3
2.1.1 國內外手眼校正方法相關研究.....................3
第三章 研究架構與方法...............................11
3.1 研究架構與流程..................................11
3.1.1 系統硬體架設..................................11
3.1.2 機械手臂硬體與相機介紹.........................12
3.2 卷積神經網路....................................13
3.2.1 輸入層(Input Layer)...........................14
3.2.2 卷積層(Convolutional Layer)...................15
3.2.3 池化層(Pooling Layer).........................17
3.2.4 全連接層(Fully Connected Layer)...............18
3.3 機械手臂與相機系統的校正與畸變補償技術.............19
3.3.1 相機成像原理...................................19
3.3.2 成像畸變矯正...................................21
(1) 徑向畸變.........................................21
(2) 切向畸變.........................................22
(3) 畸變矯正公式......................................22
3.3.3 機械手臂姿態校正................................23
(1) 尤拉角與平移旋轉矩陣座標轉換.......................24
(2) 手眼校正.........................................27
第四章 實驗結果與討論.................................29
4.1 神經網路辨識棋盤格及四個角位結果...................29
4.1.1 建立棋盤格及四個角位之訓練模型...................29
4.1.2 訓練模型各曲線及混淆矩陣結果分析.................30
(1) 精確率(Precision)................................31
(2) 召回率(Recall)...................................32
(3) 平均精度均值(Mean Average Precision, mAP).........32
(4) F1值.............................................33
(5) 混淆矩陣(Confusion matrix).......................34
4.1.3 訓練模型的辨識結果分析..........................34
4.2 自動化手眼校正系統................................36
4.2.1 自動手眼校正系統之人機介面.......................36
4.2.2 自動手眼校正系統之運作流程.......................36
(1) 拍攝照片.........................................38
(2) 神經網路檢測......................................38
(3) 計算最近角位......................................38
(4) 機械手臂移動......................................39
4.3 自動手眼校正結果...................................41
4.3.1 相機內部參數與成像畸變補償係數校正.................42
(1) 不同照片數量對相機內參和畸變參數穩定性的影響..........42
(2) 固定拍攝數量探討內參和畸變參數的標準偏差..............42
4.3.2 機械手臂與相機姿態校正............................45
第五章 結論與未來展望...................................48
參考文獻...............................................49

[1]L. Dongmin, W. Yu, M. Wenping, L. Xiujie, D. Guowei, Z. Guohui, and F. Jiaqi, “Automatic Hand-Eye Calibration Method of Welding Robot Based on Linear Structured Light,” J. Robot. Mechatron., Vol.36 No.2, pp. 438-448, 2024.
[2]Hua, J.; Su, Y.; Xin, D.; Guo, W. A High-Precision Hand–Eye Coordination Localization Method under Convex Relaxation Optimization. Sensors 2024, 24, 3830. https://doi.org/10.3390/s24123830
[3]Jing Xu, Jian Li Hoo, Stylianos Dritsas, Javier Gomez Fernandez,Hand-eye calibration for 2D laser profile scanners using straight edges of common objects,Robotics and Computer-Integrated Manufacturing,Volume 73,2022,102221,ISSN 0736-5845,https://doi.org/10.1016/j.rcim.2021.102221.
[4]Yipeng Tang, Wenxu Luo, Qing Wang, Jinyang Li, Liang Cheng, Jiangxiong Li, Yinglin Ke,A novel hand-eye semi-automatic calibration process for laser profilometers using machine learning,Measurement,Volume 216,2023,112936,ISSN 0263-2241,https://doi.org/10.1016/j.measurement.2023.112936.
[5]Zhengping Deng, Yisheng Ruan, Fei Hao, Tianyao Liu,Hand-eye calibration of line structured-light sensor by scanning and reconstruction of a free-placed standard cylindrical target,Measurement,Volume 229,2024,114487,ISSN 0263-2241,https://doi.org/10.1016/j.measurement.2024.114487.
[6]Sara Sharifzadeh, Istvan Biro, Peter Kinnell,Robust hand-eye calibration of 2D laser sensors using a single-plane calibration artefact,Robotics and Computer-Integrated Manufacturing,Volume 61,2020,101823,ISSN 0736-5845,https://doi.org/10.1016/j.rcim.2019.101823.
[7]Bu, Z.; Sun, C.; Wang, P.; Dong, H. Calibration of Camera and Flash LiDAR System with a Triangular Pyramid Target. Appl. Sci. 2021, 11, 582. https://doi.org/10.3390/app11020582
[8]K. Koide and E. Menegatti, "General Hand–Eye Calibration Based on Reprojection Error Minimization," in IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1021-1028, April 2019, doi: 10.1109/LRA.2019.2893612.
[9]杜國忠，2021，“沖壓工件之動態立體視覺建模與手臂取放”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[10]施硯智，2024，“以神經網路辨識車牌圖像檢測系統”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[11]鄭淯騰，2024，“以結構光建模與神經網路預測輔助機械手臂夾取系統”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[12]LabelImg. Available: https://github.com/tzutalin/labelImg
[13]YoloV7. Available: https://github.com/WongKinYiu/yolov7
[14]楊福氣，2024，“基於多參考影像的人工智慧辨識以完成機械手臂自動組裝扣件研究”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[15]Z. Zhang, “A flexible new technique for camera calibration” ,IEEE Transactions on Pattern Analysis and Machine Intelligence,22(11):1330–1334, 2000.
[16]R. Y. Tsai and R. K. Lenz, "A new technique for fully autonomous and efficient 3D robotics hand/eye calibration," in IEEE Transactions on Robotics and Automation, vol. 5, no. 3, pp. 345-358, June 1989, doi: 10.1109/70.34770.
[17]Open Source Computer Vision Library(OpenCV), " https://opencv.org/"
[18]許貴聞，2021，“基於虛實整合之機械手臂遠端路徑規劃系統開發”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[19]黃立仁，2020，“基於點對特徵匹配應用機械手臂於堆疊物件之夾取系統”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[20]簡駿宥，2019，“基於物件姿態估測導引機械手臂智慧取放與加工系統”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。