臺灣博碩士論文加值系統

機械手臂能夠高速執行精確且重複性的任務，因此近年來被廣泛應用於製造業生產線中，大幅提高了生產效率，並提高了產品品質的穩定性以及製造流程的一致性，但在以往機械手臂應用時，須對工作目標工件進行空間座標定位，通常是操作教導器校正座標系定位，將得取到的數值輸入到機械手臂的程式中或座標系定位區塊，但使用此方法的定位時間較為冗長，且容易因人為疏失導致撞機。因此普遍做法是使用影像基準點進行定位，然而使用傳統的影像基準點定位方法存在著一些侷限和挑戰，如硬體等級限制、環境光照變化或物件表面紋路不明顯等因素，導致定位精度下降和穩定性不足，同時需要操作人員依據不同的使用環境與作業目的重新設定，距離無人化目標有些許差距。同時，神經網路和硬體運算能力的快速發展為視覺檢測和空間定位提供了更多的解決方案，使物件辨識和分類的能力大幅提高的同時也提升了產線的效能和價值，進一步推動了智慧製造的無人化發展。因此本研究以C++程式語言作為開發環境，整合OpenCV、PCL與結構光3D建模搭配PointNet++深度學習神經網路辨識物件，用以輔助機械手臂進行定位和夾取，並搭建了一個小型簡易的作業平台，以模擬近年來產業自動化與無人化趨勢下的情境。該系統使用微型投影機投影結構光紋路至產品上，再由相機擷取當前影像，並由解碼結構光影像及近似射線交點三角量測法建立3D點雲模型，同時結合PointNet++神經網路進行物件類別辨識，並分析物件模型的姿態將3D點雲模型的位置轉換為六軸機械手臂的基座座標系，實現產品的夾取和放置，使本研究達成機械手臂快速定位與自動化之目的。

Robotic arms can execute tasks quickly, precisely, and repeatedly, making them widely used in manufacturing production lines in recent years. Manufacturing production by robotic arms has significantly increased production efficiency, enhanced product quality stability, and ensured consistency in the manufacturing process. In the past, the spatial coordinate positioning of the target workpiece was required to calibrate when applying robotic arms. The spatial coordinate positioning typically involved calibrating the coordinate system using a teach pendant, inputting the obtained values into the robotic arm's program, or coordinate positioning block. However, this method of positioning calibration is time-consuming and prone to human errors, leading to collisions. The common practice was to use image reference points for positioning. However, traditional image-based positioning methods face limitations and challenges, such as hardware constraints, variations in environmental lighting, or unclear surface patterns on objects, resulting in decreased positioning accuracy and stability. Additionally, most operators need to reset parameters based on different operating environments and purposes, creating a gap in achieving full automation goals. The rapid development of neural networks and hardware computational capabilities has provided more solutions for visual detection and spatial positioning, significantly improving object recognition and classification abilities, thereby enhancing the efficiency and value of production lines. This further propels the development of unmanned smart manufacturing. In this study, the programming language serves as the development environment, integrating OpenCV, PCL, and structured light 3D modeling with the PointNet++ deep learning neural network for object recognition. This integration assists the robotic arm in positioning and gripping. A small and simple operating platform was built to simulate scenarios under the recent trends of industrial automation and unmanned operation. The system projects structured light patterns onto the product using a miniature projector, captures the current image with an industrial camera, decodes the structured light image, and uses a triangulation measurement algorithm to create a 3D point cloud model. Additionally, the PointNet++ neural network is employed for object category recognition, and the analysis of the object model's pose transforms the position of the 3D point cloud model into the base coordinate system of the six-axis robotic arm. This achieves the goal of rapid positioning and automation for the robotic arm in this study.

摘要 .................................................................................................................... i
Abstract .............................................................................................................. ii
誌謝 .................................................................................................................. iv
目錄 ................................................................................................................... v
表目錄 ............................................................................................................ viii
圖目錄 .............................................................................................................. ix
符號說明 ........................................................................................................ xiii
第一章 緒論...................................................................................................... 1
1.1 研究背景與動機 .................................................................................. 1
1.2 研究目的 .............................................................................................. 1
1.3 論文架構 .............................................................................................. 2
第二章 文獻探討 .............................................................................................. 3
2.1 視覺輔助機械手臂國內外相關研究 ................................................... 3
2.2 3D點雲辨識國內外相關研究 .............................................................. 6
第三章 研究架構與方法 ................................................................................ 11
3.1 研究架構與流程 ................................................................................ 11
3.2 硬體及系統架構 ................................................................................ 11
3.2.1 結構光掃描3D建模硬體設備 ................................................ 12
3.2.2 機械手臂硬體設備 ................................................................... 13
3.3 結構光3D點雲模型建置 .................................................................. 14
3.3.1 相機成像原理........................................................................... 14
3.3.2 成像變形失真矯正 ................................................................... 16
(1) 徑向失真 ............................................................................... 17
(2) 切向失真 ............................................................................... 17
vi
(3) 失真矯正公式 ....................................................................... 18
3.3.3 結構光建模原理 ....................................................................... 19
(1) 結構光編碼 ........................................................................... 19
(2) 結構光解碼 ........................................................................... 24
(3) 近似射線交點三角量測法 .................................................... 27
3.3.4 3D點雲建置與後處理 .............................................................. 29
(1) 最小移動平方演算法 ........................................................... 29
(2) 直通濾波 ............................................................................... 30
(3) 統計異常消除濾波 ............................................................... 30
(4) 網格體素濾波 ....................................................................... 31
3.4 神經網路訓練資料庫建置與點雲辨識分類 ..................................... 32
3.4.1 神經網路物件外型CAD模型點雲資料庫建置 ...................... 32
3.4.2 神經網路點雲辨識分類與特徵分割系統 ................................ 33
3.5 機械手臂空間定位與分類夾取整合系統開發 .................................. 35
3.5.1 機械手臂姿態校正 ................................................................... 36
(1) 尤拉角與平移旋轉矩陣座標轉換 ........................................ 36
(2) 手眼校正 ............................................................................... 38
第四章 實驗結果與討論 ................................................................................ 42
4.1 結構光建模系統 ................................................................................ 42
4.1.1 相機與投影機校正 ................................................................... 42
4.1.2 機械手臂與相機姿態關係校正 ............................................... 47
4.1.3 結構光3D點雲建模尺寸精度 ................................................ 50
4.2 神經網路輔助夾取系統 ..................................................................... 53
4.2.1 物件辨識姿態誤差實驗 ........................................................... 53
4.2.2 物件辨識準確度實驗 ............................................................... 55
第五章 結論與未來展望 ................................................................................ 57
vii
參考文獻 ......................................................................................................... 58
Extended Abstract ............................................................................................ 60

[1]Ya-Ling Chen, Yan-Rou Cai and Ming-Yang Cheng,2023, "Vision-Based Robotic Object Grasping—A Deep Reinforcement." Machines 11.2 (2023): 275.
[2]吳宜儒，2019，”基於深度學習與立體影像之機械手臂智慧夾取”，國立清華大學，動力機械工程學系電機控制組，碩士學位論文。
[3]J. Piao, H. Jo and J. -B. Song, "Robotic Tidy-up Tasks using Point Cloud-based Pose Estimation," 2020 20th International Conference on Control, Automation and Systems (ICCAS), Busan, Korea (South), 2020, pp. 316-319, doi: 10.23919/ICCAS50221.2020.9268313.
[4]Lars Berscheid, Pascal Meißner, and Torsten Kröger. "Robot learning of shifting objects for grasping in cluttered environments." 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2019.
[5]Ji S-Q, Huang M-B, Huang H-P. Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors. 2019; 19(7):1595. https://doi.org/10.3390/s19071595
[6]Soyoung Yoo, Namwoo Kang, Explainable artificial intelligence for manufacturing cost estimation and machining feature visualization, Expert Systems with Applications, Volume 183,2021,115430,ISSN 0957-4174.
[7]Dering, M. L., and Tucker, C. S. (October 2, 2017). "A Convolutional Neural Network Model for Predicting a Product's Function, Given Its Form." ASME. J. Mech. Des. November 2017; 139(11): 111408. https://doi.org/10.1115/1.4037309
[8]Y. Ben-Shabat, M. Lindenbaum and A. Fischer, "3DmFV: Three-Dimensional Point Cloud Classification in Real-Time Using Convolutional Neural Networks," in IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3145-3152, Oct. 2018, doi: 10.1109/LRA.2018.2850061.
[9]Chenke Yue, Yong Wang, Xintong Tang, Qiuyi Chen, DRGCNN: Dynamic region graph convolutional neural network for point clouds, Expert Systems with Applications, Volume 205,2022,117663,ISSN 0957-4174,https://doi.org/10.1016/j.eswa.2022.117663.
[10]Jinho Park, Martin B.G. Jun, Huitaek Yun,Development of robotic bin picking platform with cluttered objects using human guidance and convolutional neural network (CNN),Journal of Manufacturing Systems, Volume 63,2022,Pages 539-549,ISSN 0278-6125,https://doi.org/10.1016/j.jmsy.2022.05.011.
[11]Zongyue Wang, Qiming Xia, Jing Du, Shangfeng Huang, Jinhe Su, José Marcato Junior, Jonathan Li, Guorong Cai,3D MSSD: A multilayer spatial structure 3D object detection network for mobile LiDAR point clouds, International Journal of Applied Earth Observation and Geoinformation, Volume 102,2021,102406,ISSN 1569-8432,https://doi.org/10.1016/j.jag.2021.102406.
[12]R. Y. Tsai and R. K. Lenz, "A new technique for fully autonomous and efficient 3D robotics hand/eye calibration," in IEEE Transactions on Robotics and Automation, vol. 5, no. 3, pp. 345-358, June 1989, doi: 10.1109/70.34770.
[13]Qi Charles R., Yi L, Su H, and Guibas LJ, "Pointnet++: Deep hierarchical feature learning on point sets in a metric space." Advances in neural information processing systems 30 (2017).
[14]Qi Charles R., H. Su, M. Kaichun and L. J. Guibas, "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 77-85, doi: 10.1109/CVPR.2017.16.
[15]Open Source Computer Vision Library(OpenCV), " https://opencv.org/"
[16]Point Cloud Library(PCL), "https://pointclouds.org/"
[17]許嘉洧，2018，”基於格雷碼結構光輔助工具機之工件自動定位”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。
[18]黃立仁，2020，”基於點對特徵匹配應用機械手臂於堆疊物件之夾取系統”，國立虎尾科技大學，機械與電腦輔助工程系，碩士學位論文。