臺灣博碩士論文加值系統

隨著工業技術的進步，機械手臂已成為大多數工業領域不可或缺的產線工具，以往機械手臂多用於重複處理固定的路徑工作，這些工作需事先由有經驗的操作人員透過示教盒進行繁瑣又費時的點對點教導動作，如此將限制機械手臂發展空間的可能性，因此利用機器視覺輔助機械手臂已成為發展的趨勢，透過機器視覺將提高機械手臂生產附加價值及效能，搭配視覺輔助將物體事先進行辨識，並由機械手臂讀取其資訊完成指定動作，藉此將提升機械手臂移動路徑的靈活性。有鑑於此，本研究開發一套3D物件姿態估測系統，由本系統辨識出物件的六個自由度座標變化，使機械手臂可自動生成移動路徑並進行任意夾取之功能，並將夾取物件放置氣動虎鉗由另一機械手臂進行離線編程銑削加工。研究主要分為四個部分，其一是透過點雲處理技術，由點雲前置雜訊處理、特徵點演算法及點雲匹配演算法等方式，將取得樣本點雲及目標點雲的旋轉及平移矩陣，由此方式得知任意物體的姿態變化量，並由實驗得知演算法對系統造成的影響趨勢以及本系統估測的精準度為0.57~1.95mm與0.5~1.37度；其二是機械手臂智慧夾取系統，透過相機與機械手臂的世界座標系統及工具座標系統進行整合，藉由撰寫控制器PLC與MACRO程式完成控制及路徑規劃部分，接著使估測系統所取得的數值透過TCP/IP通訊協定將機械手臂與電腦進行通訊連線並上傳至控制器，達到機械手臂智慧取料之功能；其三是離線編程系統，由編程軟體建立Fanuc機械手臂的系統參數檔，接著藉由工法編程規劃加工路徑並進行模擬碰撞測試，再由後處理建構器將刀具路徑檔轉換成機械手臂程式碼並上傳至機械手臂，將智慧夾取系統所抓取的工件放置氣動虎鉗位置由另一機械手臂進行銑削加工，並搭配本研究撰寫的Fanuc通訊監控系統及影像基準點校正系統進行系統管控與快速工件座標系定位；其四為雲端聯網系統，由蒐集層擷取機械手臂狀態資訊，透過MQTT雲端通訊進行訊息管理，最後再由雲端網頁將機械手臂狀態資訊呈現於看板頁面。本研究建置物件姿態估測系統、手臂智慧夾取系統、離線編程系統及雲端聯網系統，將上述系統進行整合模擬整體產線情境，提升機械手臂之生產價值與靈活性，目的達到自動化、無人化與智能化之指標。

With the improvement of industrial technology, robots have become an indispensable tool of production line in most industrial fields. In the past, robots were often used to repeatedly work in fixed paths. These tasks were required time-consuming point-to-point teaching actions through a teaching box by an experienced operator, which limit the developing space of robots. The use of machine vision-assisted robots has become a trend of development. With assistance of visual to identify objects in advance, machine vision will increase the additional value and effectiveness of robot production. The robot reads its information to complete the specified action to increase the flexibility of the robot's movement path. For the above reasons, this research develops a 3D object modeling and pose estimation system, which recognizes the change of object's six-degree-of-freedom coordinate, so that the robot can automatically generate the moving path and execute any picking function. The object was pick by another robot and placed to the pneumatic vise to do the offline programming. The research is mainly divided into four parts. The first is using point cloud processing technology to obtain the rotation and translation matrix of the sample point cloud and the target point cloud by point cloud pre-processing, feature point cloud identification, and point cloud matching. The experiment shows that the accuracy of the system estimation is 0.57~1.95mm and 0.5~1.37 degrees. The second is the robot intelligent pick-and-place system. The world coordinate system and the tool coordinate system of cameras and robots will be integrated by PLC and MACRO program of controller. And then, the value obtained by the estimation system is used to communicate and upload the robot and the computer by the TCP/IP protocol to reach robot’s intelligent pick-and-place function. The third is the offline programming system. The software establish system parameter file of Fanuc robots. The machining path is planned by the engineering method and the simulated collision test is performed, and then the post-processing constructor converts the tool path file into the robot code and uploads it to the robot for milling. Fanuc robot communication monitoring system and image G54 correction system will do the fast workpiece coordinate system positioning. The fourth is the cloud system of robot, which take the robot status information by collected layer and manage the message through MQTT cloud communication. The cloud page displays the robot status information on the webpage. This paper integrates the object pose estimation system, the robot Intelligent pick-and-place system, the offline programming system and the cloud networking system, and increases the production value and flexibility of the robot. The goal is to achieve automation, unmanned and intelligent indicators.

摘要....................................i
英文摘要(Abstract)....................................ii
誌謝....................................iv
目錄....................................v
表目錄....................................vii
圖目錄....................................viii
符號說明....................................xi
第 一 章 緒論....................................1
1.1 研究背景與動機....................................1
1.2 研究目的....................................2
1.3 論文架構....................................3
第 二 章 文獻回顧....................................4
2.1 特徵點描述與提取相關研究....................................4
2.2 3D視覺輔助機械手臂相關研究....................................6
第 三 章 研究架構與方法....................................12
3.1 研究架構與流程....................................12
3.2 硬體及系統架構....................................13
3.2.1 機械手臂智慧夾取系統硬體介紹....................................13
3.2.2 機械手臂離線編程硬體介紹....................................14
3.2.3 結構光掃描硬體設備....................................15
3.3 點雲數據擷取與前處理計算....................................16
3.3.1 點雲資料擷取方式....................................16
3.3.2 統計異常消除濾波器....................................17
3.3.3 網格體素濾波器....................................17
3.3.4 點雲平面分割....................................18
3.4 點雲數據特徵點描述演算....................................20
3.4.1 表面法向量估計....................................21
3.4.2 點特徵直方圖....................................22
3.4.3 快速點特徵直方圖....................................23
3.5 點雲數據姿態匹配演算....................................24
3.5.1 剛體自由度....................................24
3.5.2 特徵點匹配演算法....................................26
3.5.3 最近點迭代匹配演算法....................................26
3.6 機械手臂通訊與控制系統....................................27
3.6.2 機械手臂座標系統建置....................................28
3.6.3 新代控制器通訊系統建置....................................29
3.6.4 新代控制器PLC程式....................................30
3.6.5 MACRO巨集程式控制....................................32
3.7 機械手臂離線編程系統....................................34
3.7.1 離線編程系統架構....................................34
3.7.2 離線編程軟體系統建置....................................35
3.7.3 後處理建構器介紹....................................36
3.7.4 Fanuc機械手臂通訊與控制....................................38
3.8 雲端平台系統建置....................................40
3.8.1 雲端系統架構介紹....................................40
3.8.2 MQTT通訊協議介紹....................................41
3.8.3 雲端網頁介紹....................................42
第 四 章 實驗結果....................................43
4.1 姿態估測系統建置....................................44
4.1.1 姿態估測系統平台建置....................................44
4.1.2 相機與投影機校正....................................44
4.1.3 樣本資料庫建置....................................47
4.1.4 姿態估測模擬結果....................................48
4.2 姿態估測系統實驗....................................50
4.2.1 結構光建模精度實驗....................................50
4.2.2 濾波器對系統影響實驗....................................51
4.2.3 點雲匹配影響實驗....................................53
4.2.4 姿態估測系統精度實驗....................................55
4.3 機械手臂智慧夾取系統建置....................................58
4.3.1 初始夾取姿態教導....................................58
4.3.2 目標物夾取實驗....................................60
4.3.3 姿態估測之智慧夾取系統....................................61
4.4 機械手臂離線編程系統建置....................................63
4.4.1 離線編程系統平台與座標系整合....................................63
4.4.2 機械手臂路徑銑削之工法編程....................................64
4.4.3 Fanuc機械手臂通訊監控系統....................................66
4.4.4 影像基準點校正系統....................................66
4.5 雲端機聯網平台建置....................................67
4.5.1 MQTT雲端通訊管理系統建置....................................67
4.5.2 雲端網頁建置....................................69
第 五 章 結論與未來展望....................................70
參考文獻....................................71
Extended Abstract....................................73

[1]Hirofumi Nakai, Daisuke Iwai, Kosuke Sato,” 3D shape measurement using fixed camera and handheld laser scanner”, SICE Annual Conference, 2008.
[2]Christian Piccolo, “A LOW-COST, EFFICIENT AND ACCURATE HUMAN BODY SCANNER”, Dipartimento di Ingegneria Informatica , Università Degli Studi di Padova.
[3]D. Scharstein, R. Szeliski, R. Zabih,” A taxonomy and evaluation of dense two-frame stereo correspondence algorithms”, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), 2001.
[4]中華民國行政院, “智慧機械產業推動方案規劃”.
[5]Siemens NX, ”https://www.plm.automation.siemens.com/global/en/products/nx/”.
[6]Dassault CATIA, ”https://www.3ds.com/products-services/catia/”.
[7]RobotMaster,”https://www.robotmaster.com/en/”.
[8]SprutCAM,“https://www.sprutcam.com/”.
[9]Radu Bogdan Rusu, Nico Blodow, Zoltan Csaba Marton, Michael Beetz, ”Aligning point cloud views using persistent feature histograms”, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008.
[10]Radu Bogdan Rusu, Gary Bradski, Romain Thibaux, John Hsu,” Fast 3D recognition and pose using the Viewpoint Feature Histogram”, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010.
[11]Aitor Aldoma, Markus Vincze, Nico Blodow, David Gossow, Suat Gedikli, Radu Bogdan Rusu, Gary Bradski,” CAD-model recognition and 6DOF pose estimation using 3D cues”, IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2012.
[12]MuhammadRizal, Jaharah A.Ghania, Mohd ZakiNuawia and Che Hassan Che Harona ”A combined 2D-3D vision system for automatic robot picking” Proceedings of the 2014 International Conference on Advanced Mechatronic Systems, 2014.
[13]Si Yongsheng, Liu Gang, Feng Juan,” Location of apples in trees using stereoscopic vision” Procedia Computer Science, Volume 112, March 2015, Pages 68-74.
[14]曾毓書,2017, ”任意擺放物件之三維定位與抓取”, 國立雲林科技大學電機工程系, 碩士學位論文.
[15]陳柏鈞, 2016, ”基於3D物件辨識與對位之機械手臂夾取系統”, 國立台北科技大學自動化科技研究所, 碩士學位論文.
[16]王偉毅, 2015, “使用非線性輪廓套合演算法於基於CAD模型之三維物體姿態估測”, 淡江大學電機工程學系機器人工程碩士班, 碩士學位論文.
[17]Changhyun Choi, Yuichi Taguchi, Oncel Tuzel, Ming-Yu Liu, Srikumar Ramalingam, ”Voting-Based Pose Estimation for Robotic Assembly Using a 3D Sensor”, IEEE International Conference on Robotics and Automation, 2012.
[18]Jason Geng, “Structured-light 3D surface imaging: a tutorial”, IEEE Intelligent Transportation System Society, March 2011.
[19]Radu Bogdan Rusu, “Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments”, Institut für Informatik , der Technischen Universität München.
[20]Fengjun Hu, Yanwei Zhao, “DISCRETE POINT CLOUD FILTERING AND SEARCHING BASED ON VGSO ALGORITHM”, 27th Conference on Modelling and Sumulation, May 2013.
[21]Zhaojin Lu, Seungmin Baek, Sukhan Lee,” Robust 3D Line Extraction from Stereo Point Clouds”, IEEE Conference on Robotics, Automation and Mechatronics, 2008.
[22]Radu Bogdan Rusu, Nico Blodow, Michael Beetz,” Fast Point Feature Histograms (FPFH) for 3D registration”, IEEE International Conference on Robotics and Automation, 2009.
[23]P.J. Besl, N.D. McKay,” A Method for Registration of 3D Shapes”, IEEE Trans on Pattern Analysis and Machine Intelligence, 1992.
[24]Z. Zhang, “A Flexible New Technique for Camera Calibration”, Technical Report of Microsoft Research, MSR-TR-98-71.
[25]OpenCV, “https://opencv.org/”.