臺灣博碩士論文加值系統

本研究結合深度學習框架 (Deep Learning Architecture) Tensorflow 與機器手臂，設計一套藉由影像辨識分類物品的系統，分為兩個部分：用於影像分類的機器學習、六軸機械手臂控制。本架構透過機械手臂分類螺栓螺絲進行驗證。

研究過程中,需預先將資料進行標註並輸入至神經網路進行訓練，透過損失率的觀察可得知模型收斂與否，訓練完成後將神經網路輸出並儲存。將圖片輸入至系統進行辨識得到座標，使用六軸機械手臂將得到的座標點進行路徑規劃 (Path Planning) 並執行分類。

本論文以「上銀智慧機械手實作競賽」之「智慧分類」題目驗證，應用深度學習結合影像與機器人的可行性，設計本次影像辨識之方法：將散落於盒內的螺栓螺帽，透過影像辨識，進行分類。

從實際驗證可得知，本論文提出之方法可有效率達成下列三個目的：(1) 透過虛擬環境進行標註資料; (2) 解決堆疊物品的辨識; (3) 使機器手臂準確地吸取物件並進行分類。

This study combines the Deep Learning Architecture Tensorflow with the robotic arm to design a system for identifying items by image recognition. It is divided into two parts: machine learning for image classification and six-axis robotic arm control. This architecture is verified by the mechanical arm classification of bolts and nuts.

During the research,the data needs to be marked in advance and input into the neural network for training.Through the observation of the loss rate, it can be known whether the model converges or not. After the training is completed, the neural network is output and stored. Afterwards, enter the image into the system for identification to obtain the coordinates, and use the six-axis robotic arm to perform path planning and classify the obtained coordinate points.

This paper is validated by the "intelligent classification" topic of "Hiwin Intelligent Robotic Arm Competition". Applying deep learning combined with the feasibility of images and robots, this method of image recognition is designed: the bolts and nuts scattered in the box, Sort by image recognition.

From the actual verification, the method proposed in this paper can effectively achieve the following three objectives: (1) labeling data through the virtual environment; (2) solving the identification of stacked items; (3) making the robot arm accurately pick up the objects and classify.

中文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
英文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
誌謝 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
表目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
圖目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
符號說明.............................................. ix
第一章 緒論........................................... 1
1.1 研究動機與目的 ........................... 1
1.2 文獻探討............................... 1
1.2.1 六軸機械手臂數學模型.................... 1
1.2.2 影像辨識與校正 ....................... 1
1.2.3 深度學習影像辨識 ...................... 1
1.3 論文架構............................... 2
第二章 機械手臂運動學................................... 3
2.1 齊次坐標............................... 3
2.2 機械手臂座標系定義......................... 5
2.3 順向運動學.............................. 8
2.4 逆向運動學..............................10
2.5 歐拉角表示法.............................16
第三章 人工神經網路....................................18
3.1 人工神經網路.............................18
3.1.1 神經網路框架......................... 18
3.1.2 神經網路計算理論 ...................... 19
3.1.3 反向傳播 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 卷積神經網路.............................21
3.2.1 卷積層.............................22
3.2.2 池化層.............................22
3.2.3 線性整化層.......................... 24
3.2.4 全連結層 . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 FasterR-CNN............................. 26
3.3.1 RegionProposalNetwork(RPN) ............... 26
3.3.2 訓練FasterR-CNN網路................... 30
第四章 影像校正.......................................34
4.1 攝影機參數..............................34
4.1.1 內部參數 . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1.2 外部參數 . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.3 畸變參數 . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
第五章 實驗結果與探討...................................39
5.1 實驗架構...............................39
5.1.1 辨識物件架構......................... 39
5.1.2 內部參數與畸變參數校正架構................ 39
5.1.3 外部參數校正架構 ...................... 42
5.1.4 FasterRCNN網路訓練架構 ................. 42
5.2 軟硬體簡介..............................43
5.2.1 Tensorflow框架........................ 43
5.2.2 人機介面 . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.2.3 機械手臂 . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.3 實驗步驟...............................49
5.3.1 FasterR-CNN識別實驗 ................... 49
5.3.2 機械手臂進行分類實驗.................... 51
第六章 結論與未來方向...................................55
6.1 結論..................................55
6.2 未來規劃...............................55
參考文獻. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
英文論文大綱. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

[1] L.-W. Tsai, “Robot Analysis: The Mechanics of Serial and Parallel Manipu- lators.” 1999.
[2] S. Kucuk and Z. Bingul, “Robot Kinematics:Forward and Inverse Kinemat- ics,” 2006, pp. 117–148.
[3] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.
[4] R. Laganière, “OpenCV 2 Computer Vision Application Programming Cook- book.” 2011.
[5] M. Goljan and J. Fridrich, “Estimation of Lens Distortion Correction from Single Images.” 2014.
[6] “Camera Calibration and 3D Reconstruction.” https://docs.opencv.org/2.4/ modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#id2 [Online accessed 2019-06-27].
[7] W. Liu et al., “Ssd: Single Shot Multibox Detector,” in European conference on computer vision, 2016, pp. 21–37.
[8] R. Girshick, “Fast R-Cnn,” in Proceedings of the ieee international confer- ence on computer vision, 2015, pp. 1440–1448.
[9] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-Cnn: Towards Real-Time Object Detection with Region Proposal Networks,” in Advances in neural informa- tion processing systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds. Curran Associates, Inc., 2015, pp. 91–99.
[10] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” in Proceedings of the ieee conference on computer vision and pattern recognition, 2016, pp. 779–788.
[11] Y.-H. Chang, P.-L. Chung, and H.-W. Lin, “Deep Learning for Object Iden- tification in Ros-Based Mobile Robots,” in 2018 ieee international conference on applied system invention (icasi), 2018, pp. 66–69.
[12] M. Quigley et al., “ROS: An Open-Source Robot Operating System.” 2009.
[13] S. Caldera, A. Rassau, and D. Chai, “Review of Deep Learning Methods in Robotic Grasp Detection,” Multimodal Technologies and Interaction, vol. 2, no. 3, p. 57, 2018.
[14] H. A. Pierson and M. S. Gashler, “Deep Learning in Robotics: A Review of Recent Research,” Advanced Robotics, vol. 31, no. 16, pp. 821–835, 2017.
[15] L. Tai and M. Liu, “Deep-Learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why Not.” Dec-2016.
[16] M. Abadi et al., “TensorFlow: Large-Scale Machine Learning on Heteroge- neous Systems.” 2015.
[17] Y. Tian, X. Li, K. Wang, and F.-Y. Wang, “Training and Testing Object Detectors with Virtual Images,” IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 2, pp. 539–546, 2018.
[18] J. Huang et al., “Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors,” in Proceedings of the ieee conference on computer vision and pattern recognition, 2017, pp. 7310–7311.
[19] X. Ling, Y. Zhao, L. Gong, C. Liu, and T. Wang, “Dual-Arm Cooperation and Implementing for Robotic Harvesting Tomato Using Binocular Vision,” Robotics and Autonomous Systems, vol. 114, pp. 134–143, 2019.
[20] Hartenberg, “Denavit–Hartenberg Parameters.” https://en.wikipedia.org/ wiki/Denavit%E2%80%93Hartenberg_parameters [Online accessed 2018-06-12], 2008.
[21] “Artificial Neural Network.” https://en.wikipedia.org/wiki/Artificial_ neural_network [Online accessed 2019-06-27].
[22] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Representa- tions by Back-Propagating Errors,” Nature, vol. 323, pp. 533–536, Oct. 1986.
[23] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet Classification with Deep Convolutional Neural Networks,” in Advances in neural information pro- cessing systems, 2012, pp. 1097–1105.