臺灣博碩士論文加值系統

在工業高速發展的時代，機械手臂抓取及放置是自動化生產線的基本關鍵任務。自動化生產線所需的精度和速度得到了大幅的提升。然而，沖壓工業中大部分工件取放的工作仍然是手工操作，僅有少數研究使用結構光三維掃描儀建立靜止狀態的工件模型，以利於機械手臂取放工件。故本研究提出了一種基於雙立體視覺動態掃描及整合機械手臂取放，以應用於沖壓工件之全自動生產線的智能系統。特別是，輸送帶上放置的工件可以即時被建模為機械手臂動態抓取的依據。工件的點雲模型是通過立體校准後利用兩個深度相機Azure Kinect Microsoft（飛行時間原理傳感器)來獲得的，且可以通過應用多種算法來估計工件的 6D 姿態包括 3 個平移向量和 3 個旋轉角度，此演算法主要包括用於物件分割的歐幾里德聚類提取；快速點特徵直方圖（FPFH）用於評估目標和參考對象點雲的特徵；傳統粗疊合 (SAC-IA) 和迭代最近點 (ICP) 分別用於粗略和完成，將目標點與參考點對齊。對工件進行建模後，由微控制器控制輸送帶將動態工件傳送給機械手臂。為了完成動態任務，一個方程式實現了輸送帶速度和機械手臂移動速度。此外，還通過軟體來補償誤差--非線性校正算法，來減少機械手臂的誤差以及系統誤差。我們的系統與實際測量的 6D 姿態信息平均誤差低於 7%。儘管硬體有限，但結果顯示本研究具有很高的前景且適用於實際工業，並具有進一步調查和研究的潛力。

At the era of rapid development in industry, the automatic production line is the fundamental and crucial mission for robotic pick and place. The accuracy and speed required by automated production lines have been greatly improved. However, most of the production works for picking and placing workpieces are still manual operations in the stamping industry. Few studies used the robot to pick and place workpieces that are required in the static state to build the workpiece model using structure light scanning. Therefore, an intelligent system that fully automatic production line integrated with the robotic arm instead of human work needs to be developed. The study proposes a dynamic workpiece modeling with robotic pick-place based on two stereo vision scanning for the stamping industry. In particular, the accumulative workpieces on the conveyor belt can be modeled for robotic dynamic grasping. The point cloud models of workpieces are acquired by leveraging two depth cameras, type Azure Kinect Microsoft (time of flight principle sensor) after stereo calibration. The 6D poses of a workpiece included 3 translations and 3 rotations can be estimated by applying a number of algorithms: Euclidean cluster extraction for object segmentation; fast point feature histogram (FPFH) for evaluating the characteristic features of the target and reference object point clouds; sample consensus initial alignment (SAC-IA) and iterative closest point (ICP) for coarse and finish, respectively, aligning the target with the reference. After modeling the workpiece, a conveyor controlled by a microcontroller will deliver the dynamic workpiece to the robot. In order to accomplish the dynamic task, an equation related to the velocity of the conveyor and the moving speed of the robot is implemented. Furthermore, the error of the robot is also focused to reduce the system error by applying the nonlinear correction algorithm - a software error compensation. The average error of 6D pose information between our system and real measurement is lower than 7%. Despite limited hardware, the result demonstrated that this study achieves high promise that is suitable for practical industry and has a high potential for further investigation and research.

摘要...i
Abstract...ii
Acknowledgment...iv
Contents...v
List of Tables...vii
List of Figures...viii
Symbol Description...xii
Chapter 1: Introduction...1
1.1 Research background and motivation...2
1.2 Thesis purpose...2
1.3 Organization of thesis...3
1.4 Adopted technologies of thesis...3
Chapter 2: Literature review...4
2.1 Research on development of metal forming automation line...4
2.2 Research on computer vision applications in pose estimation...8
Chapter 3: Research methodology...14
3.1 System devices and specification...14
3.1.1 Intelligent gripping robotic arm...15
3.1.2 Angular and distance rangefinder...15
3.1.3 Depth camera for modeling...15
3.1.4 Conveyor control devices...16
3.2 Point cloud construction and pre-processing...18
3.2.1 Point cloud generator using ToF camera...19
3.2.2 Stereo calibration principle for two Azure Kinect cameras...21
3.2.3 Setup region of interest...25
3.2.4 Outlier noise filter...26
3.2.5 Point cloud downsampling...28
3.2.6 Plane estimation...30
3.2.7 Object segmentation...31
3.3 Pose estimation system construction...33
3.3.1 Surface normal vector estimation...33
3.3.2 Feature point descriptor...34
3.3.3 Coarse alignment matching...35
3.3.4 Finish alignment matching...36
3.3.5 Conversion from transformation matrix to translation and angular position...37
3.4 Robotic arm controller and communication construction...39
3.4.1 Syntec robot arm error compensation...39
3.4.2 Gripping pose of the robot arm calculation...41
3.4.3 Syntec robot arm communication construction...42
3.4.4 Syntec robot arm program construction...43
3.5 Conveyor controller and communication construction...45
3.5.1 Transmission system design and communication...46
3.5.2 Synchronization of transmission conveyor and robot arm...47
Chapter 4: Experimental result...49
4.1 Post-processing experiment...50
4.1.1 Experiment setup...50
4.1.2 Azure Kinect camera stereo calibration results...51
4.1.3 Database references in experiment...53
4.2 Pose estimation system accuracy experiments...54
4.2.1 Accuracy of point cloud to point cloud...54
4.2.2 Influence of workpiece position on modeling accuracy...56
4.2.3 Influence of workpiece height on modeling accuracy...58
4.2.4 Accuracy of the pose estimation system...59
4.3 Smart feeding system experiment...63
4.3.1 Robot error compensation results...63
4.3.2 Smart feeding system Human-machine interface program...65
4.3.3 Experiment of static stack workpiece feeding system...66
4.3.4 Experiment of dynamic stack workpiece feeding system...70
Chapter 5: Conclusion and development direction...73
References...74
Appendix...77
Extended abstract...82

[1] Hossein Karbasian, A. Erman Tekkaya, "A review on hot stamping," Journal of Materials Processing Technology, vol. 210, no. 15, pp. 2103-2118, 2010.
[2] L. Li, "Time-of-Flight Camera – An Introduction," Technical White Paper SLOA190B, 2014.
[3] Tyler Bell, Beiwen Li, Song Zhang, "Structured Light Techniques and Applications," Wiley Encyclopedia of Electrical and Electronics Engineering, 2016.
[4] Cao. Qixian, Xiao. Xiaoting, "Development of Stamping Process CAD System on Automatic Production Line for Robotic Manipulation," Applied Mechanics and Materials, vol. 556, no. 562, pp. 2256-2261, 2014.
[5] Gang Zheng, LiangWang, Qiang Wang, Yi Sun, Ju Meng, Yisheng Zhang, "Research on the cooperative work of multi manipulator in hot stamping production line," in 5th International Conference on Advanced Design and Manufacturing Engineering (ICADME 2015), 2015.
[6] M. Yang, "Smart metal forming with digital process and IoT," International Journal of Lightweight Materials and Manufacture, vol. 1, no. 4, pp. 207-214, 2018.
[7] Tobias Handreg, Pascal Froitzheim, Normen Fuchs, Wilko Flügge, Michael Stoltmann, Christoph Woernle, "Concept of an automated framework for sheet metal cold forming," Tagungsband des 4. Kongresses Montage Handhabung Industrieroboter, pp. 117-127, 2019.
[8] Ireneusz Wrobel, Marcin Sidzina, "Design Study for Automatic Production Line of a Sub-Assemblies of New Generation Car Body Structures Compliant with the “Industry 4.0” Concept," Sensors, vol. 21, no. 7, pp. 2434-2441, 2021.
[9] Woo-han Yun, Jaeyeon Lee, Joo-Haeng Lee, Jaehong Kim, "Object recognition and pose estimation for modular manipulation system: Overview and initial results," in 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, South Korea , 2017.
[10] Richard Vock, Alexander Dieckmann, Sebastian Ochmann, Reinhard Klein, "Fast template matching and pose estimation in 3D point clouds," Computers & Graphics, vol. 79, pp. 36-45, 2019.
[11] MyungHwan Jeon, Yeongjun Lee, Young-Sik Shin, Hyesu Jang, Ayoung Kim, "Underwater Object Detection and Pose Estimation using Deep Learning," IFAC-PapersOnLine, vol. 52, no. 21, pp. 78-81, 2019.
[12] Deping Li, Hanyun Yang, Ning Liu, Xiaoming Yang, Jin Xu, "3D Object recognition and Pose estimation from Point cloud using Stably Observed Point Pair Feature," IEEE Access, vol. 8, pp. 44335-44345, 2020.
[13] L.-R. Huang, “Pose Estimation based on Point-Feature Matching for Robotic Random Bin-Picking of Multiple Objects,” Master's degree Thesis in National Formosa University, 2020.
[14] Do Quoc Trung, Wen-yang Chang, Li-wei Chen, "Solving jigsaw puzzle by using Robot-Vision integrated system Based on KAZE algorithm," in 37th National Conference on Mechanical Engineering of CSME, Yunlin, Taiwan, 2020.
[15] "Arduino Mega 2560 Rev3," Ivrea Interaction Design Institute, 2021. [Online]. Available: https://store.arduino.cc/usa/mega-2560-r3. [Accessed 24 September 2021].
[16] "Stepper Motor controller IC," Texax Instruments, September 2011. [Online]. Available: https://www.elecrow.com/download/drv8825.pdf. [Accessed 26 September 2021].
[17] "2-Phase stepping systems F2 catalog," Sanyo Denki, October 2012. [Online]. Available: https://www.sanyodenki.com/archive/document/product/servo/catalog_E_pdf/SANMOTION_F2_E.pdf. [Accessed 29 September 2021].
[18] Tetyana Sych, Phil Meadows, Brent Allen, "Azure Kinect DK depth camera," Microsoft, 26 June 2019. [Online]. Available: https://docs.microsoft.com/en-us/azure/kinect-dk/depth-camera. [Accessed 4 October 2021].
[19] S. Ghosh, "Stereo calibration using C++ and OpenCV," 9 September 2016. [Online]. Available: https://sourishghosh.com/2016/stereo-calibration-cpp-opencv/. [Accessed 30 September 2021].
[20] R. B. Rusu, "Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments," PhD Dissertation in Technischen Universität München, 2010.
[21] Fengjun Hu, Yanwei Zhao, Wanliang Wang, Xianping Huang, "Discrete Point Cloud Filtering And Searching Based On VGSO Algorithm," in 27th Conference on Modelling and Simulation, 2013.
[22] Martin A. Fischler, Robert C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Apphcatlons to Image Analysis and Automated Cartography," Communications of the Association for Computing Machinery, vol. 24, no. 6, pp. 381-395, 1981.
[23] Radu Bogdan Rusu, Nico Blodow, Michael Beetz, "Fast Point Feature Histograms (FPFH) for 3D Registration," in 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009.
[24] Paul J. Besl, Neil D. McKay, "A Method for Registration of 3-D Shapes," IEEE Transactions on Pattern analysis and Machine Interlligence, vol. 14, no. 2, 1992.
[25] Z. Ji-ming, "Research on Nonlinear Correction Algorithm of 2D PSD Based on Polynomial," Ship Science and Technology, vol. 41, no. 6, pp. 89-93, 2019.