臺灣博碩士論文加值系統

菇類栽培與採收過程中經常需要進行太空包之取放作業，不但耗費人力，對於菇體也會有一定程度損傷風險。另外，如何安全且不傷害菇體的前提下進行取放，並對菇體生長趨勢數據進行取放姿態數據採集，對未來達成自動化採收作業將有很重要貢獻。本研究嘗試將在模擬太空包取放環境的條件下(菇包籃、菇包支架)，使用六軸機械手臂並整合深度相機及yolo與PointNetGPD等演算法，完成抓取姿勢檢測(Grasp Pose Detection)，始能對菇類太空包進行安全且低風險之自動取放作業，並可以在遠端進行自動與手動控制系統切換，快速移動至指定目標位置完成後續目標取放作業。本機械手臂取放系統使用機器人作業系統(ROS ，Robot Operating System)設計與製作機器人功能整合架構，六軸機械手臂利用Moveit運動規劃器為機械手臂姿態移動的規劃運算。視覺系統基於eye in hand 架構在機械手臂末端link上加裝intel realsense D435i深度相機感測模組，為了高效率在虛擬環境中建立實際物件，使用Octmap將深度相機獲得的點雲數據轉換成空間物件達成三維地圖建構。結合 yolo 物件辨識偵測模型 與 PointNetGPD 點雲集生成抓握姿態系統，在實驗中驗證可行性與有效的降低菇體損傷率與風險。本研究所設計之遠端機械手臂控制器結合SQLite資料庫功能，可以在遠端對機械手臂動作姿態做查詢、新增、修改、刪除。本項功能對解決菇類產業自動化取放問題有很重要的貢獻。

Entering the era of big data, the demand for data collection of mushroom growth trends in mushroom cultivation has increased. Traditional pick-and-place space packs for mushroom bodies will have a certain degree of risk of mushroom body damage. How to be safe and not harm the mushroom body Picking and placing will be an important part of collecting mushroom body growth data. The research attempts to use a six-axis robotic arm to integrate depth vision and add an algorithm in a restricted environment (mushroom packs basket, mushroom packs holder), to safely and low-risk pick and place mushroom space packs and it can switch between automatic and manual control systems at the remote end, and quickly move to the designated target position to complete subsequent target pick-and-place operations. The robotic arm pick-and-place system uses the ROS (Robot Operating System) to design the robot's functional, The six-axis robotic arm uses the Moveit motion planner to plan and calculate the posture movement of the robotic arm. Based on the eye on hand architecture, the vision system is equipped with an intel realsense D435i depth camera on the link at the end of the robot arm. In order to efficiently create actual objects in the virtual environment, use Octmap to convert the point cloud data obtained by the depth camera into spatial objects to achieve 3D map construction. Combining the yolo object recognition and detection model with the PointNetGPD point cloud to generate a grasping attitude system, in the experiment, the feasibility and effective reduction of mushroom body damage rate and risk were verified. Research and Design The remote robot arm controller combined with the SQLite database can query, add, modify, and delete the movement and posture of the robot arm at the remote end. It has made a very important contribution to solving the automatic pick-and-place problem of the mushroom industry.

摘要……………………………………………………………………………………………………………………………i
Abstract………………………………………………………………………………………………………………………ii
誌謝…………………………………………………………………………………………………………………………iii
目錄…………………………………………………………………………………………………………………………iv
表目錄………………………………………………………………………………………………………………………vi
圖目錄………………………………………………………………………………………………………………………vii
第一章 緒論…………………………………………………………………………………………………………………1
1.1 前言………………………………………………………………………………………………………………………1
1.2 研究背景…………………………………………………………………………………………………………………1
1.3 研究動機與目的…………………………………………………………………………………………………………2
1.4 論文架構…………………………………………………………………………………………………………………2
第二章 文獻回顧……………………………………………………………………………………………………………3
2.1 機器人系統………………………………………………………………………………………………………………3
2.1.1 MoveIt!…………………………………………………………………………………………………………………6
2.1.2 Universal Robots ROS Drive………………………………………………………………………………………7
2.1.3 TF座標轉換……………………………………………………………………………………………………………8
2.1.4 四元數(Quaternion)…………………………………………………………………………………………………9
2.2 路徑規劃………………………………………………………………………………………………………………10
2.3.1 RRT路徑演算法……………………………………………………………………………………………………10
2.3.2 PRM路徑演算法 ……………………………………………………………………………………………………12
2.3 物件偵測 ………………………………………………………………………………………………………………14
2.3.1 YOLO …………………………………………………………………………………………………………………14
2.4 三維空間數據建立 ……………………………………………………………………………………………………16
2.5 夾取姿態規劃 …………………………………………………………………………………………………………18
第三章 研究方法 ……………………………………………………………………………………………………………19
3.1 取放系統架構圖 ………………………………………………………………………………………………………19
3.2 流程架構圖 ……………………………………………………………………………………………………………20
3.3 控制系統 ………………………………………………………………………………………………………………21
3.3.1 主控制系統 …………………………………………………………………………………………………………21
3.3.2 邊緣控制系統 ………………………………………………………………………………………………………21
3.4 驅動系統………………………………………………………………………………………………………………22
3.4.1 UR5 …………………………………………………………………………………………………………………22
3.4.2 Robotiq 2f-85電動夾爪 ……………………………………………………………………………………………24
3.4.3 柔性夾爪改良設計 …………………………………………………………………………………………………25
3.5 路徑規劃系統…………………………………………………………………………………………………………28
3.5.1 初始化參數調整……………………………………………………………………………………………………28
3.5.2 執行路徑……………………………………………………………………………………………………………30
3.5.3 菇類太空包取放姿態規劃…………………………………………………………………………………………32
3.6 視覺系統………………………………………………………………………………………………………………37
3.6.1 深度視覺感測器初始化設定………………………………………………………………………………………37
3.6.2 easy_handeye………………………………………………………………………………………………………38
3.6.3 Octomap……………………………………………………………………………………………………………41
3.6.4 Yolov4_darknet……………………………………………………………………………………………………42
3.6.5 Grasp Pose Detection………………………………………………………………………………………………46
3.7 SQLite資料庫…………………………………………………………………………………………………………48
3.8 網頁式UI介面……………………………………………………………………………………………………………49
3.8.1 後端_Flask……………………………………………………………………………………………………………49
3.8.2 前端_ros_web………………………………………………………………………………………………………51
3.8.3 ros3djs…………………………………………………………………………………………………………………52
3.9 菇包承座設計……………………………………………………………………………………………………………53
第四章 結果與討論……………………………………………………………………………………………………………55
4.1 菇類太空包取放姿態測試………………………………………………………………………………………………56
4.2 機械手臂整合深度視覺取放性能測試…………………………………………………………………………………58
4.2.1 Yolov4_darknet夾取性能測試 (菇包垂直擺放)………………………………………………………………………60
4.2.2 Yolov4_darknet夾取性能測試 (菇包水平擺放)………………………………………………………………………63
4.2.3 Grasp Pose Detection夾取性能測試(菇包垂直擺放)…………………………………………………………………67
4.3 遠端UI控制介面……………………………………………………………………………………………………………68
4.3.1 自動控制介面……………………………………………………………………………………………………………69
4.3.2 手動控制介面……………………………………………………………………………………………………………70
4.3.3 SQL資料庫介面…………………………………………………………………………………………………………72
4.3.4 開發者測試介面………………………………………………………………………………………………………74
第五章 結論與未來展望………………………………………………………………………………………………………75
5.1 結論………………………………………………………………………………………………………………………75
5.2 未來展望…………………………………………………………………………………………………………………75
參考文獻……………………………………………………………………………………………………………………76
附錄一 YOLOv4與YOLOv7性能比較……………………………………………………………………………………80
Extended Abstract……………………………………………………………………………………………………………81

[1]黃文信，2019，“基於深度學習之杏鮑菇自動化分級系統之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[2]蘇爾曼，2020，“創新菇類栽培物聯網”，國立虎尾科技大學機械設計工程系士班碩士論文，雲林縣。
[3]林明毅，2020，“杏鮑菇自動化採收機構之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[4]古丙駿，2020，“杏鮑菇採收深度學習物件辨識系統之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[5]顏靖峰，2022，“袋栽杏鮑菇完株採收機構之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[6]胡原愷，2021，“袋栽菇類製包流程智慧檢測系統之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[7]邱弘丞，2021，“自主移動式智慧物聯模組在菇類栽培之研究”，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[8]紀錦川，2023，“應用於菇類栽培感測與資料擷取之 ROS自主移動機器人研發” ，國立虎尾科技大學機械設計工程系碩士班碩士論文，雲林縣。
[9]Z. Cui, H. Xu, Z. Chen, H. Yang, S. Huang and M. Gong, "Design of a novel AGV with automatic pick-and-place system based on scissor lifting platform," 2020 Chinese Automation Congress (CAC), Shanghai, China, 2020, pp. 4435-4440.
[10]Y. Mohamed and S. Lemaignan, "ROS for Human-Robot Interaction," 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 2021, pp. 3020-3027.
[11]T. Xu, H. -Z. Liu, R. Huang, Z. Yao and J. -Y. Xu, "Research on Motion Trajector Planning of Industrial Robot Based on ROS," 2022 7th International Conference on Automation,Control and Robotics Engineering (CACRE), Xi'an, China, 2022, pp. 58-62.
[12]C. He, C. Deng, N. Li and Z. Miao, "Design of Vision Control System of Tomato Picking Robot," 2021 40th Chinese Control Conference (CCC), Shanghai, China, 2021, pp.4267-4271.
[13]T. Haldankar, S. Kedia, R. Panchmatia, D. Parmar and D. Sawant, "Design of Robotic Manipulator for Welding using ROS and Gazebo," 2022 IEEE Delhi Section Conference
(DELCON), New Delhi, India, 2022, pp. 1-6.
[14]David Coleman, Ioan A. Șucan, Sachin Chitta, Nikolaus Correll, Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study, Journal of Software Engineering for Robotics, May 2014, 5(1):pp.3–16.
[15]ROS.org ，2019年12月，Universal Robots ROS driver supporting CB3 and e-Series。2022年7月，取自http://wiki.ros.org/ur_robot_driver
[16]Jaiganesh Sekar，2019 年 11 月， " Ec6003 robotics and automation notes"，2023年3月，取自https://www.slideshare.net/JaiGanesh11/ec6003-robotics-and-automation-notes
[17]趙清榮，2016，“結合機械視覺導引於六軸機械手臂定位運動與取放控制之應用“，國立高雄應用科技大學模具系碩士在職專班碩士論文，高雄市。
[18]黃子懿，2019，“機械手臂加工精度提升探討”，國立成功大學航空太空工程學 系碩士在職專班碩士論文，台南市。
[19]C. Yuan, W. Zhang, G. Liu, X. Pan and X. Liu, "A Heuristic Rapidly-Exploring Random Trees Method for Manipulator Motion Planning," in IEEE Access, vol. 8, 2020, pp. 900-910.
[20]Shubham Shukla, Lokesh Kumar, Titas Bera, Ranjan Dasgupta, "A Levy Flight based Narrow Passage Sampling Method for Probabilistic Roadmap Planners" 2021 arXiv:2107.00817
[21]Alexey Bochkovskiy and Chien-Yao Wang and Hong-Yuan Mark Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection" 2020 arXiv: 2004.10934,2020
[22]A. Gouda, A. Ghanem and C. Reining, "DoPose-6D dataset for object segmentation and 6D pose estimation," 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas, 2022, pp. 477-483.
[23]C. -W. Chen, A. -C. Tsai, Y. -H. Zhang and J. -F. Wang, "3D object detection combined with inverse kinematics to achieve robotic arm grasping," 2022 10th International Conference on Orange Technology (ICOT), Shanghai, China, 2022, pp. 1-4.
[24]Andreas ten Pas, Marcus Gualtieri, Kate Saenko, and Robert Platt. October 2017”Grasp Pose Detection in Point Clouds. The International Journal of Robotics Research, Vol 36, Issue 13-14, pp. 1455-1473.
[25]Marcus Gualtieri, Andreas ten Pas, Kate Saenko, and Robert Platt. High precision grasp pose detection in dense clutter. IROS 2016, pp. 598-605.
[26]J. Gonçalves and P. Lima, "Grasp Planning with Incomplete Knowledge About the Object to be Grasped," 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Porto, Portugal, 2019, pp. 1-6.
[27]K. Miatliuk, M. Pietrzak and A. G. Buch, "Controlling an industrial robotic system based on received visual information," 2019 20th International Carpathian Control Conference (ICCC), Krakow-Wieliczka, Poland, 2019, pp. 1-6.
[28]X. Zhen, J. C. Y. Seng and N. Somani, "Adaptive Automatic Robot Tool Path Generation Based on Point Cloud Projection Algorithm," 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain, 2019, pp. 341-347.
[29]S. Gatesichapakorn, J. Takamatsu and M. Ruchanurucks, "ROS based Autonomous Mobile Robot Navigation using 2D LiDAR and RGB-D Camera," 2019 First International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP), Bangkok, Thailand, 2019, pp. 151-154.
[30]方怡丹、蔡清榮，2016，“臺灣菇類產業之發展與輔導”，菇類生技產業研討會專刊。
[31]行政院農業委員會農業知識入口網，2017年8月，杏鮑菇栽培技術（二）。2022年12月，取自https://kmweb.coa.gov.tw/theme_data.php?theme=news&sub_theme=variety&id=54786
[32]邱相文、石信德、呂昀陞，2019，“菇類栽植盛盤自動化上下架系統之開發”菇類智慧化生產與農場經營管理研討會專刊。2022年9月，取自https://scholars.tari.gov.tw/bitstream/123456789/8031/1/no216-6.pdf
[33]周榮源、石信德、黃文信，2020，“袋栽菇類製包技術回顧與智慧化發展初探”，行政院農委會農業科技決策資訊平台。2023年3月，取自https://agritechforesight.atri.org.tw/article/contents/3144
[34]林思妤、杜元凱、游舜期、傅月英、林大鈞。2021。植物表型體分析平台於蔬菜自動外表 型分析之應用。台灣農業研究 70卷，pp.11-23，2022年9月。
[35]R. K. Megalingam, R. V. Rohith Raj, T. Akhil, A. Masetti, G. N. Chowdary and V. S. Naick, "Integration of Vision based Robot Manipulation using ROS for Assistive Applications," 2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2020, pp. 163-169.
[36]S. S. Harithas, H. V. Mekali and S. Sunkara, "GENERIK: An Objective function Optimization Approach to Inverse Kinematics," 2020 IEEE International Symposium on Smart Electronic Systems (iSES) (Formerly iNiS), Chennai, India, 2020, pp. 238-241.
[37]Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao, "Scaled-YOLOv4:Scaling Cross Stage Partial Network” 2020 arXiv:2011.08036v2
[38]T. Huang, H. Xu and D. Liu, "ROS Based Obstacle Avoidance Motion Planning of UR5 Manipulator," 2021 IEEE International Conference on Networking, Sensing and Control (ICNSC), Xiamen, China, 2021, pp. 1-6.
[39]BUSAM, Benjamin; JUNG, Hyun Jun; NAVAB, Nassir. "I like to move it: 6d pose estimation as an action decision process. " arXiv preprint arXiv:2009.12678, 2020.
[40]C. C. B. Viturino, D. M. de Oliveira, A. G. S. Conceição and U. Junior, "6D Robotic Grasping System using Convolutional Neural Networks and Adaptive Artificial Potential Fields with Orientation Control," 2021 Latin American Robotics Symposium (LARS), 2021 Brazilian Symposium on Robotics (SBR), and 2021 Workshop on Robotics in Education (WRE), Natal, Brazil, 2021, pp. 144-149.
[41]R. Osorio-Comparán et al., "Recognition and simulation of a vision system for a KUKA KR-5 industrial manipulator," 2021 IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), Valparaíso, Chile, 2021, pp. 1-6.
[42]V. Lanka, V. B, D. Vinjamuri and B. Bhattacharyya, "Designing and Implementing Robot-Web-Suite, A Cloud Based Robotics Platform," 2021 IEEE International Conference on Robotics, Automation, Artificial-Intelligence and Internet-of-Things (RAAICON), Dhaka, Bangladesh, 2021, pp. 46-49.
[43]A. Wendt and T. Schüppstuhl, "Proxying ROS communications — enabling containerized ROS deployments in distributed multi-host environments," 2022 IEEE/SICE International Symposium on System Integration (SII), Narvik, Norway, 2022, pp. 265-270.
[44]W. Wang, W. Cao, W. Gao and Z. Du, "A planning system for robot sampling task-based on ROS framework," 2017 IEEE International Conference on Unmanned Systems (ICUS), Beijing, China, 2017, pp. 347-352.
[45]Y. Ogiwara, A. Yorozu, A. Ohya and H. Kawashima, "Making ROS TF Transactional," 2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS), Milano, Italy, 2022, pp. 318-319
[46]H. Deng, J. Xiong and Z. Xia, "Mobile manipulation task simulation using ROS with MoveIt," 2017 IEEE International Conference on Real-time Computing and Robotics (RCAR), Okinawa, Japan, 2017, pp. 612-616.
[47]C. Sun, X. Liu and R. Shen, "Research on Collision Avoidance Path Planning of Dual Manipulator Robot Based on Fusion Algorithm," 2022 International Conference on Automation, Robotics and Computer Engineering (ICARCE), Wuhan, China, 2022, pp. 1-6.
[48]L. Miranda, D. Jiménez, D. Benítez, N. Peréz, D. Riofrío and R. F. Moyano, "Robotic Arm Handling Based on Real-time Gender Recognition Using Convolutional Neural Networks," 2022 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC), Ixtapa, Mexico, 2022, pp. 1-6.
[49]J. W. Lai, H. R. Ramli, L. I. Ismail and W. Z. W. Hasan, "Real-Time Detection of Ripe Oil Palm Fresh Fruit Bunch Based on YOLOv4," in IEEE Access, vol. 10, pp. 95763-95770, 2022
[50]H. Bildstein, A. Durand-Petiteville and V. Cadenat, "Visual Predictive Control Strategy for Mobile Manipulators," 2022 European Control Conference (ECC), London, United Kingdom, 2022, pp. 1672-1677.
[51]Hornung, Armin and Wurm, Kai M and Bennewitz, Maren and Stachniss, Cyrill and Burgard, 2013, OctoMap: An efficient probabilistic 3D mapping framework based on octrees,Wolfram, Autonomous Robots.
[52]P. Junare, M. Deshmukh, M. Kulkarni and P. Bartakke, "Deep Learning based end-to-end Grasping Pipeline on a lowcost 5-DOF Robotic arm," 2022 IEEE 19th India Council International Conference (INDICON), Kochi, India, 2022, pp. 1-6.
[53]M. Mahmeen, R. D. D. Sanchez, M. Friebe, M. Pech and S. Haider, "Collision Avoidance Route Planning for Autonomous Medical Devices Using Multiple Depth Cameras," in IEEE Access, vol. 10, 2022., pp. 29903-29915