臺灣博碩士論文加值系統

在當今工業4.0的發展背景下，智慧製造已成為提升製造業競爭力的重要方向。本研究著重於機械手臂的智能化應用，尤其是結合視覺辨識技術和多物件辨識，以實現更靈活和自動化的生產流程。利用TinyML （微型機器學習模型）技術和機器視覺算法，本研究探索了如何通過樹莓派這類邊緣裝置來控制機械手臂，實現對生產線上物件的準確識別和操作。

研究中使用了SparkFun Edge開發板和Himax CMOS Imaging Camera（超低功耗影像感測器）組成邊緣裝置，並在樹莓派上進行資料處理和機械手臂控制。通過TensorFlow Lite模型（輕量化的TensorFlow架構模型），成功實現了即時的影像處理和物件辨識。系統在識別準確率和運行速度方面均達到了預期效果，展示了TinyML技術在提升機械手臂智能化方面的潛力。此外，本研究選用MariaDB（輕量級關聯式資料庫）作為數據儲存的工具，其輕量級的特性適合在資源有限的樹莓派系統上運行，並配合Grafana（網頁線上即時數據圖形化頁面）進行數據的即時監控和視覺化，這不僅增強了系統的運行透明度，也便於進行即時數據分析和系統狀態監控，本研究實驗結果顯示，在本研究之使用環境中，作業成功率可達100%。

本研究的初步實現展示了智慧製造中機械手臂結合先進影像處理技術的可行性，並指出了進一步整合機器學習和自動化技術以適應不斷變化生產需求的潛力。這為未來工廠提供了向高度自動化和智能化過渡的技術參考和思路。

In the context of Industry 4.0, smart manufacturing has become a crucial direction for enhanc-ing the competitiveness of the manufacturing sector. This study focuses on the intelligent ap-plication of robotic arms, particularly integrating visual recognition technologies and multi-object recognition, to achieve more flexible and automated production processes. Utilizing TinyML (Tiny Machine Learning) technology and machine vision algorithms, this research ex-plores how to control robotic arms using edge devices like Raspberry Pi to accurately identify and manipulate objects on production lines.
The study employed the SparkFun Edge development board and Himax CMOS Imaging Camera, a low-power image sensor, to form an edge computing setup, with data processing and robotic arm control conducted on Raspberry Pi. By implementing TensorFlow Lite, a lightweight TensorFlow framework model, real-time image processing and object recognition were successfully achieved. The system met expected outcomes in terms of recognition accu-racy and operational speed, demonstrating the potential of TinyML technology in enhancing the intelligence of robotic arms. Additionally, MariaDB, a lightweight relational database, was selected for data storage due to its suitability for running on resource-constrained Raspberry Pi systems. Combined with Grafana, a web-based real-time data visualization tool, this setup not only enhanced the transparency of the system’s operation but also facilitated real-time da-ta analysis and system status monitoring. Experimental results showed a 100% success rate in the study's operational environment.
This preliminary implementation demonstrates the feasibility of integrating advanced image processing technologies with robotic arms in smart manufacturing. It also highlights the poten-tial for further integration of machine learning and automation technologies to adapt to the ever-changing production demands. This provides a technological reference and insights for future factories transitioning towards high-level automation and intelligence.

摘要……………………………………………………………………i
Abstract………………………………………………………………ii
誌謝……………………………………………………………………iii
目錄……………………………………………………………………iv
表目錄…………………………………………………………………vi
圖目錄………………………………………………………………vii
第一章 緒論…………………………………………………………1
1.1 研究背景……………………………………………………………1
1.2 研究動機與目的………………………………………………1
1.3 研究流程……………………………………………………………2
第二章 文獻探討………………………………………………………4
2.1 機器學習…………………………………………………………4
2.1.1 深度神經網路（Deep Neural Networks, DNNs）………5
2.1.2 TensorFlow與TensorFlow Lite………………………………6
2.1.3 TinyML…………………………………………………………8
2.2 微控制器 （Microcontroller Unit, MCU）…………………10
2.2.1 ESP32…………………………………………………………11
2.2.2 SparkFun Edge……………………………………………...12
2.3 樹莓派（Raspberry Pi）…………………………………...13
2.4 機械手臂………………………………………………………15
第三章 研究方法…………………………………………………17
3.1 研究架構………………………………………………………17
3.2 邊緣裝置選擇………………………………………………..17
3.2.1 開發板……………………………………………………….18
3.2.2 相機模組……………………………………………………18
3.3 機械手臂控制………………………………………………..21
3.4 模型部署………………………………………………………23
3.5 資料庫與資料視覺化………………………………………24
第四章 研究分析與成果……………………………………….25
4.1 研究設備………………………………………………………25
4.1.1 模型訓練設備……………………………………………25
4.1.2 模型運行環境……………………………………………25
4.1.3 UR5機械手臂…………………………………………….25
4.1.4 治具…………………………………………………………26
4.1.5 相機…………………………………………………………26
4.1.6 超音波距離感測器……………………………………...27
4.1.7 硬體接線圖………………………………………………..27
4.2 系統實作………………………..……………………………29
4.2.1 機器學習模型建立………………………………………29
4.2.2 機械手臂自動定位………………………………………31
4.2.3 機械手臂控制之實作……………………………………32
4.2.4 系統運作狀態監控記錄與視覺化…………………...33
4.3 實驗成果………………………………………………………35
4.3.1 放置一個物件之實驗記錄……………………………..37
4.3.2 放置兩個物件之實驗記錄……………………………..40
4.3.3 放置三個物件之實驗記錄……………………………..43
4.3.4 放置四個物件之實驗記錄……………………………..44
4.3.5 放置五個物件之實驗記錄……………………………..47
4.4 實驗結果說明………………………………………………..49
4.5 辨識目標準確度改善………………………………………50
第五章 結論與未來發展……………………………………….53
5.1 結論……………………………………………………………..53
5.2 未來發展………………………………………………………54
參考文獻…………………………………………………………...56
附錄一………………………………………………………………61
Extended Abstract………………………………………………..65

[1]Soroush, M., Baldea, M., & Edgar, T. F. (Eds.). (2020). Smart manufacturing: concepts and methods. Elsevier.
[2]Moore, M. R., & Buckner, M. A. (2012). Learning-from-signals on edge devices. IEEE Instrumentation & Measurement Magazine, 15(2), 40-44.
[3]蔡.,Pei-Ching.(2023).具即時聽覺辨識之機械手臂應用於螺栓鬆脫檢測之研究. http://ir.lib.ncu.edu.tw:88/thesis/view_etd.asp?URN=111322032111322032
[4]El Naqa, I., & Murphy, M. J. (2015). What is machine learning? (pp. 3-11). Springer International Publishing.
[5]Hastie, T., Tibshirani, R., Friedman, J., Hastie, T., Tibshirani, R., & Friedman, J. (2009a). Overview of supervised learning. The elements of statistical learning: Data mining, inference, and prediction, 9-41.
[6]Hastie, T., Tibshirani, R., Friedman, J., Hastie, T., Tibshirani, R., & Friedman, J. (2009b). Unsupervised learning. The elements of statistical learning: Data mining, inference, and prediction, 485-585.
[7]Zhu, X. J. (2005). Semi-supervised learning literature survey.
[8]Aueawatthanaphisut, A. (2024, May 18). Introduction to Generative AI. Medium. https://medium.com/@aueawatth.aue/introduction-to-generative-ai-b089832f5069
[9]AIF 技術發展中心. (2023, March 6). SVM的利器：基於三角函數的新核函數. 知勢. https://edge.aif.tw/express-svm/
[10]Jaber, S. (2023). 2023 Edge AI Technology Report. Chapter V: TinyML. Wevolver. https://www.wevolver.com/article/2023-edge-ai-technology-report-chapter-v-tinyml
[11]Eshratifar, A. E., Abrishami, M. S., & Pedram, M. (2019). JointDNN: An efficient training and inference engine for intelligent mobile cloud computing services. IEEE Transactions on Mobile Computing, 20(2), 565-576.
[12]Allen-Zhu, Z., Li, Y., & Song, Z. (2019, May). A convergence theory for deep learning via over-parameterization. In International conference on machine learning (pp. 242-252). PMLR.
[13]Chen, L., Lin, S., Lu, X., Cao, D., Wu, H., Guo, C., ... & Wang, F. Y. (2021). Deep neural network based vehicle and pedestrian detection for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 22(6), 3234-3246.
[14]Lu, X., Kang, X., Nishide, S., & Ren, F. (2019, December). Object detection based on SSD-ResNet. In 2019 IEEE 6th International Conference on Cloud Computing and Intelligence Systems (CCIS) (pp. 89-92). IEEE
[15]Li, X., Xu, Z., Shen, X., Zhou, Y., Xiao, B., & Li, T. Q. (2021). Detection of cervical cancer cells in whole slide images using deformable and global context aware faster RCNN-FPN. Current Oncology, 28(5), 3585-3601.
[16]Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., ... & Zheng, X. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467.
[17]Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747.
[18]Tensorflow. (2023). Basic Classification: Classify Images of Clothing. TensorFlow. https://www.tensorflow.org/tutorials/keras/classification
[19]Warden, P., & Situnayake, D. (2019). Tinyml: Machine learning with tensorflow lite on arduino and ultra-low-power microcontrollers. O'Reilly Media.
[20]Demosthenous, G., & Vassiliades, V. (2021). Continual learning on the edge with tensorflow lite. arXiv preprint arXiv:2105.01946.
[21]Rovai, M. (2021, January 27). Building an Intelligent Voice Assistant from Scratch. Medium. https://medium.com/towards-data-science/building-an-intelligent-voice-assistant-from-scratch-3d5749f4af07
[22]Lin, J., Zhu, L., Chen, W. M., Wang, W. C., & Han, S. (2023). Tiny machine learning: progress and futures [feature]. IEEE Circuits and Systems Magazine, 23(3), 8-34.
[23]Bagheri, M., Farshforoush, N., Bagheri, K., & Shemirani, A. I. (2023). Applications of artificial intelligence technologies in water environments: From basic techniques to novel tiny machine learning systems. Process Safety and Environmental Protection.
[24]Hayajneh, A. M., Aldalahmeh, S. A., Alasali, F., Al‐Obiedollah, H., Zaidi, S. A., & McLernon, D. (2024). Tiny machine learning on the edge: A framework for transfer learning empowered unmanned aerial vehicle assisted smart farming. IET Smart Cities, 6(1), 10-26.
[25]Liu, H., Wei, Z., Zhang, H., Li, B., & Zhao, C. (2022). Tiny machine learning (tiny-ml) for efficient channel estimation and signal detection. IEEE Transactions on Vehicular Technology, 71(6), 6795-6800.
[26]Siang, Y. Y., Ahamd, M. R., & Abidin, M. S. Z. (2021). Anomaly detection based on tiny machine learning: A review. Open International Journal of Informatics, 9(Special Issue 2), 67-78.
[27]Tsoukas, V., Boumpa, E., Giannakas, G., & Kakarountas, A. (2021, November). A review of machine learning and tinyml in healthcare. In Proceedings of the 25th Pan-Hellenic Conference on Informatics (pp. 69-73)..
[28]Rahman, M. H., Rana, H. K., Peng, S., Hu, X., Chen, C., Quinn, J. M., & Moni, M. A. (2021). Bioinformatics and machine learning methodologies to identify the effects of central nervous system disorders on glioblastoma progression. Briefings in Bioinformatics, 22(5), bbaa365.
[29]Capogrosso, L., Cunico, F., Cheng, D. S., Fummi, F., & Cristani, M. (2024). A machine learning-oriented survey on tiny machine learning. IEEE Access.
[30]Gousev, E. (2020, November). Big Opportunities for TinyML Applications: Everywhere and Always-On TinyML. TinyML Asia 2020. Online. Retrieved from https://cms.tinyml.org/wp-content/uploads/asia2020/tinyMLAsia2020d2p1-Welcome-Gousev.pdf
[31]Bolanakis, D. E. (2019). A survey of research in microcontroller education. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 14(2), 50-57.
[32]Gridling, G., & Weiss, B. (2007). Introduction to microcontrollers. Vienna University of Technology Institute of Computer Engineering Embedded Computing Systems Group, 25.
[33]Chien, T. K., Chiou, L. Y., Sheu, S. S., Lin, J. C., Lee, C. C., Ku, T. K., ... & Wu, C. I. (2016). Low-power MCU with embedded ReRAM buffers as sensor hub for IoT applications. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 6(2), 247-257.
[34]Zhang, Y., Li, J., Li, D., Tan, L., Yang, L., & Yu, B. (2019, October). Application Verification of Power IOT Low-power MCU in Laboratory Environment. In 2019 3rd International Conference on Electronic Information Technology and Computer Engineering (EITCE) (pp. 1366-1369). IEEE.
[35]Maier, A., Sharp, A., & Vagapov, Y. (2017, September). Comparative analysis and practical implementation of the ESP32 microcontroller module for the internet of things. In 2017 Internet Technologies and Applications (ITA) (pp. 143-148). IEEE.
[36]台灣智能感測科技有限公司. (2024). ESP32-DevKitC Espressif Systems 樂鑫原廠 WROOM-32D 開發板 ESP32-D0WD 板載天線. 台灣智能感測科技有限公司. https://www.taiwansensor.com.tw/product/esp32-devkitc-espressif-systems-%e6%a8%82%e9%91%ab%e5%8e%9f%e5%bb%a0-wroom-32d-%e9%96%8b%e7%99%bc%e6%9d%bf-esp32-d0wd-%e6%9d%bf%e8%bc%89%e5%a4%a9%e7%b7%9a/
[37]Rajasundhar, D., Perumal, P., Ravaneeshwaran, J., & Ayyachamy, S. (2023, September). IoT based health monitoring system. In AIP Conference Proceedings (Vol. 2831, No. 1). AIP Publishing.
[38]Al-Mashhadani, M., & Shujaa, M. (2022). IoT security using AES encryption technology based ESP32 platform. Int. Arab J. Inf. Technol., 19(2), 214-223.
[39]Setiawan, F. B. (2021, October). Securing data communication through MQTT protocol with AES-256 encryption algorithm CBC mode on ESP32-based smart homes. In 2021 International Conference on Computer System, Information Technology, and Electrical Engineering (COSITE) (pp. 166-170). IEEE.
[40]Koushal, A., Gupta, R., Jan, F., Kamaldeep, K., & Kumar, V. (2022, November). Home Automation System Using ESP32 and Firebase. In 2022 Seventh International Conference on Parallel, Distributed and Grid Computing (PDGC) (pp. 228-231). IEEE.
[41]Kholik, M. A., Setiawan, F. B., & Fauzi, A. (2023). Design and Implementation of A Smart Home System With The Internet of Things (IoT) Using Esp32. Inspiration: Jurnal Teknologi Informasi dan Komunikasi, 13(1), 22-30.
[42]Francisco, G. C. (2022). Dispositivo IoT para monitorar indicador de pressão através de processamento de imagem (Bachelor's thesis, Universidade Federal do Rio Grande do Norte).
[43]Watson, D. (2021, November 23). ESP32 MQTT. TheEngineeringProjects.Com. https://www.theengineeringprojects.com/2021/11/esp32-mqtt.html
[44]Hong, C. K., Abu, M. A., Shapiai, M. I., Haniff, M. F., Mohamad, R. S., & Abu, A. (2023). Analysis of Wind Speed Prediction using Artificial Neural Network and Multiple Linear Regression Model using Tinyml on Esp32. Journal of Advanced Research in Fluid Mechanics and Thermal Sciences, 107(1), 29-44.
[45]Sudharsan, B., Salerno, S., & Ranjan, R. (2022, October). TinyML-CAM: 80 FPS image recognition in 1 kB RAM. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking (pp. 862-864).
[46]Kareem, H., & Dunaev, D. (2021, May). The working principles of ESP32 and analytical comparison of using low-cost microcontroller modules in embedded systems design. In 2021 4th International Conference on Circuits, Systems and Simulation (ICCSS) (pp. 130-135). IEEE.
[47]Macheso, P. S., & Thotho, D. (2022). ESP32 Based Electric Energy Consumption Meter. International Journal of Computer Communication and Informatics, 4(1), 23-35.
[48]Lewis, P. (2020). Make a hack-proof garage door opener: A new breakout board offers cryptographic security-[hands on]. IEEE Spectrum, 57(3), 16-18.
[49]Sparkfun electronics. (2019, September 26). SparkFun Edge 2 Development Board - Artemis. Sparkfun START SOMETHING. https://www.sparkfun.com/products/retired/15420
[50]Huang, B. (2015, June). SparkFun Inventor's Kit with Arduino–Curriculum Exchange. In 2015 ASEE Annual Conference & Exposition (pp. 26-1388).
[51]Noak, C. J., Wang, S., Andrei, S., & Tsan, J. L. (2022, October). Introducing engineering and programming concepts to middle school and high school students using SparkFun Inventor’s Kit, Scratch, and Java. In 2022 IEEE Frontiers in Education Conference (FIE) (pp. 1-8). IEEE.
[52]Armbrust, M., Das, T., Davidson, A., Ghodsi, A., Or, A., Rosen, J., ... & Zaharia, M. (2015). Scaling spark in the real world: performance and usability. Proceedings of the VLDB Endowment, 8(12), 1840-1843.
[53]Daurai, B., Ramchiary, S. S., & Gogoi, M. (2023, March). Comparison of Sparkfun TRIAD AS7265x spectroscopy sensor device with a Spectrophotometer for qualitative and quantitative analysis. In 2023 4th International Conference on Computing and Communication Systems (I3CS) (pp. 1-3). IEEE.
[54]Brock, J. D., Bruce, R. F., & Cameron, M. E. (2013). Changing the world with a Raspberry Pi. Journal of Computing Sciences in Colleges, 29(2), 151-153.
[55]Raspberry pi. (2022). Raspberry Pi 4. Raspberry Pi. https://www.raspberrypi.com/products/raspberry-pi-4-model-b/
[56]Abdul Kadhar, K. M., Anand, G., Abdul Kadhar, K. M., & Anand, G. (2021). Introduction to the Raspberry Pi. Data Science with Raspberry Pi: Real-Time Applications Using a Localized Cloud, 49-78.
[57]Vedavalli, P., Kondaveeti, H. K., & Ch, D. (2022, March). A Review on Automated monitoring applications of Raspberry Pi. In 2022 8th International Conference on Advanced Computing and Communication Systems (ICACCS) (Vol. 1, pp. 485-492). IEEE.
[58]Karthikeyan, S., Raj, R. A., Cruz, M. V., Chen, L., Vishal, J. A., & Rohith, V. S. (2023). A systematic analysis on raspberry pi prototyping: Uses, challenges, benefits, and drawbacks. IEEE Internet of Things Journal, 10(16), 14397-14417.
[59]Brock, J. D., & Bruce, R. F. (2014). Sensing the world with a Raspberry pi. Journal of Computing Sciences in Colleges, 30(2), 174-175.
[60]Graves, R. (2022, April 23). Minecraft 1.18.2 on the Raspberry Pi 4, ODroid N2+, and PineBook Pro. Syonyk’s Project Blog. https://www.sevarg.net/2022/04/23/aarch64-minecraft-on-the-pi-4-and-others/
[61]Yang, K., Blaauw, D., & Sylvester, D. (2017). Hardware designs for security in ultra-low-power IoT systems: An overview and survey. IEEE Micro, 37(6), 72-89.
[62]Wachs, M., & Ip, D. (2015, June). Design and integration challenges of building security hardware IP. In Proceedings of the 52nd Annual Design Automation Conference (pp. 1-6).
[63]Duc, A. N., Jabangwe, R., Paul, P., & Abrahamsson, P. (2017, May). Security challenges in IoT development: a software engineering perspective. In Proceedings of the XP2017 scientific workshops (pp. 1-5).
[64]Rao, N. K., Avinash, N. J., Moorthy, H. R., Karthik, K., Rao, S., & Santosh, S. (2021, December). An automated robotic arm: a machine learning approach. In 2021 IEEE International Conference on Mobile Networks and Wireless Communications (ICMNWC) (pp. 1-6). IEEE.
[65]Moran, M. E. (2007). Evolution of robotic arms. Journal of robotic surgery, 1(2), 103-111.
[66]國泰綜合醫院. (2024, April 30). 達文西機器手臂微創手術介紹. 國泰綜合醫院. https://www.cgh.org.tw/ec99/rwd1320/category.asp?category_id=139
[67]Sharma, G., Ganesh, A. S., Kumar, S., Kharub, M., Anitha, A., & Kanwar, V. (2023, September). Substantial Capabilities of 3-Axis Robotic Arm in Industry Automation Applications. In 2023 3rd International Conference on Innovative Sustainable Computational Technologies (CISCT) (pp. 1-6). IEEE.
[68]Kiefer, D., Luo, X., Reimer, A., & Evans, D. (2023, August). Robotic machining: Status, challenges and future trends. In 2023 28th International Conference on Automation and Computing (ICAC) (pp. 1-6). IEEE.
[69]Yang, G. Z., Bellingham, J., Dupont, P. E., Fischer, P., Floridi, L., Full, R., ... & Wood, R. (2018). The grand challenges of science robotics. Science robotics, 3(14), eaar7650.
[70]Specian, A., Mead, R., Kim, S., Matarić, M., & Yim, M. (2021). Quori: A community-informed design of a socially interactive humanoid robot. IEEE Transactions on Robotics, 38(3), 1755-1772.
[71]Ockerbloom, M. M. (2020, February 19). File:Quori Socially Interactive Robot Platform IMG 20200219 165323219 02.Jpg. WIKIPEDIA. https://en.m.wikipedia.org/wiki/File:Quori_socially_interactive_robot_platform_IMG_20200219_165323219_02.jpg
[72]Wiredworkers. (2019). Universal Robots UR5. Wiredworkers. https://www.wiredworkers.io/cobot/brands/universal-robots/ur5/
[73]Onrobot. (2020). RG2 - 具有寬行程的靈活 2 指機械人夾持器. Onrobot. https://onrobot.com/zh-hant/%E7%94%A2%E5%93%81/rg2-jiachiqi
[74]台灣智能感測科技有限公司. (2019). Himax CMOS Imaging Camera – HM01B0 鏡頭模組. 台灣智能感測科技有限公司. https://www.taiwansensor.com.tw/product/himax-cmos-imaging-camera-hm01b0-%e9%8f%a1%e9%a0%ad%e6%a8%a1%e7%b5%84/
[75]台灣智能感測科技有限公司. (2018). HC-SR04P 超音波測距模組 HC-SR04 Plus 寬電壓 3.3-5V 輸入. 台灣智能感測科技有限公司. https://www.taiwansensor.com.tw/product/hc-sr04p-%e8%b6%85%e9%9f%b3%e6%b3%a2%e6%b8%ac%e8%b7%9d%e6%a8%a1%e7%b5%84-hc-sr04-plus-%e5%af%ac%e9%9b%bb%e5%a3%93-3-3-5v-%e8%bc%b8%e5%85%a5/
[76]TensorFlow Models. (2024). TensorFlow Model Garden: Object Detection Zoo [Documentation]. GitHub. Retrieved July 15, 2024, from https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md