臺灣博碩士論文加值系統

AI技術發展至今，從物件偵測領域的YOLOv7與YOLOv8到影像分割領域的kMaX與SAM，可以發現這幾年的AI進步得非常快。而在這樣的環境下，我們認為，AI模型的發展會逐漸朝向個人化差別應用來發展。若是要發展個人化AI，我們認為需要改進的部分有兩項，其中一項是AI架構必須朝向「雲端服務化」或是「輕量化架構」的方向來做調整；另一項則是訓練資料必須以多種類(例如：影像+距離+環境)、少數量的訓練資料作為蒐集目標，才能解決訓練資料有限的問題。
為了改善物件偵測模型產生誤判結果的問題與探討能否使用多種類、少數量的訓練資料來訓練AI模型，我們透過YOLOv7與「視覺距離演算法(Visual-Distance Algorithm, VDA)」，實作了透過「影像特徵」與「視覺距離」共同決定偵測結果的物件偵測系統，而從實測結果來看，該系統不僅改善了誤判結果的問題，也證明了使用多種類、少數量的訓練資料來訓練AI模型是可行的。此外，為了簡化訓練資料標註的繁瑣流程，我們使用kMaX全景分割模型建立了一套自動標註系統，而經過實測，使用其產生的訓練資料確實可用於物件偵測模型。

With the development of AI technology so far, from YOLOv7 and YOLOv8 in the field of object detection to kMaX and SAM in the field of image segmentation, it can be found that AI has progressed very fast in recent years. Under such an environment, we believe that the development of AI models will gradually develop towards personalized and differentiated applications. If we are going to develop personalized AI, I think there are two parts that need to be improved. One is that the AI architecture must be adjusted in the direction of "cloud service" or "lightweight architecture". The other is that the training data must be collected with the goal of multiple types and small amount to solve the problem of limited training data.
In order to improve the problem of misjudgment results of the object detection model and explore whether it is possible to use multiple types and a small amount of training data to train the AI model. Through YOLOv7 and "Visual-Distance Algorithm(VDA)", we have implemented an object detection system that determines the detection result through "image features" and "visual-distance". From the actual measurement results, the system not only improves the problem of misjudgment results, but also proves that it is possible to use multiple types and a small amount of training data to train the AI model. In addition, in order to simplify the cumbersome process of labeling training data, we use the kMaX panoptic segmentation model to establish an auto labeling system. After actual testing, the training data generated by using it can indeed be used for object detection models.

摘要 ............................................................................................................................................. i
Abstract ....................................................................................................................................... ii
誌謝 ........................................................................................................................................... iii
目錄 ........................................................................................................................................... iv
表目錄 ....................................................................................................................................... vi
圖目錄 ...................................................................................................................................... vii
第 1 章 緒論 .............................................................................................................................. 1
1.1 研究背景 ..................................................................................................................... 1
1.2 研究動機與目的 ......................................................................................................... 1
1.3 論文架構 ..................................................................................................................... 5
第 2 章 文獻回顧 ...................................................................................................................... 6
2.1 全景分割 ..................................................................................................................... 6
2.2 物件偵測 ..................................................................................................................... 6
2.3 kMaX-Deeplab ............................................................................................................. 7
2.4 YOLOv7 ....................................................................................................................... 8
2.5 線性回歸 ..................................................................................................................... 9
2.5.1 均方誤差 ........................................................................................................ 10
2.5.2 最小平方法 .................................................................................................... 10
2.5.3 多項式回歸 .................................................................................................... 10
2.6 評估指標 ................................................................................................................... 11
2.6.1 混淆矩陣 ........................................................................................................ 11
2.6.2 精確率 ............................................................................................................ 12
2.6.3 召回率 ............................................................................................................ 12
2.6.4 Bounding Box .................................................................................................. 12
2.6.5 IoU ................................................................................................................... 13
2.6.6 平均精確度 .................................................................................................... 13
2.6.7 COCO 平均精確率 ......................................................................................... 16
第 3 章 系統架構 .................................................................................................................... 18
3.1 使用軟、硬體 ........................................................................................................... 18
3.2 自動標註 ................................................................................................................... 19
3.2.1 資料擷取 ........................................................................................................ 19
3.2.2 kMaX 全景分割 .............................................................................................. 20
3.2.3 資料集產生器 ................................................................................................ 23
3.3 模型訓練 ................................................................................................................... 24
3.3.1 YOLOv7 實例分割模型 ................................................................................. 24
3.3.2 VDA 模型 ........................................................................................................ 25
3.4 模型部署 ................................................................................................................... 26
3.4.1 距離過濾器 .................................................................................................... 27
3.4.2 結合距離過濾器的物件偵測系統 ................................................................ 27
第 4 章 實驗與結果 ................................................................................................................ 28
4.1 Intel D435i Depth Camera .......................................................................................... 28
4.2 自動標註資料集 ....................................................................................................... 30
4.3 YOLOv7 實例分割模型 ............................................................................................ 32
4.4 VDA 模型 ................................................................................................................... 34
4.5 結合距離過濾器之物件偵測系統 ........................................................................... 35
第 5 章 結論與未來研究 ........................................................................................................ 39
參考文獻 .................................................................................................................................. 40
Extended Abstract ..................................................................................................................... 42

[1] A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, "Improving language understanding by generative pre-training," 2018.
[2] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 779-788.
[3] A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
[4] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, "Hierarchical text-conditional image generation with clip latents," arXiv:2204.06125 [cs.CV], 2022.
[5] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, "High-resolution image synthesis with latent diffusion models," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10684-10695.
[6] A. Kirillov et al., "Segment anything," arXiv:2304.02643[cs.CV], 2023.
[7] OpenAI, "GPT-4 Technical Report," arXiv:2303.08774 [cs.CL], 2023.
[8] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464-7475.
[9] Q. Yu et al., "k-means Mask Transformer," in Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXIX, 2022: Springer, pp. 288-307.
[10] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
[11] A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dollár, "Panoptic segmentation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9404-9413.
[12] 張家銘. "最新的物件偵測王者 YOLOv7 介紹." https://medium.com/ai-academy-taiwan/%E6%9C%80%E6%96%B0%E7%9A%84%E7%89%A9%E4%BB%B6%E5%81%B5%E6%B8%AC%E7%8E%8B%E8%80%85-yolov7-%E4%BB%8B%E7%B4%B9-206c6adf2e69 (accessed June 7th, 2023).
[13] 秋庭伸也、杉山阿聖、寺田學, 零基礎入門的機器學習圖鑑. 采實文化, 2021, pp. 64-72.
[14] 維基百科. "范德蒙矩陣." https://zh.wikipedia.org/zh-tw/%E8%8C%83%E5%BE%B7%E8%92%99%E7%9F%A9%E9%99%A3 (accessed June 7th, 2023).
[15] Kukil. "Mean Average Precision (mAP) in Object Detection." https://learnopencv.com/mean-average-precision-map-object-detection-model-evaluation-metric/ (accessed June 7th, 2023).
[16] T.-Y. Lin et al., "Microsoft coco: Common objects in context," in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, 2014: Springer, pp. 740-755.
[17] Nitfianr. "Python中使用YOLOv7进行实例分割以及Detectron2的使用." https://news.sangniao.com/p/4277861659 (accessed June 7th, 2023).