臺灣博碩士論文加值系統

商船無人化為目前航運業的趨勢，其中船舶影像辨識之相關研究，多半集中在避碰。而本研究著重在於針對船舶上的特徵圖形辨識，提供特殊需求如拖船作業，船身與其他船舶接觸或與特定位置接近之海事作業。辨識目標為標塗於船殼外側之各種船舷標誌，有橫向結構位置較耐頂推之拖船標誌、提醒其他船舶滿吃水時處於水線下球形艏(Bulbous bow)位置之艏形標誌、側推器(Side thruster)位置之標誌。

本論文為研究即時影像辨識應用於船舶船舷標誌。資料集取材自進出高雄港第二港口之船舶，於實驗室內商用型電腦架設之影像辨識訓練演算法訓練權重，並移植至攜帶式裝置較常見之ARM架構電腦搭配深度視覺攝影機，即時辨識目標並加上其目視距離訊息，並利用圖形處理器加速演算法。在樣本總數相同的情況下，調整訓練資料集、驗證資料集的比例得出的結果中，驗證資料集佔比30%有較高的成功辨識機率。

The trend of the current shipping industry is to make merchant ships humanless, and most of the research on ship image recognition is focused on collision avoidance. This study focuses on the identification of features on ships, providing special needs such as tugboat operations, hull contact with other ships or maritime operations close to specific locations. The identification targets are various ship side markings painted on the outside of the hull, tug markings that are more resistant to thrusting in the horizontal structure, bow markings that alert other ships to the position of the bulbous bow below the waterline at a full draught, and markings of the side thruster position.

This thesis is to study the application of real-time image recognition to ship side markings. The dataset was took when the ships entering and leaving the second port of Kaohsiung port. The image recognition training algorithm was installed on a commercial computer in the laboratory to train the weights and transported to an ARM-based computer, which is more common in portable devices. With a depth vision camera to recognize the targets and add their visual distance information in real time, and to accelerate the algorithm using a graphics processor. In the case of the same sample size, by adjusting the ratio of training dataset and validation dataset. The best result rate of the validation dataset is at 30%.

摘要 I
ABSTRACT II
誌謝 III
目錄 IV
圖目錄 VI
表目錄 XI
符號說明 XIII
第一章 緒論 1
1.1 研究動機 1
1.2 文獻回顧 2
1.3 論文架構 3
第二章 技術概要 4
2.1 電腦視覺 - COMPUTER VISION 4
2.1.1 感光元件 - Image Sensor 4
2.1.2色彩模型 - Color Model 5
2.2 影像辨識 – YOLO演算法 6
2.2.1影像辨識預訓練模型 - YOLOv4 tiny 7
2.2.2權重分類能力指標 10
2.3 圖形處理需要之電腦運算 14
2.3.1 中央處理單元 - Central Processing Unit 14
2.3.2 圖形處理單元 - Graphics Processing Unit 17
第三章 硬體架構 20
3.1開發平台 – NVIDIA® JETSON TX2® 21
3.2深度視覺攝影機 – ZED® CAMERA 22
第四章 實驗設計與實現 23
4.1 開發環境建立 24
4.1.1 作業系統 - Ubuntu® 18.04 LTS 25
4.1.2 平行運算開發環境 - CUDA® Toolkit 27
4.1.3 影像處理資料庫 - OpenCV® 29
4.1.4 程式語言開發環境 - Python® 30
4.1.5 深度攝影機開發套件 - ZED® SDK 30
4.1.6 影像辨識演算法框架 - Darknet 31
4.2 訓練影像辨識船舷標誌之流程 33
4.2.1蒐集船舶進出港口影像 35
4.2.2標記船舷標誌影像 39
4.2.3訓練權重模型 41
第五章 結果分析與討論 45
5.1驗證資料集不同佔比下之訓練結果 46
5.1.1權重訓練過程 46
5.1.2測試資料集辨識結果 55
5.1.3船舶圖片辨識結果(1) - 驗證資料集不同佔比下之結果 75
5.2不同平台辨識測試資料集之結果 93
5.2.1船舶圖片辨識結果(2) - 不同平台下之辨識結果 94
5.3即時影像辨識結果(附加深度資訊) 107
5.3.1驗證資料集佔比20%之訓練結果 108
5.3.2驗證資料集佔比30%之訓練結果 112
5.3.3驗證資料集佔比40%之訓練結果 120
第六章 結論與建議 128
參考文獻 130
附錄一 –NVIDIA® JETSON TX2® 詳細規格 134

[1]S. Ren, K. He, R. Girshick, J. Sun, 2017, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, IEEE Trans Pattern Anal Mach Intell, Vol.39, No.6, pp.1137-1149.
[2]Zhi Tian, Chunhua Shen, Hao Chen, Tong He, 2019, “FCOS: Fully Convolutional One-Stage Object Detection”, 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South).
[3]Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao, 2020, “YOLOv4: Optimal Speed and Accuracy of Object Detection”, ArXiv, abs/2004.10934.
[4]Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao, 2020, “Scaled-YOLOv4: Scaling Cross Stage Partial Network”, ArXiv, abs/2011.08036v2.
[5]Chien-Yao Wang, Hong-Yuan Mark Liao, Yueh-Hua Wu, Ping-Yang Chen, Jun-Wei Hsieh, I. Hau Yeh, 2020, “CSPNet: A New Backbone that can Enhance Learning Capability of CNN”, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
[6]Joseph Redmon, Ali Farhadi, 2018, “YOLOv3: An Incremental Improvement”, ArXiv, abs/1804.02767.
[7]Joseph Redmon, Ali Farhadi, 2017, “YOLO9000: Better, Faster, Stronger”, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. https://arxiv.org/abs/1612.08242
[8]Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, 2016, “You Only Look Once: Unified, Real-Time Object Detection”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://arxiv.org/abs/1506.02640
[9]H. Z. Mao, S. Yao, T. Q. Tang, B. X. Li, J. Yao, Y. Wang, 2018, “Towards Real-Time Object Detection on Embedded Systems”, Ieee Transactions on Emerging Topics in Computing, Vol.6, No.3, pp.417-431, doi:10.1109/Tetc.2016.2593643
[10]杜柏宏, 2019, 訓練資料集自我擴增問題之分析-以 YOLO 分類器為例 , 國立中山大學, 碩士論文。
[11]Zicong Jiang, Liquan Zhao, Shuaiyang Li, Yanfei Jia, 2020, “Real-time object detection method for embedded devices”, ArXiv, abs/2011.04244.
 
[12]Zhang Qiang, Wang Yuanyu, Zhu Liang, Zhang Jin, Lin Yu, Lin Dandan, 2021, “Research on Real-Time Reasoning Based on JetSon TX2 Heterogeneous Acceleration YOLOv4”, 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA).
[13]Huibai Wang, Hao Yu, 2020, “Traffic Sign Detection Algorithm based on improved YOLOv4”, 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC).
[14]莫雅婷, 2020, 應用 Tensorflow 深度學習機制於影像辨識之研究— 以中共軍機影像辨識為例 , 國防大學 ,碩士論文。
[15]Yingkun Yang, Guorong Xie, Yi Qu, 2021, “Real-time Detection of Aircraft Objects in Remote Sensing Images Based on Improved YOLOv4”, 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC).
[16]鍾曜鴻, 2021, 探討 YOLO 神經網路的效能與應用 , 國立雲林科技大學, 碩士論文。
[17]Cecie Starr, Christine A. Evers, Lisa Starr, 2016, Biology: Concepts & Applications, Tenth ed, Cengage Learning, Canada.
[18]John D. Cressler, 2016, Silicon Earth - Introduction to Microelectronics and Nanotechnology, 2th ed., Taylor & Francis Group, LLC, Boca Raton, FL, USA.
[19]John Lester Miller, 1994, Principles of Infrared Technology, Van Nostrand Reinhold, NY, USA.
[20]E. R. Fossum, D. B. Hondongwa, 2014, “A Review of the Pinned Photodiode for CCD and CMOS Image Sensors”, Ieee Journal of the Electron Devices Society, Vol.2, No.3, pp.33-43, doi:10.1109/Jeds.2014.2306412
[21]Teledyne DALSA, 2011, “The Evolution of CMOS Imaging Technology”.
[22]DALSA Corporation, 2007, “Applications Set Imager Choices”.
[23]Michael E. Miller, 2019, Color in Electronic Display Systems, Springer Nature Switzerland AG, Cham, Switzerland.
[24]Rastislav Lukac, 2011, Computational Photography: Methods and Applications, Rastislav Lukac Ed., CRC Press, NY, USA.
[25]Bryce E. Bayer, 1976, “COLOR IMAGING ARRAY”, United States Patent No. 3971065, United States Patent Office.
[26]Jyotismita Chaki, Nilanjan Dey, 2021, Image Color Feature Extraction Techniques, Springer Nature Singapore Pte Ltd., Singapore.
[27]Helmut Kipphan, 2001, Handbook of Print Media, Helmut Kipphan Ed., Springer -Verlag Berlin Heidelberg, Berlin, Germany.
[28]Saurabh Kapur, 2017, Computer Vision with Python 3, Saurabh Kapur Ed., Packt Publishing Ltd., Birmingham, UK.
[29]Alberto Fernández Villán, 2019, “Machine Learning and Deep Learning in OpenCV”, Mastering OpenCV 4 with Python, Packt Publishing Ltd., Birmingham, UK.
[30]Joseph Howse, Prateek Joshi, Michael Beyeler, 2016, OpenCV: Computer Vision Projects with Python, Packt Publishing Ltd., Birmingham, UK.
[31]AlexeyAB, 2021, “Darknet”. https://github.com/AlexeyAB/darknet
[32]Tingting Liang, Yongtao Wang, Guosheng Hu, Zhi Tang, Haibin Ling, 2021, “OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection”, ArXiv, abs/2103.04507.
[33]Youngwan Lee, Joong-won Hwang, Sangrok Lee, Yuseok Bae, Jongyoul Park, 2019, “An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection”, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
[34]Jesse Davis, Mark Goadrich, 2006, “The relationship between Precision-Recall and ROC curves”, 23rd international conference on Machine learning - ICML '06.
[35]Alaa Tharwat, 2020, “Classification assessment methods”, Applied Computing and Informatics, Vol.17, No.1, pp.168-192. doi:10.1016/j.aci.2018.08.003
[36]A. Linden, 2006, “Measuring diagnostic and predictive accuracy in disease management: an introduction to receiver operating characteristic (ROC) analysis”, Journal of Evaluation in Clinical Practice, Vol.12, No.2, pp.132-139. doi:10.1111/j.1365-2753.2005.00598.x
[37]T. Fawcett, 2006, “An introduction to ROC analysis”, Pattern Recognition Letters, Vol.27, No.8, pp.861-874. doi:10.1016/j.patrec.2005.10.010
[38]David M W Powers, 2008, “Evaluation: From Precision, Recall and F-Factor to ROC, Informedness, Markedness & Correlation”, Technical Report, Flinders University, Adelaide, Australia.
[39]M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, A. Zisserman, 2010, “The Pascal Visual Object Classes (VOC) Challenge”, International Journal of Computer Vision, Vol.88, No.2, pp.303-338. doi:10.1007/s11263-009-0275-4
[40]Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick, 2017, “Mask R-CNN”, 2017 IEEE International Conference on Computer Vision (ICCV).
[41]Ze Yang, Shaohui Liu, Han Hu, Liwei Wang, Stephen Lin, 2019, “RepPoints: Point Set Representation for Object Detection”, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).
[42]David J. Kuck, 1978, The Structure of Computers and Computations, Vol. 1, John Wiley & Sons, Inc., NY, USA.
[43]NVIDIA Corporation, 2021, “CUDA C++ Programming Guide”, San Tomas Expressway, Santa Clara, CA, USA.
[44]John von Neumann, 1945, “First Draft of a Report on EDVAC”.
[45]NVIDIA Corporation, 2020, “NVIDIA CUDA Installation Guide for Linux”, San Tomas Expressway, Santa Clara, CA, USA.
[46]NVIDIA Corporation, 2021, “CUDA Compatibility”, San Tomas Expressway, Santa Clara, CA, USA.
[47]NVIDIA Corporation, 2021, “cuDNN Developer Guide”, San Tomas Expressway, Santa Clara, CA, USA.
[48]NVIDIA Corporation, 2021, “Jetson TX2 Module”. https://developer.nvidia.com/embedded/jetson-tx2
[49]NVIDIA Corporation, 2021, “NVIDIA Jetson TX2 Developer Kit including module, reference carrier, and camera module”. https://developer.nvidia.com/blog/jetson-tx2-delivers-twice-intelligence-edge/
[50]Stereolabs, “ZED Stereo Camera”, 2021. https://www.stereolabs.com/zed/
[51]NVIDIA Corporation, 2019, “CUDA Runtime API”, San Tomas Expressway, Santa Clara, CA, USA.