臺灣博碩士論文加值系統

影像目標追蹤近年來已是相當熱門的研究主題，但是在特定情況下例如：天氣，光線等因素會嚴重影響目標追蹤的效果，而RGB彩色影像和熱紅外影像能透過特徵互補來提升目標在惡劣環境下的追蹤效果。
近年來RGB-T目標追蹤已成為熱門的研究主題也誕生許多演算法及網路模型，我們分析主要的RGB-T追蹤文獻並總結後提出一個基於Siamese網路框架進行特徵融合的 RGB-T 單目標追蹤網路模型，該模型將熱紅外影像及 RGB 彩色影像中，各自的特徵進行提取，並進行特徵融合，藉此提高單目標追蹤能力。本研究採用 SENet及 CBAM來對影像圖進行特徵強化，並融合進RGB-T 特徵圖像中，進而提升單目標跟蹤的追蹤效果。

Visual object tracking has been attracted recently. But, In some special conditions like: weather and sunlight which has a major effect on object tracking. RGB visible and infrared images can provide complementary features to boost the tracking performance under adverse conditions.
Recently, RGB-T object tracking has become a popular research topic and various RGB-T tracking modules. We analyze the major RGB-T trackers in literature and categorize them. We present a RGB-T single object tracking model based on feature fusion by Siamese networks. This model extracts thermal infrared image features and RGB image features and then combine both features to enhance the tracking capability. In addition, we employ SENet and CBAM for enhance image features to promote the tracking effective.

摘要 I
ABSTRACT II
目錄 IV
圖目錄 VI
表目錄 VII
第一章 緒論 1
1.1 研究動機 1
1.2 研究方法 1
1.3 論文架構 2
第二章 文獻探討 3
2.1 Siamese Network目標追蹤 3
2.1.1 SiamFC 3
2.1.2 SiamRPN 4
2.1.3 基於Siamese Networks應用於RGB-T目標追蹤 5
2.2 MDNet 5
2.3 mfDiMP 7
2.4 RGB-T資料集 8
第三章 研究方法 11
3.1 系統架構與流程圖 11
3.1.1 Siamese網路結構 12
3.1.2 SENet 12
3.1.3 CBAM 14
3.1.4 FCOS 15
3.2 資料集 16
3.2.1 GOT-10K資料集 16
3.2.2 3.2.1 RGBT234 17
第四章 實驗結果 19
4.1 實驗環境 19
4.2 評估方法 19
4.2.1 評估資料集 19
4.2.2 評估指標 20
4.3 實驗結果 20
4.3.1 GTOT資料集上測試數據結果 20
4.3.2 Occlusion 21
4.3.3 Large scale variation 23
4.3.4 Fast motion 23
4.3.5 Low illumination 24
4.3.6 Thermal crossover 25
4.3.7 Small object 26
4.3.8 Deformation 27
第五章 結論 29
參考文獻 30

[1] Davide Chicco. Siamese neural networks: An overview. Artificial Neural Networks, 2190:73–94, 2021.
[2] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. Fully-convolutional siamese networks for object tracking. In European conference on computer vision, pages 850–865. Springer, 2016.
[3] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. High performance visual tracking with siamese region proposal network. In CVPR, 2018.
[4] Z. Tang, T. Xu, and X.-J. Wu, “A survey for deep RGBT tracking,” arXiv cs.CV 2201.09296 (2022).
[5] H. Nam and B. Han, “Learning multi-domain convolutional neural networks for visual tracking,” in Proc. IEEE CVPR, 2016, pp. 4293–4302.
[6] L. Zhang, M. Danelljan, A. Gonzalez-Garcia, J. van de Weijer, and F. S. Khan. Multi-modal fusion for end-to-end rgb-t tracking. In IEEE International Conference on Computer Vision, ICCV Workshops, 2019. [7] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning discriminative model prediction for tracking. In ICCV, 2019.
[8] C. Li, H. Cheng, S. Hu, X. Liu, J. Tang, L. Lin, Learning collaborative sparse representation for grayscale-thermal tracking, IEEE Transactions on Image Processing 25 (2016) 5743–5756.
[9] Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, and Jin Tang. RGB-T object tracking: benchmark and baseline. Pattern Recognition, 96:106977, 2019.
[10] C. Li, W. Xue, Y. Jia, Z. Qu, B. Luo, J. Tang, D. Sun, Lasher: A large-scale high-diversity benchmark for RGBT tracking, IEEE Transactions on Image Processing 31 (2022) 392–404.
[11] N. Jiang, K. Wang, X. Peng, X. Yu, Q. Wang, J. Xing, G. Li, J. Zhao, G. Guo, and Z. Han, “Anti-uav: A large multi-modal benchmark for UAV tracking,” CoRR, vol. abs/2101.08466, 2021.
[12] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In CVPR, 2018.
[13] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018.
[14] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. FCOS: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9627–9636, 2019.
[15] Dongyan Guo, Jun Wang, Ying Cui, Zhenhua Wang, and Shengyong Chen. SiamCAR: Siamese fully convolutional classification and regression for visual tracking. In CVPR, 2020.
[16] Lianghua Huang, Xin Zhao, and Kaiqi Huang. GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1562–1577, May 2021.
[17] B. Babenko, M.-H. Yang, and S. Belongie, “Robust object tracking with online multiple instance learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1619–1632, Aug. 2011.
[18] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, “High-speed tracking with kernelized correlation filters,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 583–596, Mar. 2015.
[19] Y. Wu, J. Lim, and M.-H. Yang, “Online object tracking: A benchmark,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2013, pp. 2411–2418 .
[20] Chenglong Li, Andong Lu, Aihua Zheng, Zhengzheng Tu, and Jin Tang. Multi-adapter RGBT tracking. In IEEE International Conference on Computer Vision Workshop, pages 2262–2270, 2019.
[21] Tianlu Zhang, Xueru Liu, Qiang Zhang, and Jungong Han. SiamCDA: Complementarity- and distractor-aware RGB-T tracking based on siamese network. IEEE Transactions on Circuits and Systems for Video Technology, 99:1–16, 2021.
[22] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan. SiamRPN++: Evolution of siamese visual tracking with very deep networks. In CVPR, 2019.