臺灣博碩士論文加值系統

本論文提出一種動態背景中之多方向移動物切割暨追蹤系統，能夠在攝影裝置於自體運動的拍攝環境下，能夠即時偵測出畫面中的移動物體並框選標示之，可以運用在手持式攝影裝置如智慧型手機、穿戴式運動攝影機，或是移動式載具攝影裝置如汽機車之行車紀錄器、安全監控錄影裝置，或者是嵌入無人駕駛載具系統、各式機器人視覺系統等等，將拍攝畫面中的移動物體進行目標的提取與追蹤，以利於視覺系統的後續處理與應用。本系統主要分為三個部分：(1)目標提取模組：將輸入的序列影像使用雙線性插值法(Bilinear Interpolation)進行縮減取樣的前處理，並使用加速-KAZE(Accelerated-KAZE, A-KAZE)演算法進行特徵點(Feature Point)的偵測與匹配，並利用視角幾何(View Geometry)原理進行特徵點的分類；(2)目標定位模組：將分類好的特徵點群進行個別處理，針對背景特徵點群利用透視變換(Perspective Transformation)以及計算其平均移動向量來還原攝影機自體運動(Ego-motion)造成的額外輪廓，而前景特徵點群則是進行時間域的特徵點累計以增強其穩健性，最後將透視變換之影像與當前的實際影像進行差值運算，進而獲得移動物之初步輪廓；(3)目標追蹤模組：根據目標物輪廓的歷史移動軌跡(Motion History)對其進行形態學處理(Morphological Processing)，並利用最小定界框(Minimum Bounding Box)進行移動目標框選，最後使用卡爾曼濾波器(Kalman Filter)進行目標追蹤，並運用Kuhn–Munkres演算法(Kuhn–Munkres algorithm, Munkres assignment algorithm)或稱為匈牙利演算法(Hungarian algorithm)進行卡爾曼濾波器與移動目標的分配，進而達到多個移動物體的追蹤。

This paper proposes a multi-directional moving object segmentation and tracking system in dynamic background. The system can detect moving targets in a scene by image sequence and bound them in real time from a monocular camera. It can be used in handheld Photographic devices such as smart phones or wearable sports cameras, mobile vehicle photographic devices such as dashboard cameras or security surveillance, and can be embedded in unmanned vehicle systems, smart robot vision systems, etc. Therefore, moving object detection and tracking plays an important role in intelligent surveillance. The main scheme is described as follow: (1) Target extraction module: The input image sequence is subjected to bilinear interpolation for pre-processing of downsampling. Next, the Acceleration-KAZE (A-KAZE) method is used to detect and match feature points in a scene Then, feature points are classified as belonging to background or foreground with the assistance of multiple-view geometry. (2) Target location module: Using perspective transformation with background features to estimate the motion of background regions, and calculate its average motion vector to restore the extra contour caused by the ego-motion of the camera. The foreground features are accumulate in the time domain to enhance its robustness. Finally, the image difference scheme is applied to the transformed frame and the current frame to obtain regions of moving objects. (3) Target tracking module: Morphological processing is performed according to the motion history of the target contours, and use the minimum bounding box to locate the moving objects, apply Kalman filters to every to track objects in the scene and use Hungarian algorithm (Munkres assignment algorithm) for object assignment.

摘要 I
ABSTRACT III
致謝 V
目錄 VI
圖目錄 X
表目錄 XII
第一章 、緒論 1
1.1 研究動機 1
1.2 本系統簡介與架構 2
1.3 論文大綱 3
第二章 、習知處理技術與知識 4
2.1 影像縮減技術 4
2.1.1 最近相鄰內插法(Nearest Neighbor Interpolation) 4
2.1.2 雙線性內插法(Bilinear Interpolation) 5
2.1.3 雙三次內插法(Bicubic Interpolation) 6
2.1.4 影像縮減技術總結 7
2.2 特徵檢測技術 8
2.2.1 Harris角點偵測(Harris Corner Detection) 8
2.2.2 Shi-Tomasi角點偵測(Shi-Tomasi Corner Detection) 11
2.2.3 尺度不變特徵轉換(Scale-Invariant Feature Transform, SIFT) 11
2.2.3.1 尺度空間的極值偵測(Scale-space Extrema Detection) 12
2.2.3.2 關鍵點定位(Keypoint Localization) 13
2.2.3.3 主要特徵方向定位(Orientation Assignment) 14
2.2.3.4 建立關鍵點描述子(Keypoint Descriptor) 15
2.2.4 加速穩健特徵(Speeded Up Robust Feature, SURF) 16
2.2.4.1 特徵點偵測(Interest Point Detection) 16
2.2.4.2 特徵點描述子與匹配(Feature Description And Matching) 19
2.2.5 KAZE演算法 20
2.2.5.1 非線性擴散濾波簡介(Nonlinear Diffusion Filtering) 20
2.2.5.2 建構非線性尺度空間(Computation of the Nonlinear Scale Space) 22
2.2.5.3 特徵點偵測(Feature Detection) 23
2.2.5.4 特徵點描述(Feature Description) 23
2.2.6 加速-KAZE(Accelerated-KAZE, A-KAZE)演算法 24
2.2.6.1 快速顯式擴散簡介(Fast Explicit Diffusion, FED) 24
2.2.6.2 建構非線性尺度空間(Computation of the Nonlinear Scale Space) 26
2.2.6.3 特徵點偵測(Feature Detection) 27
2.2.6.4 特徵點描述(Feature Description) 28
2.2.7 特徵檢測技術總結 29
2.3 群集分析技術 30
2.3.1 Density-based Spatial Clustering of Applications with Noise 30
2.3.2 k-近鄰演算法(k-Nearest Neighbors Algorithm, k-NN) 31
2.3.3 群集分析技術總結 33
2.4 影像校正技術 34
2.4.1 仿射變換(Affine Transform) 34
2.4.2 透視變換(Perspective Transform) 36
2.4.3 單應性矩陣(Homography Matrix) 37
2.4.4 影像校正技術總結 38
2.5 目標追蹤技術 39
2.5.1 卡爾曼濾波器(Kalman Filter) 39
2.5.2 粒子濾波器(Particle Filter) 41
2.5.3 目標追蹤技術總結 42
第三章 、相關文獻探討及介紹 43
3.1 移動物體偵測之文獻 43
3.1.1 Moving object detection, classification and its parametric evaluation 43
3.1.2 Application of graph segmentation method in thermal camera object detection 46
3.1.3 Motion Multi-object Detection Method under Complex Environment 49
3.1.4 Moving object detection and tracking from video captured by moving camera 53
3.1.5 Moving object detection and tracking from moving camera 57
3.1.6 Moving object detection from moving platforms using Lagrange multiplier 61
第四章 、動態背景中之多方向移動物切割暨追蹤系統方法之研究 66
4.1 目標提取模組 68
4.1.1 縮減取樣 68
4.1.2 特徵點搜尋 69
4.1.3 特徵點匹配 71
4.1.4 特徵點分類 73
4.1.4.1 視角幾何(View Geometry) 73
4.1.4.2 基本矩陣求取 75
4.1.4.3 分類前、背景特徵點 77
4.2 目標定位模組 79
4.2.1 前景特徵增強 79
4.2.2 背景影像重建 81
4.2.2.1 前置判斷 81
4.2.2.2 透視變換 82
4.2.2.3 影像差值 84
4.2.2.4 自體運動(Ego-motion)補償 86
4.3 目標追蹤模組 87
4.3.1 歷史移動軌跡 88
4.3.2 形態學處理 90
4.3.3 移動物體追蹤 92
4.3.3.1 移動目標框選 92
4.3.3.2 卡爾曼濾波器 94
4.3.3.3 追蹤目標之標定與分配 96
第五章 、實驗結果 101
5.1 實驗設備與環境 101
5.2 架設環境與測試樣本 101
5.3 實驗評估與分析 102
5.3.1 主觀評估 103
5.3.2 客觀評估 106
5.3.3 整體性比較 107
第六章 、結論與未來方向 110
6.1 結論 110
6.2 未來方向 111
參考文獻 112

[1] C. Harris and M. Stephens, "A combined corner and edge detector," Proceedings of the Fourth Alvey Vision Conference, pp. 147-151, 1988.
[2] J. Shi and C. Tomasi, "Good features to track," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 593-600, June 1994.
[3] D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, vol. 60, pp. 91-110, November 2004.
[4] H. Bay, T. Tuytelaars and L. Van Gool, "SURF: Speeded Up Robust Features," Computer Vision – ECCV 2006, pp. 404-417, 2006.
[5] P. F. Alcantarilla, A. Bartoli and A. J. Davison, "KAZE features," Computer Vision – ECCV 2012, pp. 214-227, 2012.
[6] P. Perona and J. Malik, "Scale-space and edge detection using anisotropic diffusion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, July 1990.
[7] J. Weickert, "Efficient image segmentation using partial differential equations and morphology," Pattern Recognition, vol. 34, no. 9, pp. 1813-1824, 2001.
[8] P. F. Alcantarilla, J.Nuevo and A. Bartoli, "Fast explicit diffusion for accelerated features in nonlinear scale spaces," Proceedings of the British Machine Vision Conference, pp. 13.1-13.11, 2013.
[9] S. Grewenig, J. Weickert and A. Bruhn, "From Box Filtering to Fast Explicit Diffusion," Pattern Recognition, pp. 533-542, 2010.
[10] J. Weickert, S. Grewenig, C. Schroers and A. Bruhn, "Cyclic Schemes for PDE-Based Image Analysis," International Journal of Computer Vision, vol. 118, no. 3, pp. 275-299, July 2016.
[11] X. Yang and K.-T. Cheng, "LDB: An ultra-fast feature for scalable Augmented Reality on mobile devices," IEEE International Symposium on Mixed and Augmented Reality, pp. 49-57, November 2012.
[12] M. Ester, H.-P. Kriegel, J. Sander and X. Xu, "A Density-based Algorithm for Discovering Clusters a Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise," Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 226-231, 1996.
[13] N. Altman, "An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression," The American Statistician, vol. 46, no. 3, pp. 175-185, 1992.
[14] Richard Hartley and Andrew Zisserman, Multiple View Geometry in computer vision, UK: Cambridge University Press, 2003.
[15] R. E. Kálmán, "A New Approach to Linear Filtering and Prediction Problems," ASME - Journal of Basic Engineering, no. 82 (Series D), pp. 35-45, March 1960.
[16] G. Welch and G. Bishop, "An introduction to the Kalman filter," 24 July 2006.
[17] M. S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, "A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174-188, 2002.
[18] R. R. Bharath and G. Dhivya, "Moving object detection, classification and its parametric evaluation," International Conference on Information Communication and Embedded Systems, pp. 1-6, 2014.
[19] H. V. Nguyen and L. H. Tran, "Application of graph segmentation method in thermal camera object detection," 20th International Conference on Methods and Models in Automation and Robotics, pp. 829-833, 2015.
[20] Z.-X. Pan and M. Wang, "Motion multi-object detection method under complex environment," International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration, pp. 87-90, 2016.
[21] W.-C. Hu, C.-H. Chen, T.-Y. Chen, D.-Y. Huang and Z.-C. Wu, "Moving object detection and tracking from video captured by moving camera," Journal of Visual Communication and Image Representation, vol. 30, pp. 164-180, July 2015.
[22] W. J. Kim and I.-S. Kweon, "Moving object detection and tracking from moving camera," 8th International Conference on Ubiquitous Robots and Ambient Intelligence, pp. 758-759, November 2011.
[23] A. ElTantawy and M. S. Shehata, "Moving object detection from moving platforms using Lagrange multiplier," IEEE International Conference on Image Processing, pp. 2586-2590, September 2015.
[24] Y. Peng, A. Ganesh, J. Wright, W. Xu and Y. Ma, "RASL: Robust Alignment by Sparse and Low-rank decomposition for Linearly correlated Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2233-2246, November 2012.
[25] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," Proceedings of Imaging Understanding Workshop, pp. 121-130, 1981.
[26] M. Muja and D. G. Lowe, "Fast approximate nearest neighbors with automatic algorithm configuration," International Conference on Computer Vision Theory and Applications, pp. 331-340, 2009.
[27] M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol. 24, pp. 381-395, June 1981.
[28] J. Munkres, "Algorithms for the Assignment and Transportation Problems," Journal of the Society for Industrial and Applied Mathematics, vol. 5, no. 1, pp. 32-38, March 1957.
[29] H. W. Kuhn, "The Hungarian method for the assignment problem," Naval Research Logistics Quarterly, vol. 2, pp. 83-97, 1955.
[30] H. W. Kuhn, "Variants of the hungarian method for assignment problems," Naval Research Logistics Quarterly, vol. 3, no. 4, pp. 253-258, 1956.