本論文提出一種適用於快速大幅晃動畫面之視訊穩定系統，主要是針對穿戴式攝影機於奔跑或追逐的情況下所產生的大晃動，來對畫面進行視訊穩定處理以得到穩定的畫面輸出。在快速奔跑或追逐的攝影環境下，背景更新速度快，背景複雜度大增，且晃動複雜無規律，再加上因大晃動的關係，導致有許多模糊的畫面，使得畫面穩定處理的技術難度增加了許多。
本論文方法主要包含四個部分: (1) 首先利用相鄰兩張畫面經前處理後，從兩張畫面中分別找出特徵點的位置，在與兩畫面中的特徵點(Feature Points)來進行特徵點匹配，接著透過最近鄰距離比來濾除掉錯誤的匹配，並採用自適應的方式，使匹配的數量不會過少，以取得特徵點移動向量(Motion Vector)。(2)以仿射變換(Affine Transform)計算相鄰兩張影像之仿射變換矩陣(Affine Transform Martix)，接著對目標圖像進行仿射變換(Affine Transform)使前後兩張畫面相似。(3) 透過計算每一幀所得到的仿射變換矩陣來計算出畫面的偏轉角度及畫面位移，並累加每一幀變換以獲得圖像的晃動向量軌跡，接著再使用卡爾曼濾波器(Kalman Filter)來對畫面晃動向量(偏轉角度與畫面位移量)進行運動平滑化，使得晃動向量軌跡能與平滑過後的晃動向量軌跡相同。(4) 最後將平滑過後的偏轉角度與位移來對畫面進行反向偏轉補償與反向位移補償，接著再使用雙邊濾波器(Bilateral Filter)來消除模糊的畫面，使大晃動畫面達到穩定的效果。
本系統設計主要是應用於穿戴式攝影機，例如GoPro、SJCAM、Body、DrivePro…等，也可以用在各式無人載具之視覺系統，例如機器人視覺系統、無人駕駛系統、車載視覺系統…等，但因為在市面上，三軸穩定器價格昂貴的關係，因此，本系統可以在不增加額外硬體設備的情況下來達到畫面穩定的效果，也可以搭配三軸穩定器，使畫面穩定的效果更好，以提高相關產品的價值及競爭力。

This paper proposes a real-time video stabilization system for violently shaking, mainly focuses on the large shaking generated by wearable cameras when chasing or running . In order to get stable picture output through the image to the video stability processing. In the fast running or chase photographic environment, the background update speed is fast, the background complexity is greatly increased, and the shaking is complex and irregular. In addition, due to the large swing relationship, there are many fuzzy screens, making the picture stability processing technology more difficult.
The method of this thesis mainly consists of four parts: (1) Firstly, the two adjacent pictures are pre-processed, and the position of the feature points is found from the two pictures respectively, and the feature points in the two pictures are Perform feature point matching, then filter out the wrong match by the nearest neighbor distance ratio, and adopt adaptive method so that the number of matches is not too small to obtain the motion vectors. (2) Compute the affine transform matrix of the two adjacent pictures and perform the affine transform of the target image to enhance the similarity. (3) Calculate the deflection angle and the screen displacement of the image by calculating the affine transformation matrix obtained for each frame, and accumulate each frame transformation to obtain the sway vector trajectory of the image, and then use the Kalman filter to smooth the deflection angle of the picture and the screen displacement, so that the sway vector trajectory can be the same as the smoothed sway vector trajectory. (4) Finally, the smoothed deflection angle and displacement are used to reverse the deflection compensation and reverse displacement compensation, and then the Bilateral filter is used to eliminate the blurred image, so that the large animation surface achieves a stable effect.
This proposed system is mainly designed for the wearable camera such as GoPro, SJCAM, Body, Drivepro, etc., and can also be used in the unmanned vehicle of vision systems, such as robot vision systems, unmanned system, automotive vision systems, etc.because in the market for three-axis stabilized is expensive relations, so that the system can not add additional hardware devices down to screen the effect of stability can also be used with three-axis stabilized for screen gives a better stability in order to improve the value of the related products and competitiveness.

摘要 I
ABSTRACT III
致謝 V
目錄 VI
圖目錄 IX
表目錄 XI
第一章 緒論 1
1.1 研究動機 1
1.2 系統架構與流程 2
1.3 論文大綱 4
第二章 習知相關技術 5
2.1 縮減取樣 (DOWN SAMPLING) 5
2.1.1 最近相鄰內插法 (Nearest Neighbor Interpolation) 5
2.1.2雙線性插值法 (Bilinear Interpolation) 7
2.1.3 雙三次插值法 (Bicubic Interpolation) 9
2.2 特徵偵測 (FEATURE DETECTION) 11
2.2.1 線特徵 11
2.2.2 點特徵 14
2.3 變換矩陣(TRANSFORMATION MATRIX) 29
2.3.1 仿射變換 (Affine Transform) 29
2.3.2 透視變換 (Perspective Projection) 30
2.3.3 單應性矩陣(Homography Matrix) 31
2.3.4 變換矩陣總結 32
2.4 分群(CLUSTERING) 33
2.4.1 k-Means 33
2.4.2 Density-Based Spatial Clustering of Applications with Noise 34
2.4.3 K-NearestNeighbor Classification Algorithm 34
2.4.4 分群總結 35
2.5 運動平滑化(MOTION SMOOTHING) 35
2.5.1 卡爾曼濾波器(Kalman Filter) 35
2.5.2 粒子濾波器(Particle Filter) 36
2.5.3 運動平滑化總結 37
2.6 平滑濾波器(SMOOTHING FILTER) 37
2.6.1 高斯濾波器(Gaussian Filter) 37
2.6.2 中值濾波器(Median Filter) 38
2.6.3 雙邊濾波器(Bilateral Filter) 38
2.6.4 平滑濾波器總結 39
第三章 相關文獻探討 40
3.1 視訊穩定相關文獻 40
3.1.1 Video Stabilization for Fast Moving Camera Based on Feature Point Classification 41
3.1.2 運動型攝影之畫面穩定系統 44
3.1.3 Video Stabilization using Speedeed Up Robust Features 47
3.1.4 Moving Camera Video Stabilization Using Homography Consistency 49
3.1.5 Real-Time Video Stabilization Based On Fast Block Matching And Improved Kalman Filter 51
3.1.6 Translation and rotation invariant video stabilization for real time applications 54
3.1.7 A Hybrid Filtering Approach of Digital Video Stabilization for UAV Using Kalman and Low Pass Filter 57
第四章 快速大幅晃動畫面之即時視訊穩定系統 58
4.1 影像前處理 60
4.1.1 等比例縮減取樣 60
4.2 特徵點分析 61
4.2.1 ORB-LATCH特徵點偵測 61
4.2.2 KNN特徵點匹配 65
4.3 仿射變換矩陣匹配 70
4.3.1 計算仿射變換矩陣 70
4.3.2 仿射變換 72
4.4 角度與位移計算 74
4.4.1 計算畫面偏轉角度與畫面位移 74
4.5 運動平滑化 79
4.5.1 Kalman Filter 79
4.6 補償輸出 83
4.6.1 反向偏轉補償與反向位移補償 83
4.6.2 雙邊濾波器 88
第五章 實驗結果 90
5.1 架設環境與測試影片 90
5.2 輸出結果畫面 91
5.3 畫面間變換保真度比較 95
5.4 與習知方法之畫面間變換保真度比較 103
5.5 整體性比較 107
5.6 錯誤探討 109
第六章 結論與未來方向 111
6.1 結論 111
6.2 未來方向 112
參考文獻 114

[1] I. Sobel and G. Feldman, "A 3x3 isotropic gradient operator for image processing," presented at a talk at the Stanford Artificial Project, 1968.
[2] J. M. S.Prewitt, "Object enhancement and extraction," in Picture Processing and Psychopictorics, 1970, pp. 75-149.
[3] Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool, "Surf: Speeded up Robust Features," Computer vision-ECCV, Graz, Austria ,, pp. 404-417, May 2006.
[4] D.Lowe, "Distinctive Image Feature From Scale-Invariant Kwypoints," International journal of computer vision, vol. 60, no. 2, pp. 91-110, 2004.
[5] Rublee, E, Rabaud, V, Konolige, K, and Bradski, G., "ORB: An Efficient Alternative to SIFT or SURF," Computer Vision (ICCV), 2011 IEEE International conference,Barcelona, Spain, Nov 2011.
[6] E. Rosten and T. Drummond, "Machine learning for highspeed corner detection," In European Conference on Computer Vision, vol. I, 2006.
[7] M. Calonder, V.Lepetit, C. Strecha and P. Fua, "Brief: Binary robust independent elementary features," Proceedings of the 11th European Conference on Computer Vision, Crete, Greece, Part IV, pp. 778-792, Sep 2010.
[8] G. Levi and T. Hassner, "LATCH: Learned arrangements of three patch codes," 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1-9, 3 2016.
[9] Richard Hartley and Andrew Zisserman, "Multiple View Geometry in computer vision," Cambridge University Press, 2003.
[10] J. B. MacQueen, "Some Methods for Classification and Analysis of Multivariate Observations," Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1 (University of California Press, pp. 281-297, 1967.
[11] M. Ester, H. P. Kriegel, J. Sander and X. Xu, "A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise," Proceedingd of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), pp. 226-231, 1996.
[12] Cover TM and Hart PE, "Nearest neighbor pattern classification," IEEE Transactions on Information Theory, vol. 13, pp. 21-27, Jan 1967.
[13] B. Gary and G. Welch, "An introduction to the kalman filter," Proc of SIGGRAPH, Course, 2001.
[14] Pitt, Michael K and Neil Shephard., "Filtering via simulation: Auxiliary particle filters," Journal of the American statistical association, vol. 94, pp. 590-599, 1999.
[15] C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), pp. 839-846, Jan 1998.
[16] Wu-Chih Hu,Chao-Ho Chen,Tsong-Yi Chen, Min-Yang Peng, and Yi-Jen Su, "Real-Time Video Stabilization for Fast-Moving Vehicle Cameras," Multimedia Tools and Applications (MTAP), vol. 77, pp. 1237-1260, Jan 2018.
[17] 陳昭和,陳聰毅,楊炎萍,黃登淵,文成康, “運動型攝影之畫面穩定系統,” 第17屆離島資訊技術與應用研討會(ITAOI-2018), pp. 848-853, 25-27 May 2018.
[18] B. Pinto and P. R. Anurenjan, "Video stabilization using speeded up robust features," Proc. IEEE Int. Conf. Commun. Sigmal Process, pp. 527-531, Feb 2011.
[19] Y. F. Hsu, C. C. Chou and M. Y. Shih, "Moving camera video stabilization using homography consistency," IEEE International Conference on Image Processing (ICIP), pp. 2761-2764, 2012.
[20] L. Li, X. Ma and Z. Zhao,, "Real-time video stabilization based on fast block matching and improved Kalman filter," Proc. IEEE ICICIP, pp. 394-397, Aug 2014.
[21] B. Sultan , J. Ahmed , A. Jalil , H. Nazir , M. S. U. A. Abbasi , J. Shah , A. Ali and H. Ali, "Translation and rotation invariant video stabilization for real time applications," 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 479-484, 2017.
[22] Lakshya Kejriwal and Indu Singh, "A Hybrid filtering approach of Digital Video Stabilization for UAV using Kalman and Low Pass filter," 6th International Conference On Advances In Computing & Communications, ICACC, pp. 359-366, 9 2016.