臺灣博碩士論文加值系統

視訊修補是用來移除視訊中所選擇的物件或重建影像中受損的區域，主要的挑戰是使修復後的視訊影片與原始影片有著相同的視覺品質。視訊修補必須使用強健的物件追蹤演算法，還要考慮在畫面(frame)之間的時間連續性，尤其是在攝影機移動的情況下，其考慮的因素更多。所以近年來，它已受到許多研究者的關注。本論文提出一個基於模範(Exemplar-based)方法為基礎的視訊俢補技術，首先利用哈里斯角點偵測(Harris corner detection)提取出所有畫面的特徵點，並使用仿射轉換(Affine transformation)的特性將所有的畫面疊合起來，再利用中值法建立了動態背景的全景影像(mosaic)。對於移動中的物件使用區塊匹配演算法的概念提出了一套強健的物件追蹤方法，再利用背景減法移除所有畫面的前景物件。最後使用基於模範的方法進行視訊修補。實驗證明，我們提出來的方法，不僅可以利用在動態的背景上，更可以準確的追蹤移除物件，在修補過程也能避免掉影像中線性結構的遺失，不會產生模糊，因此，修補後的視訊可以達到一定的品質。

Video inpainting is a technique to remove selected objects or reconstruct damaged area of a video automatically. The main challenge is the restored sequence should have the same visual quality as the original. It not only needs explore a robust object tracking algorithm, but also need take into account the temporal continuity among video frames, especially in the case that the video has camera motions. Hence, video inpainting has drawn much attention in the past few years. This paper proposed a video inpainting scheme based on exemplar-based method. Firstly, the Harris corner detection is used to extract the feature of the frames and Affine transformation is applied to stitch all frames up, and then the median method is used to establish a dynamic background mosaic image. After that, a new robust object tracking method is proposed for moving object using the concept of optical flow. Next, background subtraction is adopted to remove the foreground moving object among all fames. Finally, the video is repaired by the exemplar-based approach. Experimental results demonstrate the proposed method not only can be using in the dynamic background case, but also accurately tracked at remove selected objects. The inpainting process can avoid producing blur when the linear structure in image is missed. Thus, the repaired video can achieve a certain degree quality.

第一章 緒論 1
1.1 研究背景 1
1.2 研究動機與目的 3
1.3 論文架構 4
第二章 相關文獻 5
2.1 數位修補論文探討 5
2.1.1 影像修補相關論文 6
2.1.2 視訊修補相關論文 16
2.2 背景知識 26
2.2.1 仿射轉換(Affine transformation) 26
2.2.2 哈里斯角點偵測(Harris corner detection) 29
第三章 研究方法 35
3.1 建立背景模型 37
3.1.1 特徵點的提取 37
3.1.2 背景縫合 39
3.2 目標識別 41
3.2.1 物件追蹤 41
3.2.2 背景減法 43
3.3 視訊修補 44
3.3.1 建立來源區域影像 45
3.2.2 基於模範方法修補 47
第四章 實驗結果 49
第五章 結論與未來工作 55
參考文獻 56

圖目錄
圖1.1 刮痕的修補 ............................................................................................... 2
圖1.2 物件的移除 ............................................................................................... 2
圖1.3 文字的移除 ............................................................................................... 2
圖2.1 修補的結果 ............................................................................................... 7
圖2.2 簡略演算法以及擴散核心權重示意圖 ................................................... 9
圖2.3 基於模範之影像修補流程圖 ................................................................. 10
圖2.4 兩個不同修補方法的差異 ..................................................................... 11
圖2.5 基於模範之影像修補符號表 ................................................................. 12
圖2.6 初始化目標區塊的可信任程度值 ......................................................... 13
圖2.7 計算目標區塊的結構程度 ..................................................................... 14
圖2.8 目標區塊的可信任程度值及目標區塊的結構程度 ............................. 14
圖2.9 修補流程示意圖 ..................................................................................... 16
圖2.10 背景以及前景全景影像 ....................................................................... 17
圖2.11 前景修補示意圖 ................................................................................... 18
圖2.12 將所有移動的物件對齊建立出全景影像 ........................................... 20
圖2.13 前景修補過程示意圖 ........................................................................... 21
圖2.14 動作向量圖的建立 ............................................................................... 22
圖2.15 老舊影片修補過程圖 ........................................................................... 25
圖2.16 旋轉90 度示意圖 ................................................................................. 27
圖2.17 放大5 倍示意圖 ................................................................................... 28
圖2.18 平移(4,1)示意圖 ................................................................................... 29
圖2.19 搜尋視窗在影像中移動示意圖 ........................................................... 30
圖2.20 哈里斯角點偵側方法示意圖 ............................................................... 33
圖3.1 系統架構圖 ............................................................................................. 37
圖3.2 利用哈里斯角點偵測法找出的特徵點(特徵點以圓圈表示) .............. 39
圖3.3 利用交互相關性所計算出來的特徵點 ................................................. 40
圖3.4 建立背景模型示意圖[1] ......................................................................... 40
圖3.5 動態背景縫合示意圖[8] ........................................................................ 41
圖3.6 相鄰兩個畫面之間區塊的移動向量圖 .................................................. 43
圖3.7 物件追蹤示意圖 ..................................................................................... 44
圖3.9 原始 ____大小為w_h，我們將____放大成2w_2h .......................... 47
圖3.10 建立來源區域影像示意圖 ................................................................... 47
圖4.1 修補Tennis 的實驗圖 ............................................................................ 52
圖4.2 修補卡通影片的實驗圖 ......................................................................... 53
圖4.3 修補冒險島影片的實驗圖 ...................................................................... 54
圖4.4 修補行人走路影片的實驗圖 .................................................................. 55

[1]A. Colombari, A. Fusiello, and V. Murino(2007), “Segmentation and tracking of multiple video objects,” Pattern Recognition, Vol. 40, No. 4, pp. 1307-1317.
[2]A. Criminisi, P. Perez, and K. Toyama(2004), “Region filling and object removal by exemplar-based image inpainting,” IEEE Transactions on Image Processing, Vol. 13, No. 9, pp. 1200-1212.
[3]A. Koochari and M. Soryani(2010), “Exemplar-based video inpainting with large patches,” Journal of Zhejiang University Science C : Computer & Electronic, Vol. 11, No. 4, pp. 270-277.
[4]C. R. Wren, A. Azarbayejani, T. Darrell, and A. Pentland(1997), “Pfinder: Real-time tracking of the human body,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, No. 10, pp. 780-785.
[5]C. Stauffer and W. E. L. Grimson(1999), “Adaptive background mixture models for real-time tracking,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 246-252.
[6]C. H. Cheung and L. M. Po(2005), “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia, Vol. 7, No. 1, pp. 16-22.
[7]C. Harris and M. Stephens(1988), “A combined corner and edge detector,” In Proceedings of The Fourth Alvey Vision Conference, pp. 147-151.
[8]C. T. Hsu and Y. C. Tsan(2001), “Mosaics of video sequences with moving objects,” Proceedings of International Conference on Image Processing, Vol. 2, pp. 387-390.
[9]G. Wolberg(1990), “Digital Image Warping,” IEEE Computer Society Press, Los Alamitos, California.
[10]H. P. Morvavec(1980), “Obstacle avoidance and navigation in the real world by a seeing robot rover,” Technical Report,Carnegie-Mellon University, Robotics Institute, CMU-RI-TR-80-03.
[11]H. Y. Huang and C. N. Hsiao(2010), “A patch-based image inpainting based on structure consistence,” Computer Symposium (ICS), pp. 165-170.
[12]J. Y. Bouguet(2000), “Pyramidal Implementation of the Lucas Kanade Feature Tracker Description of the algorithm,” Intel Corporation, Microprocessor Research Labs.
[13]J. C. Hung, C. H. Hwang, Y. C. Liao, N C. Tang, T. J. Chen(2008), “Exemplar-based Image Inpainting base on Structure Construction,” Journal of Software, Vol. 3, No. 8, pp. 57-64.
[14]K. A. Patwardhan, G. Sapiro, and M. Bertalmio(2005), “Video inpainting of occluding and occluded objects,” In Proc. IEEE International Conference Image Process, Vol. 2, pp. 69-72.
[15]K. A. Patwardhan, G. Sapiro, and M. Bertalmio(2007), “Video inpainting under constrained camera motion,” IEEE Transactions on Image Processing, Vol. 16, No. 2, pp. 545-553.
[16]L. M. Po and W. C. Ma(1996), “A novel four-step search algorithm for fast block motion estimation,” IEEE Transformation Circuits Syst. Video Technol., Vol. 6, No. 2, pp. 313-317.
[17]M. Bertalmio, G. Sapiro, and V. Caselles(2000), “Image inpainting,” Proceedings of the 27th annual conference on Computer graphics and interactive technique, pp. 417-424.
[18]M. M. Oliveira, B. Bowen, R. McKenna, and Y. S. Chane(2001), “Fast digital image inpatinting,” Appeared in the Proceedings of the International Conference on Visualization, pp. 261-266.
[19]M. V. Venkatesh, S. S. Cheung, and J. Zhao(2009), “Efficient object-based video inpainting,” Journal of Pattern Recognition Letters, Vol. 30, No. 2, pp. 168-179.
[20]N. C. Tang, T. K. Shih, and J. N. Hwang(2009), “Exemplar-based video inpainting without ghost shadow artifacts by maintaining temporal continuity,” IEEE Trans. Circuits Syst. Video Technol., Vol. 19, No. 2, pp. 347-360.
[21]N. C. Tang, C. T. Hsu, C. W. Su, and T. K. Shih(2011), “video inpainting on digitized vintagefilms via maintaining spatiotemporal continuity,” IEEE Transactions on Multimedia., Vol. 13, No. 4, pp. 602-614.
[22]P. Perez, M. Gangnet, and A. Blake(2003), “Poisson image editing,” Proceedings of ACM SIGGRAPH 2003, pp. 313-318.
[23]Q. Zhang and J. Lin(2011), “Exemplar-based Image Inpainting Using Color Distribution Analysis,” Journal of Information Science and Engineering, Vol. 28, No. 4, pp. 1-12.
[24]S Li and M Zhao(2011), “Image inpainting with salient structure completion and texture propagation,” Journal of Pattern Recognition Letters, Vol. 32, No. 9, pp. 1256-1266.
[25]T. H. Tsai and C. L. Fang(2011), “Text video completion using structure repair and texture propagation,” IEEE Transactions on Multimedia, Vol. 13, No. 1, pp. 29-39.
[26]Y. Wexler, E. Shechtman, and M. Irani(2004), “Space-Time Video Completion,” Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 120-127.
[27]Y. Wexler, E. Shechtman, and M. Irani(2007), “Space-time completion of video,” IEEE Transactions on Pattern Analysis and Machine Intelligence., Vol. 29, No. 3, pp. 463-476.
[28]Z. Zivkovic(2004), “Improved adaptive Gaussian mixture model for background subtraction,” Proceedings of the 17th International Conference on Pattern Recognition, Vol. 2, pp. 28-31.