現今的科技時代與數位化的快速發展使得軟體、硬體都有很大的突破，AI人工智慧也飛快的進步。近年來機器學習領域研究也漸漸的成熟穩定，深度學習框架的普及不再是極困難的任務，所以人們開始漸漸往深度學習發展，也衍伸出了許多的方法與技術，像現在的TensorFlow、Caffe、Torch等主流框架都受到全世界的歡迎。在我們的生活中，往往可以發現深度學習的應用，舉例來說，能流暢對話的語音助理、自動駕駛帶來的方便、手機攝影的影像演算法，這些應用對我們生活中都有著極大影響。這些功能都與影像辨識有著重大關係，比如說車載系統，在下雨天辨識出來的圖片，因氣候關係導致辨識效果不好，可能對自動駕駛會有嚴重的影響，為了解決因雨水造成辨識困難，因而提出本論文來克服雨水造成的相關影像問題。
本論文主要目的是解決帶有雨滴雜訊圖像還原成無雨滴的圖像，提出一種基於pix2pix改進cGAN (conditional Generative Adversarial Networks, cGAN)框架並使cGAN中的生成器結合U-Net，再利用PatchGAN當判別器來與生成器做比較。主要處理步驟如下：首先對圖像進行預處理，主要目的增加資料量與把帶雨滴標籤圖像與原圖像做分割輸入，其次先建構生成器與判別器，再把預處理圖像輸入，並訓練生成器與判別器，兩者在訓練時會互相對抗，來提升除雨過後生成圖像的品質。本論文之方法主要探討生成器生成無雨滴圖像的主觀質量，將計算Peak Signal to Noise Ratio (PSNR) 及Structural SIMilarity index (SSIMS) 值呈現出該生成無雨滴圖像的可視性。

The current technological era and the rapid development of digitization have made great breakthroughs in software and hardware, and AI artificial intelligence has also made rapid progress. In recent years, research in the field of machine learning has gradually matured and stabilized, and the popularization of deep learning frameworks is no longer an extremely difficult task. Therefore, people have gradually begun to develop deep learning, and many methods and technologies have been developed, such as the current TensorFlow, Mainstream frameworks such as Caffe and Torch are popular all over the world. In our life, we can often find the application of deep learning. For example, the voice assistant that can communicate smoothly, the convenience brought by automatic driving, the image algorithm of mobile phone photography, these applications have a great impact on our life. These functions have a major relationship with image recognition. For example, the in-vehicle system, the pictures recognized in rainy days, the recognition effect is not good due to the climate relationship, which may have a serious impact on automatic driving. In order to solve the problem of recognition difficulties caused by rain, this thesis proposed a feasible method to overcome the related image problems caused by rain.
The main purpose of this thesis is to solve the problem of restoring images with raindrop noise to an image without raindrops, and to design an improved cGAN (conditional Generative Adversarial Networks, cGAN) framework based on pix2pix mechanism. The generator built on our improved U-Net structure, and then used PatchGAN as the discriminator (D) compared to the generator (G). The main processing steps are described as follows: first, preprocess the image, the main purpose is to increase the amount of data and to divide the image with the raindrop label and the original image as input, secondly construct the generator and discriminator, and then input the preprocessed image. Afterward, train the generator and the discriminator, and the two will confront each other during training to improve the quality of the generated images after rain removal. This method mainly explores the subjective quality of the raindrop-free image generated by the generator, and calculates the Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity index (SSIMS) values to show the visibility of the generated raindrop-free image.

摘要...i
Abstract...iii
誌謝...v
目錄...vi
表目錄...ix
圖目錄...xi
符號說明...xiv
第一章 緒論...1
1.1 簡介...1
1.2 研究動機與目的...1
1.3 論文架構 ...2
第二章 基礎倫理與文獻探討...3
2.1 生成對抗網路(Generative Adversarial Network, GAN)...3
2.2 條件式生成對抗網路(Conditional Generative Adversarial Nets，cGAN)...4
2.2.1 U-Net影像分割...5
2.2.2 PatchGAN...7
2.3 像素點對像素點做預測(pix2pix)...8
2.4 相關文獻...10
第三章 系統架構...17
3.1 像素點對像素點做預測(pix2pix)...19
3.2 改良式條件式生成對抗網路 (Improved Conditional Generative Adversarial Nets, IcGAN)...21
3.2.1 改良生成器(Improved Generator , G ̃ )...22
3.2.2 判別器 (Discriminator)...32
3.3 除雨圖像(De-raining image)...38
第四章 實驗結果與討論...41
4.1 實驗環境平台...41
4.2 實驗資料庫...41
4.3 實驗參數設定...45
 改良生成器 (Improved Generator)...45
 判別器 (Discriminator)...45
 IcGAN Loss function...45
4.4 實驗結果與分析...45
4.4.1 使用Isola et al.[1]dataset測試結果...47
4.4.2 使用Han et al.[5] dataset測試結果...49
4.4.3 使用Zamir et al. [6] dataset 測試結果...51
4.4.4 使用Isola et al.[1]提供的Real Rain dataset加上自製的Real Rain dataset...53
4.4.5 整體評估...55
4.5 討論...57
 Generator model 選擇...57
 Discriminator model 選擇...58
 IcGAN 中 λ 值...59
第五章 結論與未來展望...61
參考文獻...62
Extended Abstract...64

[1]Y Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros, “Image-to-Image Translation with Conditional Adversarial Networks”, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1125-1134.
[2]Mehdi Mirza and Simon Osindero, “Conditional Generative Adversarial Nets,” arXiv:1411.1784v1, Nov. 6, 2014.
[3]Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio, “Generative Adversarial Networks”, arXiv:1406.2661v1, Jun. 10, 2014.
[4]Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, arXiv:1505.04597v1, May 18, 2015.
[5]Junlin Han, Weihao Li, Pengfei Fang, Chunyi Sun, Jie Hong, Mohammad Ali Armin, Lars Petersson, and Hongdong Li, “Blind Image Decomposition,” arXiv:2108.11364v3, Aug. 25, 2021.
[6]Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao, “Multi-Stage Progressive Image Restoration,” arXiv:2102.02808v2, Mar. 16, 2021.
[7]Rui Qian, Robby T. Tan, Wenhan Yang , Jiajun Su1, and Jiaying Liu, “Attentive Generative Adversarial Network for Raindrop Removal from A Single Image,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 2482-2491.
[8]He Zhang, Vishwanath Sindagi, and Vishal M. Patel, “Image De-Raining Using a Conditional Generative Adversarial Network,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 11, pp. 3943-3956, 2019.
[9]Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley, “Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2944 - 2956, 2017.
[10]Yuhui Quan, Shijie Deng, Yixin Chen, and Hui Ji, “Deep Learning for Seeing Through Window With Raindrops,” in Proc. of the IEEE/CVF Int. Conf. on Computer Vision (ICCV), 2019, pp. 2463-2471.
[11]Yong Jia, Yong Guo, Shengyi Chen, Ruiyuan Song, Gang Wang, Xiaoling Zhong, Chao Yan, and Guolong Cui, “Multipath Ghost and Side/Grating Lobe Suppression Based on Stacked Generative Adversarial Nets in MIMO Through-Wall Radar Imaging,” IEEE Access, vol. 7, pp. 143367-143380, 2019.