天氣，尤其是霧，會降低駕駛輔助鏡頭或外部監控系統捕捉到的圖像品質。當圖像因天空或太陽而曝光過度時，使用的暗通道先驗除霧技術就會失效，本論文改進了一種基於暗通道先驗除霧技術，促使我們創建了一種自適應機制，以提高算法在這種情況下的有效性。該技術分為兩個主要階段：去除霧氣和對生成的圖像進行後期處理以提高對比度。程序最初使用 MATLAB 開發和評估。在 MATLAB 上的評估結果顯示，與無霧參考圖像相比，建議的方法可以重現無霧場景，結構相似度高達 0.97。它的處理速度是目前基於學習的系統的 1249.41 倍。
基於良好的評估結果，該方法採用 C的 HLS高級合成方法在 Zynq MPSoC FPGA 上實現。在移植到 FPGA 時，該方法的 PSNR 和 SSIM 分別僅比原始 MATLAB 實現損失 4.067% 和 1.116%。該設備以 464.667 FPS 的速度處理全高清解析度影像，同時功耗極低，僅為 0.952W。所提出的系統方法還提高了影像視覺應用的性能，如 YOLOv4 物體檢測。它將 YOLOv4 的準確率提高了 35%。

Weather, particularly fog, degrades the quality of images captured by driver assistance cameras or outside surveillance systems. This thesis enhances a dark channel prior-based technique for removing fog that otherwise fails when the image is overexposed due to the sky or the sun. This led to the creation of an adaptive mechanism to improve the algorithm's effectiveness in such situations. The technique is divided into two primary stages: removing the fog and post-processing the resulting image to increase contrast. The program was initially developed and assessed using MATLAB. The evaluation results on MATLAB reveal that, when compared to reference fog-free images, the suggested method can recreate a scene without fog with a structural similarity of up to 0.97. It can process 1249.41 times faster than current learning-based systems.
Based on the promising evaluation results, the method was implemented on Zynq MPSoC FPGAs using a C-style high level synthesis methodology. It loses only 4.067% and 1.116% of the original MATLAB implementation's PSNR and SSIM, respectively, while porting to FPGA. The device can process Full HD resolution at an incredibly fast rate of 464.667 FPS while using extremely low power at 0.952W. Additionally, the proposed system enhances the performance of high-level computer vision applications, such as YOLOv4 object detection. It improves YOLOv4’s accuracy up to 35%.

摘要........i
Abstract........ii
Acknowledgments........iii
Table of Contents........iv
List of Tables........vii
List of Figures........viii
Abbreviations........ix
Chapter 1. Introduction........1
1.1. Research Motivation........1
1.2. Contributions........1
1.3. Thesis Organization........1
Chapter 2. Background Knowledge........3
2.1. Introduction........3
2.2. Atmospheric Scattering Model........3
2.3. Single-Image Fog Removal Algorithms........4
2.3.1. Image Enhancement Methods........5
2.3.2. Prior-based Methods........5
2.3.3. Learning-based Methods........8
2.4. Fog Removal Evaluation........9
2.4.1. RESIDE dataset........9
2.4.2. Quality Metrics........10
2.4.2.1. Full-reference........11
2.4.2.2. No-reference........12
2.4.2.3. Human-subjective Scoring........12
2.5. High-Level Synthesis Design........13
2.6. Video Processing on FPGAs........14
2.6. Vitis Development Flow........15
2.8. Vitis Vision Library........16
Chapter 3. System Architecture........18
3.1. Introduction........18
3.2. Multiprocessor System-on-Chip FPGA platforms........18
3.2.1. Processing System........19
3.2.2. Programmable Logic........19
3.2.2.1. Configurable Logic Block........19
3.2.2.2. Block RAM........20
3.2.2.3. UltraRAM........20
3.2.2.4. Digital Signal Processing........20
3.2.3. PS-PL Interface........20
3.2.4. Zynq UltraScale+ MPSoC ZCU104 Evaluation Kit........21
3.3. Preprocessing Unit........22
3.3.1. Single-Image Fog Removal Subsystem........22
3.3.2. Post-Processing Subsystem........22
3.4. Input and Output Video Interfaces........23
3.5. Development Framework........24
Chapter 4. Single-Image Fog Removal Algorithm........26
4.1. Introduction........26
4.2. Fog Removal using Improved Dark Channel Prior........27
4.2.1. Dark Channel Extraction........27
4.2.2. Bright Channel Extraction........28
4.2.3. Transmission Map Estimation using Anisotropic Diffusion........30
4.2.4. Scene Recovery........32
4.2.5. Adaptive Fog Factor........33
4.3. Post-processing........38
4.3.1. Histogram Calculation........38
4.3.2. Cumulative Histogram........39
4.3.3. Contrast Stretching........39
Chapter 5. System Integration and Experiment Results........42
5.1. Fog Removal Evaluation........42
5.1.1. Visual Quality Comparison........44
5.1.2. Qualitative Comparison........44
5.2. Implement Accelerated Fog Removal System........48
5.2.1. Software Implementation........48
5.2.2. HLS Kernels Implementation........48
5.2.2.1. darkchannel_accel()........49
5.2.2.2. brightchannel_accel()........50
5.2.2.3. trans_accel()........51
5.2.2.5. contrastenhance_accel()........51
5.2.3. System Integration and Profiling........52
5.2.3.2. System Integration........52
5.2.3.3. Profiling........53
5.2.4. Embedded Fog Removal Evaluation........54
5.2.5. Comparison with Previous Studies........55
5.3. Aid to Task-Driven Computer Vision........55
Chapter 6. Conclusion........57
References........58
Extended Abstract........63

[1]A. Mukhtar, L. Xia, and T. B. Tang, "Vehicle Detection Techniques for Collision Avoidance Systems: A Review," IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 5, pp. 2318-2338, 2015.
[2]W. Ge, R. T. Collins, and R. B. Ruback, "Vision-Based Analysis of Small Groups in Pedestrian Crowds," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 1003-1016, 2012.
[3]D. G. Lee, H. I. Suk, S. K. Park, and S. W. Lee, "Motion Influence Map for Unusual Human Activity Detection and Localization in Crowded Scenes," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 10, pp. 1612-1623, 2015.
[4]E. C. Tetila, B. B. Machado, N. A. Belete, D. A. Guimarães, and H. Pistori, "Identification of Soybean Foliar Diseases Using Unmanned Aerial Vehicle Images," IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 12, pp. 2190-2194, 2017.
[5]K. Harald, "Theorie der horizontalen sichtweite: Kontrast und sichtweite," Keim and Nemnich, Munich, vol. 12, 1924.
[6]W. E. K. Middleton, "Vision through the Atmosphere," in Geophysik II / Geophysics II, J. Bartels, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 1957, pp. 254-287.
[7]J. McCartney, Optics of the Atmosphere: Scattering by Molecules and Particles (no. v. 341976). Wiley, 1976.
[8]J. A. Stark, "Adaptive image contrast enhancement using generalizations of histogram equalization," IEEE Transactions on Image Processing, vol. 9, no. 5, pp. 889-896, 2000.
[9]S. G. Narasimhan and S. K. Nayar, "Contrast restoration of weather degraded images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 713-724, 2003.
[10]J.-Y. Kim, L.-S. Kim, and S.-H. Hwang, "An advanced contrast enhancement using partially overlapped sub-block histogram equalization," IEEE transactions on circuits systems for video technology, vol. 11, no. 4, pp. 475-484, 2001.
[11]M. Kim and M. G. Chung, "Recursively separated and weighted histogram equalization for brightness preservation and contrast enhancement," IEEE Transactions on Consumer Electronics, vol. 54, no. 3, pp. 1389-1397, 2008.
[12]C. Busch and E. Debes, "Wavelet transform for analyzing fog visibility," IEEE Intelligent Systems Their Applications, vol. 13, no. 6, pp. 66-71, 1998.
[13]J. Jia and H. Yue, "A wavelet-based approach to improve foggy image clarity," IFAC Proceedings Volumes, vol. 47, no. 3, pp. 930-935, 2014.
[14]J. P. Tarel and N. Hautière, "Fast visibility restoration from a single color or gray level image," in 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 2201-2208.
[15]K. He, J. Sun, and X. Tang, "Single image haze removal using dark channel prior," IEEE transactions on pattern analysis machine intelligence, vol. 33, no. 12, pp. 2341-2353, 2010.
[16]A. K. Tripathi and S. Mukhopadhyay, "Single image fog removal using anisotropic diffusion," (in En), IET Image Processing, vol. 6, no. 7, pp. 966-975, 2012.
[17]G. Meng, Y. Wang, J. Duan, S. Xiang, and C. Pan, "Efficient Image Dehazing with Boundary Constraint and Contextual Regularization," in IEEE International Conference on Computer Vision (ICCV), 2013.
[18]C. Xiao and J. Gan, "Fast image dehazing using guided joint bilateral filter," The Visual Computer, vol. 28, no. 6, pp. 713-721, 2012/06/01 2012.
[19]K. He, J. Sun, and X. Tang, "Guided Image Filtering," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397-1409, 2013.
[20]B. H. Chen, S. C. Huang, and F. C. Cheng, "A High-Efficiency and High-Speed Gain Intervention Refinement Filter for Haze Removal," Journal of Display Technology, vol. 12, no. 7, pp. 753-759, 2016.
[21]J. M. Guo, J. y. Syue, V. R. Radzicki, and H. Lee, "An Efficient Fusion-Based Defogging," IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4217-4228, 2017.
[22]Y. Park and T. H. Kim, "Fast Execution Schemes for Dark-Channel-Prior-Based Outdoor Video Dehazing," IEEE Access, vol. 6, pp. 10003-10014, 2018.
[23]M. Zhu, B. He, and Q. Wu, "Single Image Dehazing Based on Dark Channel Prior and Energy Minimization," IEEE Signal Processing Letters, vol. 25, no. 2, pp. 174-178, 2018.
[24]P. J. Liu, S. J. Horng, J. S. Lin, and T. Li, "Contrast in Haze Removal: Configurable Contrast Enhancement Model Based on Dark Channel Prior," IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2212-2227, 2019.
[25]S. Salazar-Colores, E. Cabal-Yepez, J. M. Ramos-Arreguin, G. Botella, L. M. Ledesma-Carrillo, and S. Ledesma, "A Fast Image Dehazing Algorithm Using Morphological Reconstruction," IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2357-2366, 2019.
[26]S. Salazar-Colores, E. U. Moya-Sánchez, J. M. Ramos-Arreguín, E. Cabal-Yépez, G. Flores, and U. Cortés, "Fast Single Image Defogging With Robust Sky Detection," IEEE Access, vol. 8, pp. 149176-149189, 2020.
[27]S. C. Raikwar and S. Tapaswi, "Lower Bound on Transmission Using Non-Linear Bounding Function in Single Image Dehazing," IEEE Transactions on Image Processing, vol. 29, pp. 4832-4847, 2020.
[28]Q. Zhu, J. Mai, and L. Shao, "A fast single image haze removal algorithm using color attenuation prior," IEEE transactions on image processing, vol. 24, no. 11, pp. 3522-3533, 2015.
[29]R. Fattal, "Dehazing using color-lines," ACM transactions on graphics, vol. 34, no. 1, pp. 1-14, 2014.
[30]M. Sulami, I. Glatzer, R. Fattal, and M. Werman, "Automatic recovery of the atmospheric light in hazy images," in 2014 IEEE International Conference on Computational Photography (ICCP), 2014, pp. 1-11.
[31]D. Berman, T. Treibitz, and S. Avidan, "Non-local Image Dehazing," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1674-1682.
[32]T. M. Bui and W. Kim, "Single image dehazing using color ellipsoid prior," IEEE Transactions on Image Processing, vol. 27, no. 2, pp. 999-1009, 2017.
[33]D. Berman, T. Treibitz, and S. Avidan, "Single image dehazing using haze-lines," IEEE transactions on pattern analysis machine intelligence, vol. 42, no. 3, pp. 720-734, 2018.
[34]Q. Guo, H. Wang, and J. Yang, "Adaptive Dark Channel Prior Enhancement Algorithm for Different Source Night Vision Halation Images," IEEE Access, vol. 10, pp. 92726-92739, 2022.
[35]S. Hong, M. Kim, H. Lee, and M. G. Kang, "Nighttime Single Image Dehazing Based on the Structural Patch Decomposition," IEEE Access, vol. 9, pp. 82070-82082, 2021.
[36]G. Bi, G. Si, Y. Zhao, B. Qi, and H. Lv, "Haze Removal for a Single Remote Sensing Image Using Low-Rank and Sparse Prior," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-13, 2022.
[37]D. Singh, M. Kaur, M. Y. Jabarulla, V. Kumar, and H. N. Lee, "Evolving Fusion-Based Visibility Restoration Model for Hazy Remote Sensing Images Using Dynamic Differential Evolution," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-14, 2022.
[38]S. C. Pei and C. Y. Chen, "Underwater Images Enhancement by Revised Underwater Images Formation Model," IEEE Access, vol. 10, pp. 108817-108831, 2022.
[39]J. Yuan, Z. Cai, and W. Cao, "TEBCF: Real-World Underwater Image Texture Enhancement Model Based on Blurriness and Color Fusion," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-15, 2022.
[40]X. Ge, J. Tan, and L. Zhang, "Blind Image Deblurring Using a Non-Linear Channel Prior Based on Dark and Bright Channels," IEEE Transactions on Image Processing, vol. 30, pp. 6970-6984, 2021.
[41]F. Wen, R. Ying, Y. Liu, P. Liu, and T. K. Truong, "A Simple Local Minimal Intensity Prior and an Improved Algorithm for Blind Image Deblurring," IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 8, pp. 2923-2937, 2021.
[42]Y. Xin, Z. Jia, J. Yang, and N. K. Kasabov, "Specular Reflection Image Enhancement Based on a Dark Channel Prior," IEEE Photonics Journal, vol. 13, no. 1, pp. 1-11, 2021.
[43]W. T. Chen, Y. W. Chen, K. Y. Chen, J. J. Ding, and S. Y. Kuo, "Single Image Reflection Removal Based on Bi-Channels Prior," in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 2117-2121.
[44]B. Cai, X. Xu, K. Jia, C. Qing, and D. Tao, "DehazeNet: An End-to-End System for Single Image Haze Removal," IEEE Transactions on Image Processing, vol. 25, no. 11, pp. 5187-5198, 2016.
[45]B. Li, X. Peng, Z. Wang, J. Xu, and D. Feng, "AOD-Net: All-in-One Dehazing Network," in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4780-4788.
[46]X. Qin, Z. Wang, Y. Bai, X. Xie, and H. Jia, "FFA-Net: Feature Fusion Attention Network for Single Image Dehazing," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 11908-11915, 04/03 2020.
[47]Y. Song, Z. He, H. Qian, and X. Du, "Vision Transformers for Single Image Dehazing," Trans. Img. Proc., vol. 32, pp. 1927–1941, 2023.
[48]P. Ling, H. Chen, X. Tan, Y. Jin, and E. Chen, "Single Image Dehazing Using Saturation Line Prior," IEEE Transactions on Image Processing, vol. 32, pp. 3238-3253, 2023.
[49]S. Li et al., "Single Image Deraining: A Comprehensive Benchmark Analysis," 2019.
[50]H. Dong et al., "Multi-Scale Boosted Dehazing Network With Dense Feature Fusion," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 2154-2164.
[51]M. Ju, C. Ding, Y. J. Guo, and D. Zhang, "IDGCP: Image Dehazing Based on Gamma Correction Prior," IEEE Transactions on Image Processing, vol. 29, pp. 3104-3118, 2020.
[52]S. E. Kim, T. H. Park, and I. K. Eom, "Fast Single Image Dehazing Using Saturation Based Transmission Map Estimation," IEEE Transactions on Image Processing, vol. 29, pp. 1985-1998, 2020.
[53]A. Dudhane, P. W. Patil, and S. Murala, "An End-to-End Network for Image De-Hazing and Beyond," IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 6, no. 1, pp. 159-170, 2022.
[54]X. Liu, H. Li, and C. Zhu, "Joint Contrast Enhancement and Exposure Fusion for Real-World Image Dehazing," IEEE Transactions on Multimedia, vol. 24, pp. 3934-3946, 2022.
[55]B. Li et al., "Benchmarking Single-Image Dehazing and Beyond," IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 492-505, 2019.
[56]A. Mittal, A. K. Moorthy, and A. C. Bovik, "No-Reference Image Quality Assessment in the Spatial Domain," IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4695-4708, 2012.
[57]A. Mittal, R. Soundararajan, and A. C. Bovik, "Making a “Completely Blind” Image Quality Analyzer," IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209-212, 2013.
[58]Y. Li, S. You, M. S. Brown, and R. T. Tan, "Haze visibility enhancement: A Survey and quantitative benchmarking," Computer Vision and Image Understanding, vol. 165, pp. 1-16, 2017/12/01/ 2017.
[59]U. Pulkkinen, "Bayesian analysis of consistent paired comparisons," Reliability Engineering & System Safety, vol. 43, no. 1, pp. 1-16, 1994/01/01/ 1994.
[60]R. A. Bradley and M. E. Terry, "Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons," Biometrika, vol. 39, no. 3/4, pp. 324-345, 1952.
[61]M. C. McFarland, A. C. Parker, and R. Camposano, "The high-level synthesis of digital systems," Proceedings of the IEEE, vol. 78, no. 2, pp. 301-318, 1990.
[62]R. Gupta and F. Brewer, "High-Level Synthesis: A Retrospective," in High-Level Synthesis: From Algorithm to Digital Circuit, P. Coussy and A. Morawiec, Eds. Dordrecht: Springer Netherlands, 2008, pp. 13-28.
[63]Advanced Micro Devices - Xilinx, "Vitis High-Level Synthesis User Guide (UG1399)," 2021, Available: https://docs.xilinx.com/r/2021.2-English/ug1399-vitis-hls, Accessed on: 2023.
[64]R. Kastner, J. Matai, and S. Neuendorffer, "Parallel programming for FPGAs," arXiv preprint arXiv:.03648, 2018.
[65]Advanced Micro Devices - Xilinx. (2020). Vitis Vision Library. Available: https://www.xilinx.com/products/design-tools/vitis/vitis-libraries/vitis-vision.html
[66]G. Z. Lin, H. M. Nguyen, C. C. Sun, P. Y. Kuo, and M. H. Sheu, "A Novel Bird Detection and Identification based on DPU processor on PYNQ FPGA," in 2021 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), 2021, pp. 1-2.
[67]O. Iqbal, V. I. T. Muro, S. Katoch, A. Spanias, and S. Jayasuriya, "Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation," IEEE Access, vol. 10, pp. 90507-90522, 2022.
[68]M. J. R. Valido, E. M. Castello, P. S. Medina, and A. M. González, "Accelerating applications with Vitis unified environment. Case study: Vitis Vision Library," in 2022 Congreso de Tecnología, Aprendizaje y Enseñanza de la Electrónica (XV Technologies Applied to Electronics Teaching Conference), 2022, pp. 1-5.
[69]E. Sola-Thomas, M. A. B. Sarker, and M. Imtiaz, "FPGA-Controlled AI Vision for Prosthetics hand," in 2023 IEEE World AI IoT Congress (AIIoT), 2023, pp. 0520-0524.
[70]A. E. Wilson, "Dynamic Reconfigurable Real-Time Video Processing Pipelines on SRAM-based FPGAs," Master of Science Thesis, Ira A. Fulton College of Engineering and Technology; Electrical and Computer Engineering, Brigham Young University, Provo, Utah, 2020.
[71]Advanced Micro Devices - Xilinx. (2015). Zynq UltraScale+ MPSoC. Available: https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html
[72]Advanced Micro Devices - Xilinx. (2018). Zynq UltraScale+ MPSoC ZCU104 Evaluation Kit. Available: https://www.xilinx.com/products/boards-and-kits/zcu104.html
[73]I. Kawazome, "ArgSort-Ultra96," VHDL source code 2022, Available: https://github.com/ikwzm/ArgSort-Ultra96.
[74]P. Perona and J. Malik, "Scale-space and edge detection using anisotropic diffusion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, 1990.
[75]P. Diaconescu and V. E. Neagoe, "A Higly Configurable Deep Learning Architecture for Hyperspectral Image Classification," in 2019 IEEE 13th International Symposium on Applied Computational Intelligence and Informatics (SACI), 2019, pp. 197-200.
[76]C. S. A. Gong, C. H. S. Su, and K. H. Tseng, "Implementation of Machine Learning for Fault Classification on Vehicle Power Transmission System," IEEE Sensors Journal, vol. 20, no. 24, pp. 15163-15176, 2020.
[77]Y. Hu, Y. Jin, X. Wu, and J. Chen, "A Theory-Guided Deep Neural Network for Time Domain Electromagnetic Simulation and Inversion Using a Differentiable Programming Platform," IEEE Transactions on Antennas and Propagation, vol. 70, no. 1, pp. 767-772, 2022.
[78]C.-C. Kao, J.-H. Lai, S.-Y. J. I. T. o. c. Chien, and s. f. v. technology, "VLSI architecture design of guided filter for 30 frames/s full-HD video," vol. 24, no. 3, pp. 513-524, 2013.
[79]Z. Liang, H. Liu, B. Zhang, and B. Wang, "Real-time hardware accelerator for single image haze removal using dark channel prior and guided filter," IEICE Electronics Express, vol. 11, no. 24, pp. 20141002-20141002, 2014.
[80]B. Zhang and J. Zhao, "Hardware implementation for real-time haze removal," IEEE Transactions on Very Large Scale Integration Systems, vol. 25, no. 3, pp. 1188-1192, 2016.
[81]B. Zhang and J. Wei, "Hardware Implementation for Haze Removal With Adaptive Filtering," IEEE Access, vol. 7, pp. 142498-142506, 2019.
[82]A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," CoRR, vol. abs/2004.10934, 2020 2020.