臺灣博碩士論文加值系統

本論文提出了深度學習鳥類識別，並在具有高帶寬內存架構的Alveo加速卡上實現。新的檢測方法分為兩個子過程，即使用加速圖像處理的運動物體檢測和使用可配置神經網絡推理的鳥類識別。運動目標檢測過程基於幀差原理，配合其他圖像預處理技術，在連續視頻幀中檢測和確定目標運動的位置。在運動物體檢測中經過處理和檢測後，將具有運動物體的幀及其在圖像中的大小和位置記錄下來。確認的移動對象通過深度學習處理器單元 (DPU) 進行分類，從而得出鳥類的名稱。該方法在 Alveo U50 加速卡上實現和測試，結果表明該方法可以達到高達 81% 的高精度，處理 960x540 分辨率視頻時的執行速度為 289 FPS，處理高清清晰度時為 84 FPS (1920x1080) 視頻，此外僅 DPU 執行就可以達到 583 FPS。

This thesis proposed deep learning bird identification and implemented it on Alveo accelerator card with high bandwidth memory architecture. The novel detection method is divided into two sub-processes, which are moving object detection using accelerated image processing and bird identification using configurable neural network inference. The moving object detection process is based on the principle of frame difference, in complement with other image pre-processing techniques, to detect and determine the position of an object’s movement in consecutive video frames. After being processed and detected in the moving object detection, frames that have moving objects are recorded with their size and position within the image. The confirmed moving object is pushed through a deep-learning processor unit (DPU) for classification, resulting in the name of the bird species. The proposed method implemented and tested on Alveo U50 accelerated card yields the result showing that the proposed method can reach high accuracy of up to 81 percent, and with the execution speed of 289 FPS while processing 960x540 resolution videos, and 84 FPS while processing HD definition (1920x1080) videos, furthermore the DPU execution alone can reach 583 FPS.

摘要..............i
Abstract............ii
Acknowledgements............iii
Table of Contents............iv
List of Tables............vii
List of Figures............viii
Abbreviations............x
Chapter 1. Introduction............1
1.1. Research Motivation............1
1.2. Research Purpose............1
1.3. Research structure............2
Chapter 2. Background Knowledge............3
2.1. Introduction............3
2.2. Neural Network for Classification............3
2.3. Xilinx Vitis-AI............5
2.4. Vitis Vision............7
Chapter 3. System Architecture............8
3.1. Introduction............8
3.2. FPGA Accelerated Card............8
3.3. Host computer............10
3.4. Movement Detection Unit............10
3.4.1. Image processing techniques............10
3.4.2. Accelerated Image Processing Functions............11
3.5. Deep-Learning Processing Unit............12
3.5.1. Architecture............12
3.5.2. Quantization............13
Chapter 4. Bird Identification Algorithm............14
4.1. Introduction............14
4.2. Movement Detection............16
4.2.1. Gray conversion............17
4.2.2. Frame Difference............18
4.2.3. Opening............19
4.2.3.1. Erode............19
4.2.3.2. Dilate............21
4.2.4. Blur............22
4.2.5. Threshold............22
4.2.6. Sampling............23
4.3. Pre-processing............26
4.3.1. Crop............26
4.3.2. Resize............27
4.3.3. Mean subtraction............27
4.4. DPU Classification............29
4.4.1. Neural network models............29
4.4.1.1. MobileNet............30
4.4.1.2. Resnet50............30
4.4.1.3. Inception-v4............31
4.4.2. ImageNet dataset............31
Chapter 5. System Integration and Experiment Results............34
5.1. Implement Accelerated Movement Detection............34
5.1.1. HLS Kernel System Hardware Implementation............34
5.1.1.1. mov_det kernel............34
5.1.1.2. crop_accel kernel............35
5.1.1.3. resize_accel kernel............36
5.1.1.4. Inter-kernel connection............37
5.1.1.5. Resource utilization............38
5.1.2. Hardware Movement Detection Testbench............39
5.1.2.1. Test program structure............39
5.1.2.2. Test video............41
5.1.2.3. Test result............42
5.2. Implement Accelerated DPU Classification............43
5.2.1. DPU kernel diagram and hardware utilization............43
5.2.2. Hardware DPU Classification Testbench with Software Movement Detection............45
5.2.2.1. Test program structure............45
5.2.2.2. Test result............47
5.2.2.3. Comparison with prior studies............50
Chapter 6. Conclusion............53
References............54
Appendix - Work on Other Classes of ImageNet............57
Extended Abstract............62

1.J. P. Tracey, M. Bomford, Q. Hart, G. Saunders and R. Sinclair, “Managing Bird Damage to Fruit and Other Horticultural Crops Managing Bird Damage to Fruit and Other Horticultural Crops,” Australia Government - Bureau of Rural Sciences, 2015.
2.G. P. Zhang, “Neural Networks for Classification: A Survey,” 2000 IEEE Transactions on Systems Man and Cyberneticsm, 2000.
3.L. Y. Jin and C. Tian Sheuan, “Data and Hardware Efficient Design for Convolutional Neural Network,” IEEE Transactions on Circuits and Systems I: Regular Papers, pp. 1642-1651, May. 2018.
4.C. Huang, S. Ni, G. Chen, “A layer-based structured design of CNN on FPGA,” IEEE 12th International Conference on ASIC (ASICON), pp. 1037-1040, Oct. 2017.
5.S. Moini, B. Alizadeh, M. Emad and R. Ebrahimpour, “A Resource-Limited Hardware Accelerator for Convolutional Neural Networks in Embedded Vision Applications,” IEEE Transactions on Circuits and Systems II: Express Briefs, pp. 1217-1221, Oct. 2017.
6.T. Geng, T. Wang, A. Sanaullah, C. Yang, R. Xu, R. Patel and M. Herbordt, “FPDeep: Acceleration and Load Balancing of CNN Training on FPGA Clusters,” IEEE 26th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 2576-2621, May. 2018.
7.F. U. D. Farrukh, T. Xie, C. Zhang and Z. Wang, “Optimization for Efficient Hardware Implementation of CNN on FPGA,” IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA), pp. 88-89, May 2019.
8.K. Guo, L. Sui, J. Qiu, J. Yu, J. Wang, S. Yao, S. Han and Y. Wang, “Angel-Eye: A Complete Design Flow for Mapping CNN onto Embedded FPGA,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 35 - 47, Jan. 2018.
9.D. Song and Y. Xu, “A Low False Negative Filter for Detecting Rare Bird Species from Short Video Segments Using a Probable Observation Data Set-Based EKF Method,” IEEE Transactions on Image Processing, pp. 2321 - 2331, Apr. 2010.
10.W. Li and D. Song, “Automatic Bird Species Detection from Crowd Sourced Videos,” IEEE Transactions on Automation Science and Engineering, pp. 348 - 358, Apr. 2014.
11.D. Karmaker, “WHoG: A weighted HoG-based scheme for the detection of birds and identification of their poses in natural environments,” International Conference on Control, Automation, Robotics and Vision (ICARCV), Nov. 2016.
12.W. Li, “Automatic bird species detection using periodicity of salient extremities,” IEEE International Conference on Robotics and Automation, pp. 5775-5780, May. 2013.
13.R. Yoshihashi, R. Kawakami, M. Iida and T. Naemura, “Construction of a bird image dataset for ecological investigations,” IEEE International Conference on Image Processing (ICIP), pp. 4248-4252, Sept. 2015.
14.S. Tian and X. Cao, “Learning the state space based on flying pattern for bird detection,” Integrated Communications, Navigation and Surveillance Conference (ICNS), Apr. 2017.
15.A. Takeki, T.T. Trinh, R. Yoshihashi, R. Kawakami, M. Iida and T. Naemura, “Detection of small birds in large images by combining a deep detector with semantic segmentation,” IEEE International Conference on Image Processing (ICIP), pp. 3977-3981, Sept. 2016.
16.A. Kalantar, Z. Zimmerman and P. Brisk, “FA-LAMP: FPGA-Accelerated Learned Approximate Matrix Profile for Time Series Similarity Prediction,” 2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 40-49, 2021.
17.D. Danopoulos, C. Kachris and D. Soudris, “Covid4HPC: A Fast and Accurate Solution for Covid Detection in the Cloud Using X-Rays,” International Symposium on Applied Reconfigurable Computing (ARC): Architectures, Tools, and Applications, pp. 327-336, 2021.
18.J. Peng et al., “Multi-task ADAS system on FPGA,” 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 171-174, 2019.
19.S. Brandenburg, P. Machado, N. Lama and T.M. McGinnity, “Strawberries Detection Using a Heterogeneous Multi-Processor Platform”, 2020 IEEE International Conference on Intelligent Robots and System (IROS), Las Vegas, USA, Oct. 2020.
20.K. Cosmas and A. Kenichi, “Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation,” Aerospace, Vol.7, Issue 11, No. 159, Nov. 2020.
21.K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016.
22.A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” arXiv preprint arXiv:1704.048612017.
23.C. Szegedy, S. Ioffe, V. Vanhoucke and A. A. Alemi, “Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 2016.
24.R. Zhao, W. Song, W. Zhang, T. Xing, J. Lin, M. B. Srivastava, R. K. Gupta and Z. Zhang, “Accelerating Binarized Convolutional Neural Networks with Software Programmable FPGAs,” International Symposium on Field-Programmable Gate, pp. 15- 24, Feb. 2017.
25.D. T. Nguyen, T.N. Nguyen, H. Kim and H. J. Lee, “A High-Throughput and Power EfficientFPGA Implementation of YOLO CNN for Object Detection,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, pp. 1861 - 1873, Aug. 2019.
26.S. D. Das, and A. Kumar, “Bird Species Classification using Transfer Learning with Multistage Training,” arXiv preprint arXiv: 1810.04250v1.
27.S. Bouguezzi, H. B. Fredj, T. Beladed, C. Valderrama, H. Faiedh and C. Souani, “An Efficient FPGA-Based Convolutional Neural Network for Classification: Ad-MobileNet,” MDPI Journal of Electronics, vol. 10, issue 18, 2021.
28.S. I. Venieris, J. F. Marques and N. D. Lane, “unzipFPGA: Enhancing FPGA-based CNN Engines with On-the-Fly Weights Generation,” 29th IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM), 2021.
29.G. Gupta, M. Kshirsagar, M. Zhong, S. Gholami and J. M. L. Ferres, “Comparing recurrent convolutional neural networks for large scale bird species classification,” Scientific Report, doi: 10.1038/s41598-021-96446-w, 2021.
30.P. Gavali and J. S. Banu, “Bird Species Identification using Deep Learning on GPU platform,” 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), pp. 1-6, 2020.
31.S. Zhang, J. Cao, Q. Zhang, Q. Zhang, Y. Zhang and Y. Wang, “An FPGA-Based Reconfigurable CNN Accelerator for YOLO,” 2020 IEEE 3rd International Conference on Electronics Technology, pp. 74-78, 2020.