智慧城市提高了城市地區的效率、永續性和宜居性，從而促進經濟成長並提高公民的生活水平。這使得智慧城市對於國家的發展至關重要。在全球智慧城市領域，整合智慧交通技術對於改善城市和確保高效的交通管理至關重要。動態交通流和管理系統是這些創新的關鍵，它使用即時數據來監控和優化交通狀況。本研究旨在透過深度學習的應用，創建一個能夠自動辨識違規行為的系統。它不僅檢查交通流的檢測和分析，還研究人類行為實例的識別。 車輛辨識、車牌偵測、字元分類都是透過YOLO、WPOD-NET、PaddleOCR等深度學習技術完成的。透過採用先進的偵測技術和違規分析機制，這些系統將能夠在各種環境設定和天氣條件下正確識別車輛和違規行為。這項研究是在越南進行和分析的，與其他東南亞國家相比，越南是一個事故頻繁的國家。此外，研究還建議識別各種交通違規行為，例如闖紅燈、在未經授權的區域停車、超速行駛以及在錯誤車道上行駛。當應用於日常資料集時，實驗的準確率達到了 95% 以上。

The efficiency, sustainability, and livability of urban areas are improved by smart cities, which boost economic growth and raise living standards for citizens. This makes smart cities essential to a nation's development. In the global field of smart cities, integrating intelligent transportation technologies is critical to improving urban mobility and assuring efficient traffic management. Dynamic traffic flow and management systems are the keys to these innovations, using real-time data to monitor and optimize traffic conditions. Through the application of deep learning, the purpose of this research is to create a system that can automatically identify violations. Not only does it examine the detection and analysis of traffic flow, but it also looks into the identification of instances of human behavior. The identification of vehicles, the detection of license plates, and the classification of characters are all accomplished through the utilization of deep learning techniques like YOLO, WPOD-NET, and PaddleOCR. By employing advanced detection technologies and violation analysis mechanisms, these systems will be able to properly identify vehicles and violations across a wide range of ambient settings and weather conditions. The research was conducted and analysis in Vietnam, a country that has a big cause for concern accidents in comparison to other Southeast Asian countries. Additionally, the research suggests recognizing a variety of traffic violations, such as those that include running red lights or parking in unauthorized areas, over limitation, speed, driving in wrong lane. When applied to the daily dataset, the experiment was successful in achieving an accuracy of more than 95%.

Abstract ...................................................................................................................... i
摘要.......................................................................................................................... iii
Acknowledgments......................................................................................................v
Table of Contents..................................................................................................... vi
List of Tables ........................................................................................................... ix
List of Figures.......................................................................................................... x
Meaning................................................................................................................... xii
Chapter 1 Introduction ...............................................................................................1
1.1 Research Background........................................................................................1
1.2 Research Objective............................................................................................3
1.3 Research Structure ............................................................................................4
Chapter 2 Literature Review......................................................................................6
2.1 Background of vehicle detection.......................................................................6
2.2 Background of vehicle tracking ......................................................................10
2.3 Background of license plates extraction and characters classification...........12
2.3.1 License plates extraction by using WPOD-NET ......................................12
2.3.2 Characters classification by using Paddle OCR........................................13
Chapter 3 Research Methodology............................................................................16
3.1 Vehicle Detection............................................................................................17
3.1.1 YOLO........................................................................................................17
3.1.2 IOU Calculation Function .........................................................................19
3.1.3 Loss Function ............................................................................................20
3.1.4 YOLOv7....................................................................................................24
3.1.5 YOLOv8....................................................................................................28
3.1.6 YOLOv9....................................................................................................31
3.1.7 YOLOv10..................................................................................................34
3.2 Vehicle tracking DeepSORT ..........................................................................37
3.3 Violation Recognition .....................................................................................41
3.3.1 Rule in Vietnam and Taiwan.....................................................................41
3.3.2 Vehicle Speed Limit Violation Detection.................................................44
3.3.3 Going at the wrong traffic light.................................................................46
3.3.4 Illegal Parking ...........................................................................................48
3.3.5 Driving in the wrong lane .........................................................................49
3.3.6 No helmet during moving on the road ......................................................51
3.3.7 Carries more person specified...................................................................53
3.4 License Plate Extraction..................................................................................54
3.4.1 Vehicle license Plate Extraction With WPOD-NET .................................54
3.4.2 Character Classification with PaddleOCR................................................59
Chapter 4 Results and Discussion............................................................................62
4.1 Dataset.............................................................................................................62
4.2 Vehicle Detection............................................................................................65
4.3 Vehicle Tracking.............................................................................................73
4.4 Violation detection ..........................................................................................76
4.4.1 Speed Analysis..........................................................................................76
4.4.2 Going at wrong traffic light.......................................................................79
4.4.3 Illegal parking area....................................................................................80
4.4.4 Driving in the wrong lane .........................................................................83
4.4.5 Wrong-lane for person ..............................................................................85
4.4.6 No helmets.................................................................................................87
4.4.7 Carries more person than specified...........................................................88
4.5 License Plate Extraction Results.....................................................................89
4.5.1 WPOD-NET..............................................................................................89
4.5.2 Paddle OCR...............................................................................................90
Chapter 5 Conclusions and Future Work.................................................................96
5.1 Conclusions.....................................................................................................96
5.2 Future work .....................................................................................................97
References................................................................................................................99
Extend Abstract......................................................................................................109

[1]Azimjonov, J.; Özmen, A.; Kim, T. A Nighttime Highway Traffic Flow Monitoring System Using Vision-Based Vehicle Detection and Tracking. Soft Computing 2023, 27, 13843–13859
[2]Azimjonov, J., & Özmen, A. (2021). A real-time vehicle detection and a novel vehicle tracking systems for estimating and monitoring traffic flow on highways. Advanced Engineering Informatics, 50, pp.101393.
[3]Mohamad, F. F., Abdullah, A. S., Mohamad, J., & Karim, M. R. (2018). Understanding of speed behavior in relation to road traffic accident: a comparison between Malaysian and Vietnamese drivers. Malaysian Journal of Civil Engineering, 30(1).
[4]Vu, A. T., & Nguyen, D. V. M. (2018). Analysis of child-related road traffic accidents in Vietnam. IOP Conference Series. Earth and Environmental Science, 143, 012074.
[5]Hung, K. V., & Huyen, L. T. (2011). Education influence in traffic safety: A case study in Vietnam. IATSS Research, 34(2), 87–93.
[6]Khan, N., Bano, A., & Zandi, P. (2018). Effects of exogenously applied plant growth regulators in combination with PGPR on the physiology and root growth of chickpea (Cicer arietinum) and their role in drought tolerance. Journal of Plant Interactions, 13(1), 239–247.
[7]Jeon, H. J., Lee, Y. H., Kim, M. J., Choi, S. D., Park, B. J., & Lee, S. E. (2016). Integrated biomarkers induced by chlorpyrifos in two different life stages of zebrafish (Danio rerio) for environmental risk assessment. Environmental Toxicology and Pharmacology, 43, 166–174.
[8]Dang, T. P., Tran, N. T., To, V. H., & Thi, M. K. T. (2023). Improved YOLOv5 for real-time traffic signs recognition in bad weather conditions. the Journal of Supercomputing/Journal of Supercomputing, 79(10), 10706–10724.
[9]Van Pham, H., & Lee, B. R. (2015). Front-view car detection and counting with occlusion in dense traffic flow. International Journal of Control, Automation, and Systems/International Journal of Control, Automation, and Systems, 13(5), 1150–1160.
[10]Tran Yen, Cuong Do & Kieu, Minh. (2015). Urban traffic management utilizing intelligent transport system (ITS) in Vietnam: an overview of opportunities and challenges.. Transport Magazine.
[11]Cypto, J., & Karthikeyan, P. (2022). Automatic detection system of speed violations in a traffic based on deep learning technique. Journal of Intelligent & Fuzzy Systems, 43(5), 6591–6606.
[12]Zhang, C., Patras, P., & Haddadi, H. (2019). Deep learning in mobile and wireless Networking: a survey. IEEE Communications Surveys and Tutorials/IEEE Communications Surveys and Tutorials, 21(3), 2224–2287.
[13]Charran, R. S., & Dubey, R. K. (2022). Two-Wheeler vehicle traffic violations detection and automated ticketing for Indian Road scenario. IEEE Transactions on Intelligent Transportation Systems, 23(11), 22002–22007.
[14]Chaturvedi, P., Lavingia, K., & Raval, G. (2023). Detection of traffic rule violation in University campus using deep learning model. International Journal of System Assurance Engineering and Management, 14(6), 2527–2545.
[15]Shin, H. C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Summers, R. M. (2016). Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298.
[16]Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.
[17]Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection.
[18]Redmon, J., & Farhadi, A. (2017). YOLO9000: Better, Faster, Stronger.
[19]Du, J. (2018). Understanding of Object Detection Based on CNN Family and YOLO. Journal of Physics. Conference Series, 1004, 012029.
[20]Zuraimi, M. a. B., & Zaman, F. H. K. (2021). Vehicle Detection and Tracking using YOLO and DeepSORT.
[21]Blackman, S. S., Dempster, R. J., & Reed, R. W. (2001). Demonstration of multiple-hypothesis tracking (MHT) practical real-time implementation feasibility. Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE.
[22]Habtemariam, B., Tharmarasa, R., Thayaparan, T., Mallick, M., & Kirubarajan, T. (2013). A Multiple-Detection Joint Probabilistic Data Association Filter. IEEE Journal of Selected Topics in Signal Processing, 7(3), 461–471.
[23]Wojke, N.; Bewley, A.; Paulus, D. (2017). Simple Online and Realtime Tracking with a Deep Association Metric. 2017 IEEE International Conference on Image Processing (ICIP).
[24]Jia, W., & Xie, M. (2023). An Efficient License Plate Detection Approach With Deep Convolutional Neural Networks in Unconstrained Scenarios. IEEE Access, 11, 85626–85639.
[25]Zhang, Z., & Wan, Y. (2019). Improving the Accuracy of License Plate Detection and Recognition in General Unconstrained Scenarios. 2019 IEEE Symposium Series on Computational Intelligence (SSCI) 2019
[26]Silva, S. M., & Jung, C. R. (2018). License plate detection and recognition in unconstrained scenarios. In Lecture notes in computer science (pp. 593–609).
[27]Hassan, A., Ali, M., Durrani, M. N., & Tahir, M. A. (2022). Vehicle Recognition Using Multilevel Deep Learning Models. In Communications in computer and information science, pp. 101–113.
[28]Liao, M., Wan, Z., Yao, C., Chen, K., & Bai, X. (2020). Real-Time Scene Text Detection with Differentiable Binarization. Proceedings of the . . . AAAI Conference on Artificial Intelligence, 34(07), pp.11474–11481.
[29]Mullick, K., & Namboodiri, A. M. (2017). Learning deep and compact models for gesture recognition.
[30]Gromova, K., & Elangovan, V. (2022). Automatic Extraction of Medication Information from Cylindrically Distorted Pill Bottle Labels. Machine Learning and Knowledge Extraction, 4(4), 852–864.
[31]Monteiro, G., Camelo, L., Aquino, G., De a Fernandes, R., Gomes, R., Printes, A., Figueiredo, C. (2023). A Comprehensive Framework for Industrial Sticker Information Recognition Using Advanced OCR and Object Detection Techniques. Applied Sciences, 13(12), 7320.
[32]Shanbin, L., Haoyu, W., & Junhao, Z. (2022). Electrical Cabinet Wiring Detection Method Based on Improved YOLOv5 and PP-OCRv3.
[33]Zubil, M. S. I. M., Embi, Z. C., & Ghauth, K. I. (2024). Assessing the Efficiency of Deep Learning Methods for Automated Vehicle Registration Recognition for University Entrance. Journal of Informatics and Web Engineering, 3(2), 57–69.
[34]Li, S., Luo, Y., Sun, K., Yadav, N., & Choi, K. K. (2020). A Novel FPGA Accelerator Design for Real-Time and Ultra-Low Power Deep Convolutional Neural Networks Compared with Titan X GPU. IEEE Access, 8, 105455–105471.
[35]Chaudhary, D. H. (2020). An overview of You Only Look Once: Unified, real-time object detection. International Journal for Research in Applied Science and Engineering Technology, 8(6), 607–609.
[36]Wang, Y., Zhang, Z., & Yin, H. (2020). Detection method of dense bridge disease targets based on SE-YOLOv3. Journal of Physics. Conference Series, 1544(1), 012141.
[37]Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019b). Generalized Intersection Over Union: A Metric and a Loss for Bounding Box
[38]Ullah, M. B., & Ullah, M. B. (2020). CPU Based YOLO: A Real Time Object Detection Algorithm. 2020 IEEE Region 10 Symposium (TENSYMP).
[39]Tayebi, R. M., Mu, Y., Dehkharghanian, T., Ross, C., Sur, M., Foley, R.Campbell, C. J. V. (2022). Automated bone marrow cytology using deep learning to generate a histogram of cell types. Communications Medicine, 2(1).
[40]Soudy, M., Afify, Y., & Badr, N. (2022). RepConv: A novel architecture for image scene classification on Intel scenes dataset. International Journal of Intelligent Computing and Information Sciences 0(0), 1–11.
[41]Pan, F., Yang, Y., Zhang, L., Ma, C., Yang, J., & Zhang, X. (2020). Analysis of the Impact of Traffic Violation Monitoring on the Vehicle Speeds of Urban Main Road: Taking China as an Example. Journal of Advanced Transportation, pp.1–11.
[42]Tang, J., Zhou, H., Wang, T., Jin, Z., Wang, Y., & Wang, X. (2022). Cascaded foreign object detection in manufacturing processes using convolutional neural networks and synthetic data generation methodology. Journal of Intelligent Manufacturing, 34(7), 2925–2941.
[43]Chan, K. Y., Dillon, T. S., Singh, J., & Chang, E. (2012). Neural-Network-Based Models for Short-Term Traffic Flow Forecasting Using a Hybrid Exponential Smoothing and Levenberg–Marquardt Algorithm. IEEE Transactions on Intelligent Transportation Systems, 13(2), 644–654
[44]Aljelawy, Q. M., & Salman, T. M. (2023). License plate recognition in slow motion vehicles. Bulletin of Electrical Engineering and Informatics, 12(4), 2236–2244.
[45]Yan, Z., Alon, A. S., Alejandro, L. L., & Vergara, C. I. (2023). An Intelligent Parking Lot Management System Based on Real-Time License Plate Recognition.
[46]Liao, M., Zou, Z., Wan, Z., Yao, C., & Bai, X. (2023). Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 919–931.
[47]Du, Y., Chen, Z., Jia, C., Yin, X., Zheng, T., Li, C., . . . Jiang, Y. G. (2022). SVTR: Scene Text Recognition with a Single Visual Model. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence.
[48]Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., & Ding, G. (2024). YOLOv10: Real-Time End-to-End Object Detection.
[49]Ivašić-Kos, M., Krišto, M., & Pobar, M. (2019). Human Detection in Thermal Imaging Using YOLO
[50]Ahmad, J., Akram, S., Jaffar, A., Ali, Z., Bhatti, S. M., Ahmad, A., & Rehman, S. U. (2024). Deep learning empowered breast cancer diagnosis: Advancements in detection and classification.
[51]Li, S., Huang, H., Meng, X., Wang, M., Li, Y., & Xie, L. (2023). A Glove-Wearing Detection Algorithm Based on Improved YOLOv8. Sensors, 23(24), 9906.
[52]Wang, H., Yang, Q., Zhang, B., & Gao, D. (2024). Deep Learning based insulator fault detection algorithm for power transmission lines. Journal of Real-Time Image Processing.
[53]Liu, C. M., & Juang, J. C. (2021). Estimation of Lane-Level Traffic Flow Using a Deep Learning Technique. Applied Sciences, 11(12), 5619.
[54]Gonzalez, A. J., Leigh, J., DeMara, R. F., Johnson, A., Jones, S., Lee, S., Kobosko, S. (2013). Passing an Enhanced Turing Test – Interacting with Lifelike Computer Representations of Specific Individuals. Journal of Intelligent Systems, 22(4), 365–415.
[55]Li, C., Liu, W., Guo, R., Yin, X., Jiang, K., Du, Y., Du, Y., Zhu, L., Lai, B., Hu, X., Yu, D., & Ma, Y. (2022). More Attempts for the Improvement of Ultra Lightweight OCR System.
[56]Xu, Y., Du, L., Zhang, W., Sun, Z., Wang, Y., & Luo, P. (2022). PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System. arXiv.