臺灣博碩士論文加值系統

隨著科技的日新月異，人工智慧的技術已廣泛應用在我們的生活中，如：車牌辨識、自動駕駛、聊天機器人等。隨著近幾年疫情的影響，人工智慧在抗疫的議題中也占據了相當重要的地位，最知名的例子莫過於 AI 光達人流感測系統。在疫情期間，大眾對於「接觸」這個動作可說是避之唯恐不及，在公共場合中，前一人接觸過的設施或物品，在使用前都會拿酒精出來消毒一下再使用，但如果是觸碰式螢幕則無法使用酒精做消毒的動作。有鑒於此，以老師上課使用電腦播放簡報為例，開發一種系統，利用鏡頭偵測老師比的動作，判斷此時該切換簡報的上一頁或下一頁，使老師無須觸碰到硬體設備，即可切換簡報的頁數，而本論倫研製的系統亦是如此。本論文欲針對手勢辨識的部分進行改良，運用MediaPipe 與 Python結合Realsense深度攝影機實作一個手勢辨識的軟體。

With the rapid development of science and technology, artificial intelligence technology has been widely used in our lives, such as license plate recognition, automatic driving, chatbot, etc. With the impact of the pandemic in recent years, artificial intelligence has also occupied a very important position in the topic of anti-pandemic. The most famous example is the AI light sensor human flu detection system. During the pandemic, the public is afraid of the action of "touching". In public places, the facilities or objects that the previous person has touched will be disinfected with alcohol before use, but if it is touching Alcohol cannot be used to disinfect the screen. In view of this, taking the example of a teacher using a computer to play a presentation in class, a system was developed that uses the camera to detect the movement of the teacher, and judges whether to switch to the previous or next page of the presentation at this time, so that the teacher does not need to touch the hardware device to switch the number of pages in the presentation. This study intends to improve the gesture recognition part, using Realsense, Python language and Mediapipe library to implement gesture recognition software.

ABSTRACT i
摘要 ii
Table of Contents iii
List of figures v
List of Tables vii
1 Chapter 1 INTRODUCTION 1
1.1 Research Background 1
1.2 Research Motivation 2
1.3 Research Question 3
2 Chapter 2 LITERATURE REVIEW 4
2.1 Gesture Recognition 4
2.2 3D Sensing Technology 7
2.3 MediaPipe 17
2.4 Control Slide 20
3 Chapter 3 METHODOLOGY 24
3.1 Research Framework 24
3.2 Research Procedure 26
3.2.1 OpenCV Capture webcam streaming image 26
3.2.2 Autopy method instructions and perform action judgment 28
3.3 Expected Results 29
4 Chapter 4 SYSTEM IMPLEMENTATION AND RESULTS 32
4.1 System data 32
4.1.1 Pyrealsense2 and cv2 32
4.1.2 Numpy 33
4.1.3 HandTrackingModule 33
4.1.4 Autopy 34
4.1.5 Time 35
4.1.6 Mediapipe 36
4.2 System Process 36
4.2.1 Environment 36
4.2.2 Procedure 38
5 Chapter5 ANALYSIS AND DISCUSSION 42
5.1 Experimental results 42
5.1.1 Experiment summary 42
5.1.2 Experimental effect evaluation 42
5.2 Discussion 42
5.2.1 Experimental contributions and limitations 42
5.2.2 Differences in using traditional lenses versus depth lenses 44
5.2.3 Experimental contribution 46
6 .Chapter6 CONCLUSION 47
REFERENCES 50

1.Sharma, S., Jain, S., and Khushboo (2019), "A static hand gesture and face recognition system for blind people," 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN), India, Amity University, 534-539.
2.Pakalapati, S. S., Chary, G. G., Yadaw, A. K., Kumar, S., Phulawariya, H. K., and Kumar, R. (2017), "A prosthetic hand control interface using esp8266 wi-fi module and android application," 2017 International Conference on Innovations in information Embedded and Communication Systems (ICIIECS), Coimbatore, India.
3.Damindarov, R., Fam, C. A., Boby, R. A., Fahim, M., Klimchik, A. and Matsumaru, T. (2021), "A depth camera-based system to enable touch-less interaction using hand gestures," 2021 International Conference "Nonlinearity, Information and Robotics" (NIR), Innopolis, Russia.
4.Escopete, M. C. C., Laluon, C. G., Llarena, E. M., Reyes Jr, P. M. and Tolentino R. E. (2021), "Recognition of English Capital Alphabet in Air writing Using Convolutional Neural Network and Intel RealSense D435 Depth Camera," 2021 2nd Global Conference for Advancement in Technology (GCAT), Bangalore, India, 1-8.
5.Prasad, N., Unnikrishnan.K, Prof. and Jayakrishnan R, Mr. (2018), "Fraud Detection by Facial Expression Analysis Using Intel RealSense and Augmented Reality," Proceedings of the Second International Conference on Intelligent Computing and Control Systems (ICICCS 2018), Vaigai College Engineering, India, 919-923.
 
6.Huang, K., Tang, J., Yuan, J. and Dong, Z. (2020), "Weld Line Detection using RealSense Depth Camera based on Depth Map," 2020 IEEE 9th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China, 595-600.
7.Yuan, X., Xiao, Z., Zhu, Z. and Li, S. (2019), "Humanoid robot shadow system based on kinect depth camera detection," Proceedings 2019 Chinese Automation Congress (CAC2019), Hangzhou, China, 4999-5003.
8.Yang, D., Wu, H., Zhao, X.G., Su, C. and Xu, X. X. (2022), "Research of target detection and distance measurement technology based on YOLOv5 and depth camera," 2022 4th International Conference on Communications, Information System and Computer Engineering (CISCE), Shenzhen, China, 346-349.
9.Križnar, V., Leskovšek, M. and Batagelj, B. (2021), "Use of computer vision based hand tracking in educational environments," 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 804-809.
10.Boruah, B. J., Talukdar, A. K. and Sarma, K. K. (2021), "Development of a learning-aid tool using hand gesture based human computer interaction system," 2021 Advanced Communication Technologies and Signal Processing (ACTS), Rourkela, India.
11.Praditasari, A. A. W., Aprilliyani, R. and Kholis, I. (2021), "Design and implementation of interactive virtual museum based on hand tracking opencv in Indonesia," 2021 8th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Palembang, Indonesia, 253-256.
 
12.Al-Nuimi, A.M. and Mohammed, G. J. (2021), "Face direction estimation based on mediapipe landmarks," 2021 7th International Conference on Contemporary Information Technology and Mathematics (ICCITM), Mosul, Iraq, 185-190.
13.Sreenath, S., Daniels, D. I., Ganesh, A. S. D., Kuruganti, Y. S. and Chittawadigi, R. G. (2021), "Monocular Tracking of Human Hand on a SmartPhone Camera using MediaPipe and its Application in Robotics," 2021 IEEE 9th Region 10 Humanitarian Technology Conference (R10-HTC), Bangalore, India.
14.Osipov, A. and Ostanin, M. (2021), "Real-time static custom gestures recognition based on skeleton hand," 2021 International Conference "Nonlinearity, Information and Robotics" (NIR), Innopolis, Russia.
15.Charan, C. S., Meenakshi, K., Reddy, V. B. and Kashyap, V. (2023), "Controlling PowerPoint Presentation using Hand Gestures in Real-Time," 2023 7th International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, 251-254.
16.Osunkoya, T. and Chern, J-C. (2013), "Gesture-Based Human-Computer-Interaction Using Kinect for Windows Mouse Control and PowerPoint Presentation," 2013 46th Midwest Instruction and Computing Symposium (MICS 2013), Chicago State University, Chicago