人體動作辨識 ( Human Action Recognition)，對人體的任一項運動行為進行辨識，而辨識結果的準確性無疑是對於一個模型好壞最直觀的分別。危險駕駛在國內外都是一個常見的議題，對於一行駛的交通載具，駕駛人的不安全行為容易成為肇事的主因，對應日漸成熟的深度學習，本論文將著重於車內行為的分析與辨識，車內的行為辨識與目前主流的數據集有很大的不同。礙於車內的空間，駕駛無法做出大幅度的運動軌跡，此外考慮到車內的環境條件(白天、夜間行駛)，拍攝的畫面會有很大的差異，此外拍攝駕駛的攝影機位置更是重要，總體來說相比於主流數據集(UFC101、HMDB51等)，車內的每一個動作的相似度更高，除了使用光流辨識結果，我們也考慮到未來攝影機可能會包含多種不同的功能如紅外線這類的夜視影像或是辨識肢體的Open pose等相機功能，我們在實驗中混和搭配各種數據資料，模擬未來可能出現在駕駛行為上的多感測器辨識。

Human action recognition , to identify any movement behavior of the human body, and the accuracy of the recognition result is undoubtedly the most intuitive difference between the quality of a model. Dangerous driving is a common issue at home and abroad. For a driving traffic vehicle, the unsafe behavior of the driver is easy to become the main cause of the accident. Corresponding to the increasingly mature deep learning, this article focuses on this paper, and this paper will focus on the inside of the car. Behavior analysis and recognition, behavior recognition in the car is very different from the current mainstream data set. Due to the space in the car, the driver cannot make a large movement trajectory. Taking into account the environmental conditions in the car (day and night driving), the pictures taken will be very different. In addition, the location of the camera that shoots the driver is more It is important. Generally speaking, the average similarity of each action is higher compared to mainstream data sets (UFC101, HMDB51, etc.). In addition to using optical flow recognition results, we also consider that the future camera may contain many different functions such as infrared This kind of night vision image or camera recognition function such as Open pose, we mix and match various data in the experiment to simulate the multi-sensor recognition that may appear in driving behavior in the future.

摘要 I
ABSTRACT II
目錄 III
圖目錄 V
表目錄 VII
第一章 導論 1
1.1 研究動機 1
1.2 論文架構 1
第二章 文獻探討 2
2.1 光流法(Optical Flow) 2
2.2 人體行為辨識發展 3
2.2.1 Going Deeper with Conv 3
2.2.2 Learning Spatiotemporal Features with 3D Conv Networks 5
2.2.3 Two-Stream Conv Networks for Action Recognition in Videos 7
2.2.4 Conv Two-Stream Network Fusion for Video Action Recognition 8
2.2.5 Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset 11
第三章 系統架構與流程 14
3.1 系統架構 14
3.1.1 融合網路特徵 16
3.1.2 模型驗證 18
3.1.3 融合層的位置 19
3.1.4 融合後的網路處理 21
3.2 訓練流程 22
3.2.1 訓練資料集 22
3.2.2 損失函數 25
摘要 ……………………………………………………………………………………………I
ABSTRACT II
目錄 ………………………………………………………………………………………….III
圖目錄…………………………………………………………………………………………V
表目錄……………………………………………………………………………………….VII
第一章 導論 1
1.1 研究動機 1
1.2 論文架構 1
第二章 文獻探討 2
2.1 光流法(Optical Flow) 2
2.2 人體行為辨識發展 3
2.2.1 Going Deeper with Conv 3
2.2.2 Learning Spatiotemporal Features with 3D Conv Networks 5
2.2.3 Two-Stream Conv Networks for Action Recognition in Videos 7
2.2.4 Conv Two-Stream Network Fusion for Video Action Recognition 8
2.2.5 Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset 11
第三章 系統架構與流程 14
3.1 系統架構 14
3.1.1 融合網路特徵 16
3.1.2 模型驗證 18
3.1.3 融合層的位置 19
3.1.4 融合後的網路處理 21
3.2 訓練流程 22
3.2.1 訓練資料集 22
3.2.2 損失函數 25
第四章 實驗結果 26
4.1 測試資料集 26
4.2 實驗結果 27
4.2.1 模型參數設定 27
4.2.2 實驗結果 28
第五章 結論 29
參考文獻 30

[1]B. Horn and B. Schunck. “Determining optical flow“, Artificial intelligence 17.1-3 (1981): 185-203.
[2]A. Krizhevsky, I. Sutskever, G. Hinton “ImageNet classification with deep convolutional neural networks“,NIPS,2012. 1097–1105.
[3]K. He, X. Zhang, S. Ren, and J. Sun. “Deep residual learning for image recognition, “ CVPR, p. 770–778,2016.
[4]K. Simonyan, A. Zisserman. “Very deep convolutional networks for large-scale image recognition.” arXiv ,2014.
[5]C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. “Going deeper with convolutions,” CVPR , P. 1-9,2015.
[6]Ji S, Xu W, Yang M, et al. “3D convolutional neural networks for human action recognition,” IEEE Trans. on PAMI, 2013.
[7]K. Simonyan and A. Zisserman. “Two-stream convolutional networks for action recognition in videos,” arXiv, 2014
[8]F. Christoph, P. Axel, Z. Andrew, “ Convolutional Two-Stream Network Fusion for Video Action Recognition,” CVPR, P. 1933-1941,2016.
[9]C. Joao, Z. Andrew, “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset”, CVPR, P. 6299-6308,2017.
[10]M .Martin, A. Roitberg, M. Haurilet, M. Horne, M. Voit, S. Stiefelhagen, “Drive&act: A multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles”, IEEE , 2019 ,pp.2801-2810.
[11]C. BARKER.“Key findings from focus group research on inside-the-vehicle distractions in new Zealand,” In: INTERNATIONAL CONFERENCE ON THE DISTRACTIONS IN DRIVING, 2005, SYDNEY, NEW SOUTH WALES, AUSTRALIA, P. 213–254, 2007.
[12]A. K. Huemer, and M. Vollrath,”Ablenkung durch fahrfremde Tätigkeiten" Machbarkeitsstudie,2012.
[13]G. K. Sheila, A. D. Thomas, L. N. Vicki, D. S. Jeremy, J. R. David,”The impact of driver inattention on near-crash/crash risk: An analysis using the 100-car naturalistic driving study data”, 2006.
[14]G. Craig. “Driver distraction: An initial examination of the attention diverted bycontributory factor codes from crash reports and focus group research on perceived risks”, IPENZ Conference, Auckland,2005, P. 96-98.