臺灣博碩士論文加值系統

本論文主要實現人體動態識別(Human Activity Recognition，HAR)辨識人體短時間內作出的動態姿勢，舉例來說，例如:跌倒、跳躍等及本文以籃球運動作為範例中的運球、射籃動作。本次系統主要使用開源軟體OpenPose由卡內基梅隆大學(CMU)所發表基於卷積神經網路(Convolution Neural Network，CNN)開發的人體姿勢估計(Human Pose Estimation，HPE)模型偵測出人的身體、臉部、手腳等關鍵點的座標位置[1]，而動態動作是連續性的資料因此結合處理時間序列的長短期記憶神經網路（Long Short-Term Memory，LSTM）進行人體動態辨識。最終實驗結果模型準確率達92%驗證本文假設理論是可行，此項研究解決過往單獨使用CNN的物件偵測只能辨認靜態姿勢的問題及未來可以適用於老人照護偵測摔倒、跌倒等異常行為亦或是用於運動賽事上。

This paper mainly implements Human Activity Recognition (HAR) to recognize the dynamic action made by the human in a short time. For example, such as falling, jumping and this paper use dribbling and shooting of basketball as an example. This system mainly uses the open source OpenPose of Human Pose Estimation (HPE) model developed by Carnegie Mellon University (CMU) based on Convolution Neural Network (CNN). OpenPose can detect the coordinate positions of key points of the human, such as body, face, hand and feet, etc. The dynamic action is continuous data, so combined with Long Short-Term Memory (LSTM) that can process the time series data to realize HAR. The final experimental results showed that the model accuracy rate reached 92%, which verified that the hypothesis in this paper is feasible. This research solves the problem that object detection using CNN alone can only recognize static action in the past. It can be applied to elderly care, or used in sports events in the future.

摘要...i
Abstract...ii
誌謝...iii
目錄...iv
表目錄...vi
圖目錄...vii
第一章 緒論...1
1.1 研究動機...1
1.2 研究目的...3
1.3 研究流程...4
第二章 相關技術與文獻探討...5
2.1 人體動態識別(HAR)實現相關文獻...5
2.1.1 使用固定式穿戴裝置實現HAR...5
2.1.2 使用人體姿勢估計模型實現HAR...5
2.2 人體姿勢估計(HPE)相關文獻...6
2.2.1 AlphaPose...6
2.2.2 DeepCut & DeeperCut...7
2.2.3 Pose Machine...8
2.2.4 Convolutional Pose Machines...8
2.2.5 OpenPose...9
2.2.6 不同人體姿勢模型估計比較...10
2.3 深度學習神經網路...12
2.3.1 Convolutional Neural Network...12
2.3.2 Recurrent Neural Network...15
2.3.3 Long Short-Term Memory...15
2.4 深度學習相關套件...17
2.4.1 Tensorflow...17
2.4.2 Keras...17
2.4.3 Scikit-Learn...18
2.4.4 Matplotlib...18
2.5 深度學習模型評估指標...19
第三章 研究方法與架構...22
3.1 主要系統架構與流程...22
3.2 滑動框...23
3.3 辨識系統－HPE 系統...24
3.4 資料標記...25
3.5 資料前處理...26
3.5.1 資料量倍增放...26
3.5.2 標準化...28
3.6 辨識系統－HAR 系統...30
3.7 總體應用架構...32
第四章 實驗與結果...33
4.1 實驗環境架設需求...33
4.2 資料收集...34
4.3 資料標準化成效...37
4.4 模型效能評估...38
4.5 實際應用結果...40
第五章 結論與未來研究...42
5.1結論...42
5.2未來研究...42
參考文獻...44
Extended Abstract...46

[1] Zhe,C., Gines,H., Tomas,S., Shih-En,W. & Yaser,S. (2019) OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence. doi: 10.1109/tpami.2019.2929257
[2] UCI Machine Learning Repository Human Activity Recognition Using Smartphones DataSet ，線上檢索時間:2021年2月25日，檢自網址: https://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones
[3] Davide, A., Alessandro,G., Luca,O., Xavier,P., & Jorge L. Reyes-Ortiz. (2012) Human Activity Recognition on Smartphones using a Multiclass Hardware- Friendly Support Vector Machine. 4th International Workshop of Ambient Assited Living, IWAAL 2012 Lecture Notes in Computer Science 2012, pp 216-223.
[4] Github作者stuarteiffert — RNN for Human Activity Recognition - 2D Pose Input，線上檢索時間:2021年3月2日，檢自網址: https://github.com/stuarteiffert/RNN-for-Human-Activity-Recognition-using-2D-Pose-Input
[5] Github作者Guillaume Chevalier—LSTMs for Human Activity Recognition，線上檢索時間:2021年3月9日，檢自網址: https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition
[6] Hao-Shu,F., Shuqin,X., Yu-Wing,T.& Cewu,L. (2017) RMPE: Regional Multi-person Pose Estimation. 2017 IEEE International Conference on Computer Vision. (ICCV) doi: 10.1109/iccv.2017.256
[7] Leonid,P., Eldar,I., Siyu,T., Bjoern,A., Mykhaylo,A., Peter,G.& Bernt, S.(2015) DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation. Computer Vision and Pattern Recognition. arXiv:1511.06645
[8] Eldar,I., Leonid,P., Bjoern,A., Mykhaylo,A.& Bernt,S.(2016) DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model. Computer Vision and Pattern Recognition. arXiv:1605.03170
[9] Varun,R., Daniel,M., Martial,H., J. Andrew, B. & Yaser,S.(2014) Pose Machines: Articulated Pose Estimation via Inference Machines. In ECCV, 2014.
[10] Shih-En,W., Varun,R., Takeo,K.& Yaser,S.(2016) Convolutional Pose Machines Computer Vision and Pattern Recognition. arXiv:1602.00134
[11] Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P.(1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE. doi:10.1109/5.726791
[12] An Intuitive Explanation of Convolutional Neural Networks，線上檢索時間:2021年3月14日，檢自網址: https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/
[13] 李宏毅教授-RNN課程影片(Part I)重點整理-Part II，線上檢索時間:2021年3月14日，檢自網址: https://medium.com/@s716419/%E6%9D%8E%E5%AE%8F%E6%AF%85%E6%95%99%E6%8E%88-rnn%E8%AA%B2%E7%A8%8B%E5%BD%B1%E7%89%87-part-i-%E9%87%8D%E9%BB%9E%E6%95%B4%E7%90%86-part-ii-2fadc5ad62a6
[14] 李宏毅教授個人網站2017/11/03課程Recurrent Neural Network PDF，線上檢索時間:2021年3月14日，檢自網址: http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/RNN%20(v2).pdf
[15] Tensorflow，線上檢索時間:2021年3月11日，檢自網址: https://www.tensorflow.org/
[16] Tensorflow Github，線上檢索時間:2021年3月11日，檢自網址: https://github.com/tensorflow/tensorflow
[17] Keras，線上檢索時間:2021年3月11日，檢自網址: https://keras.io/about/
[18] Scikit-Learn，線上檢索時間:2021年3月11日，檢自網址: https://scikit-learn.org/stable/index.html
[19] Matplotlib，線上檢索時間:2021年3月11日，檢自網址: https://matplotlib.org/
[20] Confusion Matrix for Your Multi-Class Machine Learning Model，線上檢索時間:2021年3月16日，檢自網址: https://towardsdatascience.com/confusion-matrix-for-your-multi-class-machine-learning-model-ff9aa3bf7826
[21] 如何辨別機器學習模型的好壞？秒懂Confusion Matrix，線上檢索時間:2021年3月16日，檢自網址: https://www.ycc.idv.tw/confusion-matrix.html