長短期記憶(LSTM)可以解決循環神經網路(RNN)透過時間進行反向傳遞訓練時，因歷史資訊過長而產生梯度消失與梯度爆炸的問題。雙向長短期記憶(BiLSTM)可同時學習到正反兩方向時序數據時間相依性來補足只能學習到單向時序數據時間相依性的LSTM。堆疊長短期記憶(Stacked LSTM)透過堆疊更深的LSTM層以得到時間序列上更複雜的樣式。而堆疊雙向長短期記憶(Stacked BiLSTM)則是將上述兩者結合以同時學習正反兩方向時序數據時間相依性以及更複雜的樣式。有文獻分別將這些LSTM結合具有特徵擷取能力的卷積神經網路(CNN)來形成包括CNN-LSTM、CNN-BiLSTM、CNN-Stacked LSTM以及CNN-Stacked BiLSTM等模型，並應用在股票市場的預測。
CNN發展過程中衍伸出了像是早期以增加卷積深度的AlexNet，以及在同一層中利用1x1、3x3和5x5不同尺寸的卷積核來同時擷取各個尺寸特徵的Inception-v1模組。然而，在過去CNN-LSTM模型所使用的CNN架構較為簡易，因此在模型架構方面存在可嘗試探索的優化空間。因此本論文參考AlexNet架構來增加CNN-LSTM的卷積深度，還借鑒GoogLeNet Inception-v1平行卷積模組來增加CNN-LSTM卷積的寬度，提出了CNNv2-StackedNDBiLSTM和CNNv2IM-StackedNDBiLSTM兩個模型。另外，在LSTM方面，因本論文所使用的訓練數據充足，因此在模型訓練時嘗試將Dropout機制移除而衍生出CNN-StackedNDLSTM和CNN-StackedNDBiLSTM兩個CNN-LSTM訓練模型。
本論文使用近期的7.5天共41,123筆數據及相對遠期7.5天後的2.5天共7,710筆數據來進行模型測試，其中包含了每分鐘的開盤價、最高價、最低價和收盤價。實驗結果顯示，在股票市場應用中，對於近期的預測使用CNN-LSTM能進行較準確的預測，R-square為0.978，而在相對遠期的預測時，R-square則降至0.682。本論文所提出的CNNv2-StackedNDBiLSTM和CNNv2IM-StackedNDBiLSTM，在近期的預測時R-square為0.865和0.859。雖然近期時間的預測能力低於CNN-LSTM，但是在相對遠期的股票市場價格預測，R-square則優於CNN-LSTM，分別可達0.834和0.884。因此，在實際應用上可以使用CNNv2-StackedNDBiLSTM和CNNv2IM-StackedNDBiLSTM來進行相對遠期的預測，同時搭配使用CNN-LSTM來進行近期的預測，會得到較佳的整體預測結果。

Long short-term memory (LSTM) can solve the vanishing and exploding gradient problem caused by too long historical information when recurrent neural network (RNN) is trained backward through time. Bidirectional long-short-term memory (BiLSTM) can learn the temporal dependencies of time series data in both positive and negative directions to complement the LSTM that can only learn the time dependencies of time series data in one direction. Stacked long short-term memory (Stacked LSTM) obtains more complex patterns in time series by stacking multiple LSTM layers. Stacked bidirectional long-short-term memory (Stacked BiLSTM) combines the above two to simultaneously learn the time dependencies and more complex patterns of time series data in both positive and negative directions. There are literatures that combine these LSTMs with convolutional neural networks (CNNs) with feature extraction capabilities to form models including CNN-LSTM, CNN-BiLSTM, CNN-Stacked LSTM, and CNN-Stacked BiLSTM, and CNN-Stacked BiLSTM, and apply them to stock market predictions.
During the development of CNN, AlexNet, which was used to increase the convolution depth in the early stage, and the Inception-v1 module, which uses 1x1, 3x3, and 5x5 convolution kernels of different sizes in the same layer to simultaneously extract features of each size. However, in the past, the CNN architecture used by the CNN-LSTM model was relatively simple, so there is room for optimization in the model architecture. Therefore, this paper refers to the AlexNet architecture to increase the convolution depth of CNN-LSTM, and also uses the GoogLeNet Inception-v1 parallel convolution module to increase the convolution width of CNN-LSTM, and proposes two models: CNNv2-StackedNDBiLSTM and CNNv2IM-StackedNDBiLSTM. In addition, in terms of LSTM, because the training data used in this paper is sufficient, the Dropout mechanism was removed during model training to derive two CNN-LSTM training models, CNN-StackedNDLSTM and CNN-StackedNDBiLSTM.
This paper uses a total of 41,123 data in the recent 7.5 days and a total of 7,710 data in 2.5 days after 7.5 days to test the model, which includes the opening price, highest price, lowest price and closing price per minute. The experimental results show that in the stock market application, CNN-LSTM can make more accurate predictions for the short-term predictions, and the R-square is 0.978, while for the relatively long-term predictions, the R-square drops to 0.682. The CNNv2-StackedNDBiLSTM and CNNv2IM-StackedNDBiLSTM proposed in this paper have recently predicted R-squares of 0.865 and 0.859. Although the prediction ability in the near future is lower than that of CNN-LSTM, in the relatively long-term stock market price prediction, R-square is better than CNN-LSTM, which can reach 0.834 and 0.884 respectively. Therefore, in practical applications, CNNv2-StackedNDBiLSTM and CNNv2IM-StackedNDBiLSTM can be used for relatively long-term predictions, and at the same time, CNN-LSTM can be used for short-term predictions, and better overall prediction results will be obtained.

中文摘要 ............................................................................................................... i
英文摘要 .............................................................................................................. ii
誌謝 ..................................................................................................................... iv
目錄 ...................................................................................................................... v
表目錄 ................................................................................................................ vii
圖目錄 ............................................................................................................... viii
第一章 緒論 ........................................................................................................ 1
第二章 研究理論介紹 ........................................................................................ 3
2-1 LSTM模型介紹 ................................................................................... 3
2-1-1 LSTM .................................................................................................. 3
2-1-2 Bidirectional LSTM ............................................................................ 5
2-1-3 Stacked LSTM ..................................................................................... 6
2-1-4 Stacked Bidirectional LSTM ............................................................... 6
2-2 2D CNN 模型介紹 ............................................................................... 8
2-2-1 CNN .................................................................................................... 8
2-2-2 AlexNet ............................................................................................... 8
2-2-3 GoogLeNet .......................................................................................... 9
2-3 1D CNN-LSTM模型介紹 .................................................................. 12
2-3-1 1D CNN-LSTM ................................................................................. 12
2-3-2 1D CNN-Bidirectional LSTM ........................................................... 14
2-3-3 1D CNN-Stacked LSTM ................................................................... 15
2-3-4 1D CNN-Stacked Bidirectional LSTM ............................................. 16
2-4 評估參數介紹 ..................................................................................... 17
2-4-1 平均絕對誤差(MAE) ....................................................................... 17
2-4-2 平均絕對百分比誤差(MAPE) ......................................................... 17
2-4-3 均方誤差(MSE) ................................................................................ 17
2-4-4 均方根誤差(RMSE) ......................................................................... 18
2-4-5 R-square (푅2) .................................................................................... 18
第三章 模型設計與訓練之優化 ...................................................................... 19
3-1 Stacked LSTM without Dropout ......................................................... 19
3-2 Stacked Bidirectional LSTM without Dropout ................................... 20
3-3 CNN-Stacked LSTM without Dropout ............................................... 21
3-4 CNN-Stacked BiLSTM without Dropout ........................................... 22
3-5 CNNv1-Stacked BiLSTM without Dropout ....................................... 23
3-6 CNNv2-Stacked BiLSTM without Dropout ....................................... 24
3-7 CNNv2 Inception module-Stacked BiLSTM without Dropout .......... 26
第四章 實驗介紹與結果討論 .......................................................................... 28
4-1 股票市場數據集 ................................................................................. 28
4-2 模型訓練硬體與環境介紹 ................................................................. 28
4-3 模型參數 ............................................................................................. 28
4-4 模型訓練結果與分析 ......................................................................... 36
第五章 結論 ...................................................................................................... 49
參考文獻 ............................................................................................................ 50
Extended Abstract

[1] Werbos, P. J. (1974). Beyond regression: New tools for prediction and analysis in the behavioral sciences. (Doctoral dissertation). Harvard University.
[2] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.
[3] Graves, A., Mohamed, A. R., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6645-6649).
[4] Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681.
[5] Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research, 11, 3371-3408.
[6] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (pp. 3104-3112).
[7] Zhou, P., Qi, Z., Zheng, S., Xu, J., Bao, H., & Xu, B. (2016, November 21). Text Classification Improved by Integrating Bidirectional LSTM with Two-Dimensional Max Pooling. ArXiv. Retrieved from https://arxiv.org/abs/1611.06639
[8] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
[9] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097-1105).
[10] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-9).
[11] Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 1746-1751).
[12] Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of the Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (pp. 103-111).
[13] Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (pp. 655-665).
[14] Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level Convolutional Networks for Text Classification. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 649-657).
[15] Zeng, D., Liu, K., Lai, S., Zhou, G., & Zhao, J. (2015). Relation classification via convolutional deep neural network. In Proceedings of the 25th International Conference on Computational Linguistics (pp. 2335-2344).
[16] Ma, X., & Hovy, E. (2016). End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (pp. 1064-1074).
[17] https://finance.yahoo.com/quote/MGLU3.SA?p=MGLU3.SA