臺灣博碩士論文加值系統

股票市場中，投資人的情緒容易引起股市快速波動，因此在股票預測中，投資人情緒是一個重要的參考因素。本論文為研究台灣社群情緒與台灣股市漲跌關係，基於BERT使用台灣股市新聞進行領域自適應預訓練，建立專精於台灣金融領域的Chinese FinBERT模型，並使用台灣社群討論平台「PTT」股票版的每日討論內容做為台股社群情緒參考資料對Chinese FinBERT模型進行微調，建立Daily FinSentiment模型分析社群情緒並計算情緒分數做為預測股市漲跌的參考特徵。本論文提出CNN-BiLSTM-SA模型使用股票價格及情緒分數做為訓練資料進行股票漲跌預測，此模型使用CNN-BiLSTM-AM模型進行改良，將Attention Mechanism(AM)架構替換為Self Attention(SA)使其對於訓練資料理解更加全面，模型中的CNN層用於提取訓練資料的重要特徵，BiLSTM層根據提取特徵值進行預測，並使用SA層確定不同時間特徵對股票收盤價的影響後更新模型權重。在本論文實驗結果中，使用未加入情緒特徵訓練資料時，CNN-BiLSTM-SA模型預測準確率相較CNN-BiLSTM-AM模型提升22%，準確率達到87.5%，訓練資料加入情緒特徵後，準確率進一步提升至90.62%，模型在經過改良且加入投資人情緒特徵參考後的預測效果顯著提高。本論文將CNN-BiLSTM-SA模型與LINE Bot串接，使預測結果能夠方便檢視並參考。

In the stock market, investor sentiment can easily cause rapid fluctuations, making it an important reference factor in stock prediction.。This paper investigates the relationship between community sentiment in Taiwan and the fluctuations of the Taiwan stock market. We conduct domain-adaptive pre-training using BERT on Taiwan stock market news to establish a Chinese FinBERT model specialized in the Taiwanese financial sector. We then fine-tune the Chinese FinBERT model using daily discussion content from the stock section of the Taiwanese community discussion platform "PTT" as a reference for Taiwan stock market community sentiment, thereby creating a Daily FinSentiment model. This model analyzes community sentiment and calculates sentiment scores as reference features for predicting stock market fluctuations.。This paper proposes the CNN-BiLSTM-SA model, which uses stock prices and sentiment scores as training data to predict stock price movements. This model is an improvement over the CNN-BiLSTM-AM model, with the Attention Mechanism (AM) architecture replaced by Self Attention (SA) to achieve a more comprehensive understanding of the training data. The CNN layer in the model is used to extract important features from the training data, the BiLSTM layer makes predictions based on the extracted feature values, and the SA layer updates model weights by determining the impact of different temporal features on stock closing prices. In the experimental results of this paper, the CNN-BiLSTM-SA model achieves a prediction accuracy of 87.5%, representing a 22% improvement compared to the CNN-BiLSTM-AM model when trained without sentiment features. When sentiment features are included in the training data, the accuracy further improves to 90.62%. This significant enhancement in prediction performance demonstrates the effectiveness of incorporating investor sentiment features into the model. Furthermore, we integrate the CNN-BiLSTM-SA model with a LINE Bot, allowing for easy review and reference of prediction results.

摘要.............i
Abstract........ii
誌謝............iii
目錄............iv
表目錄..........vii
圖目錄..........viii
第一章 緒論......1
1.1 研究背景.....1
1.2 研究動機.....2
1.3 研究目的.....3
第二章 研究技術與探討......4
2.1 情緒分析..............4
2.2 BERT (Bidirectional Encoder Representations from Transformer).............5
2.3 領域自適應訓練........6
2.4 Fine-Tuning..........7
2.5 機器學習(Machine Learning)....8
監督式學習(Supervised Learning)...8
非監督式學習(Unsupervised Learning)......8
半監督式學習(Semi-Supervised Learning)...8
2.6 多元分類 (Multiclass Classification).9
2.7 長短期記憶 (Long Short-Term Memory, LSTM)....10
2.8 卷積雙向長短期記憶注意力機制 (Convolutional Neural Network Bi Long Short-Term Memory Attention Mechanism , CNN-Bi-LSTM-AM).........................11
2.9 注意力機制(Attention Mechanism)......12
2.10 自注意力機制(Self Attention).......13
2.11 混淆矩陣(Confusion Matrix)..........14
2.11.1 準確率(Accuracy)..................14
2.11.2 精確率(Precision).................14
2.11.3 召回率(Recall)....................15
2.11.4 調和平均數(F1-Score)..............15
2.12 近年相關研究..............16
第三章 研究方法與步驟..............17
3.1 Chinese FinBERT資料蒐集與訓練..............17
3.1.1 資料來源..............17
3.1.2 資料前處理..............18
3.1.3 Mask Language Model(MLM)預訓練..............18
3.1.4 Chinese FinBERT資料集..............19
3.1.5 Chinese FinBERT模型架構..............20
3.2 情緒分析模型Daily FinSentiment..............21
3.2.1 社群資料蒐集與說明..............21
3.2.2 資料標記..............22
3.2.3 半監督式標記 Daily FinSentiment資料集..............22
3.2.4 情緒分數計算方式..............23
3.2.5 模型訓練..............23
3.3 股票預測模型CNN-BiLSTM-SA大盤及產業類股預測..............25
3.3.1 CNN-BiLSTM-SA預測架構圖 25
Input Layer..............26
CNN Layer..............26
BiLSTM Layer..............27
Self Attention..............27
Output Layer..............29
3.4 股票產業類股分類預測..............29
3.4.1 資料集建置..............29
3.4.2 預測模型特徵值..............31
3.4.3 股票分類預測架構圖..............31
3.5 系統介面架構..............32
3.5.1 系統介面之系統架構圖..............33
第四章 研究結果..............34
4.1 Chinese FinBERT預訓練模型結果..............34
4.2 Daily FinSentiment每日情緒預測模型結果..............36
4.3 台股大盤預測模型結果..............37
4.4 台灣產業類股預測結果..............38
4.5 系統UI應用介面..............39
明日股價漲跌以及類股漲跌預測..............39
金融新聞及金融商品報價..............40
第五章 結論..............41
第六章 未來展望..............42
參考文獻..............43
Extended Abstract..............47
Abstract..............47

[1]Text mining approaches for stock market prediction. (2010).
[2]A. S. Pekka Malo, Pekka Korhonen, Jyrki Wallenius, Pyry Takala, "Good debt or bad debt: Detecting semantic orientations in economic texts," ed.
[3]W. W. Jiang, "Applications of deep learning in stock market prediction: Recent progress," in Expert Systems with Applications vol. 184, ed, 2021.
[4]S. Peng et al., "A survey on deep learning for textual emotion analysis in social networks," in Digital Communications and Networks vol. 8, ed, 2022, pp. 745-762.
[5]Z. D. Akşehir and E. Kiliç, "How to handle data imbalance and feature selection problems in CNN-based stock price forecasting," IEEE Access, vol. 10, pp. 31297-31305, 2022.
[6]J. Devlin, M.-W. Chang, and K. Lee, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), ed, pp. 4171–4186.
[7]W. Lu, J. Li, J. Wang, and L. Qin, "A CNN-BiLSTM-AM method for stock price prediction," in Neural Computing and Applications vol. 33, ed, 2020, pp. 4741-4753.
[8]莊凱翔, "對社群媒體進行文字探勘與情緒分析來 預測股票走勢:使用SVM 與LDA 演算法," in 資訊管理研究所碩士班 vol. 碩士, ed. 高雄市: 國立高雄應用科技大學, 2018, p. 80.
[9]D. Araci, "Finbert: Financial sentiment analysis with pre-trained language models," arXiv preprint arXiv:1908.10063, 2019.
[10]S. Peng et al., "A survey on deep learning for textual emotion analysis in social networks," Digital Communications and Networks, vol. 8, no. 5, pp. 745-762, 2022.
[11]G. Tsoumakas, "A survey of machine learning techniques for food sales prediction," Artificial Intelligence Review, vol. 52, no. 1, pp. 441-447, 2019.
[12]G. E. dos Santos and E. Figueiredo, "Commit Classification using Natural Language Processing: Experiments over Labeled Datasets," in CIbSE, 2020, pp. 110-123.
[13]E. Cambria, B. Schuller, Y. Xia, and C. Havasi, "New avenues in opinion mining and sentiment analysis," IEEE Intelligent systems, vol. 28, no. 2, pp. 15-21, 2013.
[14]J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[15]A. Vaswani et al., "Attention is all you need," Advances in neural information processing systems, vol. 30, 2017.
[16]Y.-A. Chung et al., "W2v-bert: Combining contrastive learning and masked language modeling for self-supervised speech pre-training," in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021: IEEE, pp. 244-250.
[17]K. Song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu, "Mass: Masked sequence to sequence pre-training for language generation," arXiv preprint arXiv:1905.02450, 2019.
[18]S. González-Carvajal and E. C. Garrido-Merchán, "Comparing BERT against traditional machine learning text classification," arXiv preprint arXiv:2005.13012, 2020.
[19]P. Shi and J. Lin, "Simple bert models for relation extraction and semantic role labeling," arXiv preprint arXiv:1904.05255, 2019.
[20]B. Van Aken, B. Winter, A. Löser, and F. A. Gers, "How does bert answer questions? a layer-wise analysis of transformer representations," in Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp. 1823-1832.
[21]S. Gururangan et al., "Don't stop pretraining: Adapt language models to domains and tasks," arXiv preprint arXiv:2004.10964, 2020.
[22]H. Wu, K. Xu, L. Song, L. Jin, H. Zhang, and L. Song, "Domain-adaptive pretraining methods for dialogue understanding," arXiv preprint arXiv:2105.13665, 2021.
[23]R. K. Jørgensen, M. Hartmann, X. Dai, and D. Elliott, "mDAPT: Multilingual domain adaptive pretraining in a single model," arXiv preprint arXiv:2109.06605, 2021.
[24]N. S. Ashish Vaswani, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin. "huggingface/transformers." https://github.com/huggingface/transformers (accessed.
[25]J. Salazar, D. Liang, T. Q. Nguyen, and K. Kirchhoff, "Masked language model scoring," arXiv preprint arXiv:1910.14659, 2019.
[26]S. Friederich, "Fine-tuning," The Stanford encyclopedia of philosophy, 2017.
[27]M. I. Jordan and T. M. Mitchell, "Machine learning: Trends, perspectives, and prospects," Science, vol. 349, no. 6245, pp. 255-260, 2015.
[28]Y. Bengio, A. C. Courville, and P. Vincent, "Unsupervised feature learning and deep learning: A review and new perspectives," CoRR, abs/1206.5538, vol. 1, no. 2665, p. 2012, 2012.
[29]X. J. Zhu, "Semi-supervised learning literature survey," 2005.
[30]M. Grandini, E. Bagli, and G. Visani, "Metrics for multi-class classification: an overview," arXiv preprint arXiv:2008.05756, 2020.
[31]S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[32]A. Berhich, F.-Z. Belouadha, and M. I. Kabbaj, "An attention-based LSTM network for large earthquake prediction," Soil Dynamics and Earthquake Engineering, vol. 165, p. 107663, 2023.
[33]A. Q. Md et al., "Novel optimization approach for stock price forecasting using multi-layered sequential LSTM," Applied Soft Computing, vol. 134, p. 109830, 2023.
[34]I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks," Advances in neural information processing systems, vol. 27, 2014.
[35]V. Badrinarayanan, A. Kendall, and R. Cipolla, "Segnet: A deep convolutional encoder-decoder architecture for image segmentation," IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481-2495, 2017.
[36]M. R. Kumar, S. Ramkumar, S. Saravanan, R. Balakrishnan, and M. Swathi, "Stock Market Prediction via Twitter Sentiment Analysis using BERT: A Federated Learning Approach," in Handbook on Federated Learning: CRC Press, pp. 333-353.
[37]H. Yang, C. Ye, X. Lin, and H. Zhou, "Stock Market Prediction Based on BERT Embedding and News Sentiment Analysis," in International Conference on Service Science, 2023: Springer, pp. 279-291.
[38]Y. Ayyappa, B. V. Kumar, S. P. Priya, S. Akhila, T. P. V. Reddy, and S. M. Goush, "Forecasting Equity Prices using LSTM and BERT with Sentiment Analysis," in 2023 International Conference on Inventive Computation Technologies (ICICT), 2023: IEEE, pp. 643-648.
[39]孫育澤, "使用文本情緒分析與混合深度學習建構ETF投資之決策輔助系統," 碩士, 數學暨資訊教育學系, 國立臺北教育大學, 台北市, 2023. [Online]. Available: https://hdl.handle.net/11296/wrj4j2
[40]王仲奇, "基於時間卷積網路之投資者情緒對股票收盤價之影響探討," 碩士, 電機工程學系乙組, 元智大學, 桃園縣, 2023. [Online]. Available: https://hdl.handle.net/11296/7bgt8b
[41]許嘉洲, "使用圖神經網路及長短期記憶模型預測台股股價," 碩士, 資訊工程系, 國立高雄科技大學, 高雄市, 2023. [Online]. Available: https://hdl.handle.net/11296/h37hz4
[42]張維倫, "應用多模態深度學習與注意力機制對台灣指數股票型基金進行投資策略分析," 碩士, 資訊管理學系所, 國立中興大學, 台中市, 2023. [Online]. Available: https://hdl.handle.net/11296/73jjuw