本論文主要分析Twitter上較多人討論的訊息內容，以了解當前社群媒體上重要的熱門話題。本研究運用預訓練的語言模型，對Twitter大數據資料進行分析，首先使用監督式學習的分類方法幫助過濾日常推文內容，再透過非監督式學習的聚類方法將過濾後的推文內容，以密度為本的聚類演算法將討論度較高的相似推文內容進行聚類，並整理成熱門話題。使用分類方法先將不重要的日常推文進行過濾，能有效地降低計算時間，同時也能降低這些不重要的日常推文出現在熱門話題的機率；而透過以密度為本的聚類演算法DBSCAN，能將相似的內容進行聚類，同時能夠標記出低密度的局外點。推文內容之間的距離計算方式，則是通過SBERT模型架構將推文內容轉換成句子向量，進而計算向量之間歐幾里得距離。
討論度較高的推文內容則作為熱門話題，將熱門話題的推文內容進行特徵處理，整理成每日的文本特徵或是情緒特徵向量，運用深度學習中的循環神經網路分析真實世界市場的未來趨勢，並對三個實際案例進行實證探討。在本研究的實驗結果中，使用Twitter熱門話題的推文內容特徵的結果相比於沒有使用Twitter特徵的結果來的好。有了社群大數據的特徵幫助，模型可以了解當前世界的情況，使用本研究之方法作為重大事件之呈現，能有效地幫助模型判斷真實世界市場的未來趨勢。

This paper mainly analyzed the information mostly discussed by people on Twitter and understood the important hot topics on Twitter social media. In this study, we used pre-trained language models to analyze Twitter big data. First, we used a supervised learning classification method to help filter the content of daily tweets. Then used an unsupervised learning clustering algorithm based on density to cluster the content of similar tweets with a high degree of discussion and organized them into hot topics. The classification method filtered unimportant daily tweets, which can effectively reduce the calculation time and the probability of these unimportant daily tweets appearing on hot topics. The density-based algorithm DBSCAN can cluster similar content and mark low-density outliers. The distance between the content of the tweets is calculated by using the model architecture of SBERT to convert tweet content to a sentence vector and calculating the Euclidean distance between two vectors.
Tweets with high degree of discussion are regarded as hot topics. Tweets of hot topics are processed and organized into daily text feature and sentiment feature vectors. Then used recurrent neural network to analyze the future trends of the financial market and conduct empirical discussions on three actual cases. In the experimental results of this study, the results of using the tweet content features of Twitter hot topics are better than those not using Twitter features. With the help of the features from social media big data, the model can understand the current situation in the world. Major events presented by our method can effectively help the model to judge the future trends of the financial market.

摘要 ------------------------------------------------------------------------------ i
ABSTRACT ------------------------------------------------------------------------------ ii
誌謝 ------------------------------------------------------------------------------ iii
目錄 ------------------------------------------------------------------------------ iv
表目錄 ------------------------------------------------------------------------------ vi
圖目錄 ------------------------------------------------------------------------------ vii
第一章、 緒論------------------------------------------------------------------------ 1
1.1 前言------------------------------------------------------------------------ 1
1.2 研究背景------------------------------------------------------------------ 1
1.3 研究動機和目的--------------------------------------------------------- 2
1.4 問題領域------------------------------------------------------------------ 4
1.5 研究步驟------------------------------------------------------------------ 4
1.6 論文組織架構------------------------------------------------------------ 5
第二章、 文獻與相關技術之探討------------------------------------------------ 6
2.1 Twitter事件偵測的相關研究----------------------------------------- 6
2.1.1 Twitter事件偵測應用於災害檢測的相關研究-------------------- 7
2.1.2 Twitter事件偵測應用於疾病傳播檢測的相關研究-------------- 7
2.2 Twitter事件分群的相關研究----------------------------------------- 8
2.3 Twitter主題分析的相關研究----------------------------------------- 8
2.4 詞嵌入的相關研究------------------------------------------------------ 9
2.4.1 句子向量的相關研究--------------------------------------------------- 10
2.5 BERT的相關研究------------------------------------------------------- 10
2.6 文本聚類與相似度計算的相關研究--------------------------------- 11
2.7 財金領域分析的相關研究--------------------------------------------- 12
2.8 深度學習相關技術介紹------------------------------------------------ 13
2.8.1 類神經網路簡介--------------------------------------------------------- 14
2.8.2 激活函數介紹------------------------------------------------------------ 16
2.8.3 損失函數介紹------------------------------------------------------------ 17
2.8.4 優化器介紹--------------------------------------------------------------- 17
2.9 遞歸神經網路簡介------------------------------------------------------ 18
2.10 Encoder-Decoder框架簡介-------------------------------------------- 21
2.11 BERT 簡介--------------------------------------------------------------- 22
2.11.1 BERT模型架構介紹---------------------------------------------------- 23
2.11.2 Transformer架構介紹-------------------------------------------------- 23
2.11.3 BERT輸入表示方法---------------------------------------------------- 26
2.11.4 BERT Pre-train方式---------------------------------------------------- 27
2.11.5 BERT 訓練資料--------------------------------------------------------- 29
2.11.6 BERT Fine-tune方法--------------------------------------------------- 30
2.12 Sentence-BERT簡介---------------------------------------------------- 31
2.12.1 Sentence-BERT架構介紹---------------------------------------------- 31
2.13 文本聚類方法簡介------------------------------------------------------ 35
2.13.1 文本相似度--------------------------------------------------------------- 36
第三章、 實驗設計與建置--------------------------------------------------------- 37
3.1 本研究之系統架構------------------------------------------------------ 37
3.2 資料數據來源與蒐集方式--------------------------------------------- 38
3.2.1 Twitter數據處理-------------------------------------------------------- 39
3.2.2 類別分類數據------------------------------------------------------------ 40
3.2.3 文本相似度數據--------------------------------------------------------- 41
3.3 實驗設計------------------------------------------------------------------ 41
3.4 實驗工具------------------------------------------------------------------ 44
第四章、 實驗結果與數據分析--------------------------------------------------- 45
4.1 實驗結果與模型表現--------------------------------------------------- 45
4.2 實驗數據分析與討論--------------------------------------------------- 47
第五章、 實際案例分析------------------------------------------------------------ 59
5.1 實例一：VIX指數數據------------------------------------------------ 63
5.2 實例二：GLD指數數據----------------------------------------------- 66
5.3 實例三：GBTC指數數據---------------------------------------------- 68
第六章、 系統效能評估------------------------------------------------------------ 70
6.1 VIX指數------------------------------------------------------------------ 70
6.2 GLD指數----------------------------------------------------------------- 72
6.3 GBTC指數--------------------------------------------------------------- 74
第七章、 論文探討與未來發展--------------------------------------------------- 76
7.1 結論與問題討論--------------------------------------------------------- 76
7.2 未來研究工作------------------------------------------------------------ 78
參考文獻 ------------------------------------------------------------------------------ 79

[1]Sarmiento, H., “A Domain-Independent and Multilingual Approach for Crisis Event Detection and Understanding”, In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1457-1457, July 2019.

[2]Li, Q. et al., “Real-time novel event detection from social media”, In Proceedings of 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pp. 1129-1139, IEEE, April 2017.

[3]Weng, J., and Lee, B. S., “Event detection in twitter”, In Proceedings of Fifth International AAAI Conference on Weblogs and Social Media, July 2011.

[4]Lee, R., and Sumiya, K., “Measuring geographical regularities of crowd behaviors for Twitter-based geo-social event detection” In Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Location Based Social Networks, pp. 1-10, November 2010.

[5]Abdelhaq, H., Sengstock, C., and Gertz, M., “Eventweet: Online localized event detection from twitter”, In Proceedings of the VLDB Endowment, Vol. 6, Issue 12, pp. 1326-1329, 2013.

[6]Sakaki, T., Okazaki, M., and Matsuo, Y., “Earthquake shakes Twitter users: real-time event detection by social sensors”, In Proceedings of the 19th International Conference on World Wide Web, pp. 851-860, April 2010.

[7]Bügel, U., and Zielinski, A., “Multilingual analysis of twitter news in support of mass emergency events”, In Proceedings of the 9th International ISCRAM Conference, Vancouver, Canada, April 2012.

[8]Doan, S. et al., “An analysis of Twitter messages in the 2011 Tohoku Earthquake”, In Proceedings of the International Conference on Electronic Healthcare, pp. 58-66, Berlin, Heidelberg, Springer, November 2011.

[9]Krieck, M. et al., “A new age of public health: Identifying disease outbreaks by analyzing tweets”, In Proceedings of Health Web-Science Workshop, ACM Web Science Conference, pp. 10-15, June 2011.

[10]Odlum, M., and Yoon, S., “What can we learn about the Ebola outbreak from tweets?”, In American Journal of Infection Control, Vol. 43, Issue 6, pp. 563-571, 2015.

[11]Mollema, L. et al., “Disease detection or public opinion reflection? Content analysis of tweets, other social media, and online newspapers during the measles outbreak in The Netherlands in 2013”, In Journal of Medical Internet Research, Vol. 17, Issue 5, 2015.

[12]Signorini, A., Segre, A. M., and Polgreen, P. M., “The use of Twitter to track levels of disease activity and public concern in the US during the influenza A H1N1 pandemic”, In Journal of PloS one, Vol. 6, Issue 5, May 2011.

[13]Schinas, M. et al., “Event detection and retrieval on social media”, In arXiv preprint arXiv:1807.03675, 2018.

[14]Zhang, C. et al., “Triovecevent: Embedding-based online local event detection in geo-tagged tweet streams”, In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 595-604, August 2017.

[15]Zhou, X., and Chen, L., “Event detection over twitter social media streams”, In the VLDB Journal, Vol. 23, Issue 3, pp. 381-400, 2014.

[16]Kim, H. G., Lee, S., and Kyeong, S., “Discovering hot topics using Twitter streaming data social topic detection and geographic clustering”, In Proceedings of 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013), pp. 1215-1220, IEEE, August 2013.

[17]Zou, L., and Song, W. W., “Lda-tm: A two-step approach to twitter topic data clustering”, In Proceedings of 2016 IEEE International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), pp. 342-347, IEEE, July 2016.

[18]Singh, J., Kaur, I., and Singh, A. K., “Event detection from Twitter data”, In Proceedings of 2019 4th International Conference on Information Systems and Computer Networks (ISCON), pp. 793-798, IEEE, November 2019.

[19]Mikolov, T. et al., “Distributed representations of words and phrases and their compositionality”, In Proceedings of Advances in Neural Information Processing Systems, pp. 3111-3119, 2013.

[20]Peters, M. E. et al., “Deep contextualized word representations”, In arXiv preprint arXiv:1802.05365, 2018.

[21]Radford, A. et al., “Improving language understanding by generative pre-training”, URL http://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf, 2018.

[22]Radford, A. et al., “Language models are unsupervised multitask learners”, In OpenAI blog, 2019.

[23]Brown, T. B. et al., “Language models are few-shot learners”, In arXiv preprint arXiv:2005.14165, 2020.

[24]Devlin, J. et al., “Bert: Pre-training of deep bidirectional transformers for language understanding”, In arXiv preprint arXiv:1810.04805, 2018.

[25]Talman, A., Yli-Jyrä, A., and Tiedemann, J., “Sentence embeddings in NLI with iterative refinement encoders”, In Journal of Natural Language Engineering, Vol. 25, Issue 4, pp. 467-482, 2019.

[26]Conneau, A., and Kiela, D., “Senteval: An evaluation toolkit for universal sentence representations”, In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC), May 2018.

[27]Conneau, A. et al., “Supervised learning of universal sentence representations from natural language inference data”, In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 670-680, September 2017.

[28]Cer, D. et al., “Universal sentence encoder”, In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 169-174, November 2018.

[29]Reimers, N., and Gurevych, I., “Sentence-bert: Sentence embeddings using siamese bert-networks”, In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3973-3983, November 2019.

[30]Liu, Y. et al., “Roberta: A robustly optimized bert pretraining approach”, In arXiv preprint arXiv:1907.11692, 2019.

[31]Sanh, V. et al., “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter”, In arXiv preprint arXiv:1910.01108, 2019.

[32]Li, B. et al., “On the Sentence Embeddings from Pre-trained Language Models”, In arXiv preprint arXiv:2011.05864, 2020.

[33]Vijaymeena, M. K., and Kavitha, K., “A survey on similarity measures in text mining”, In Journal of Machine Learning and Applications, Vol. 3, Issue 2, pp. 19-28, 2016.

[34]Amine, A., Elberrichi, Z., and Simonet, M., “Evaluation of text clustering methods using wordnet”, In International Arab Journal of Information Technology (IAJIT), Vol. 7, Issue 4, pp. 349-357, 2010.

[35]Huang, A., “Similarity measures for text document clustering”, In Proceedings of the sixth New Zealand Computer Science Research Student Conference (NZCSRSC2008), Vol. 4, pp. 9-56, Christchurch, New Zealand, April 2008.

[36]Jing, L. et al., “Ontology-based distance measure for text clustering”, In Proceedings of SIAM SDM Workshop on Text Mining, Bethesda, Maryland, USA, April 2006.

[37]Nelson, D. M. et al., “Stock market's price movement prediction with LSTM neural networks”, In Proceedings of 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1419-1426. IEEE, May 2017.

[38]Roondiwala, M. et al., “Predicting stock prices using LSTM”, In International Journal of Science and Research (IJSR), Vol. 6, Issue 4, pp. 1754-1756, 2017.

[39]Mehtab, S., Sen, J., and Dutta, A., “Stock price prediction using machine learning and LSTM-based deep learning models”, In arXiv preprint arXiv:2009.10819, 2020.

[40]Kumar, P. H., and Patil, S. B., “A Hybrid Regression and Deep Learning LSTM Based Technique for Predicting Volatility IndeX (VIX) Direction of Change (Trend)”, In Indian Journal of Science and Technology, Vol. 11, Issue 47, 2018.

[41]Baker, M., and Wurgler, J., “Investor sentiment and the cross‐section of stock returns”, In The journal of Finance, Vol. 61, Issue 4, pp. 1645-1680, 2006.

[42]Guo, X., and Li, J., “A Novel Twitter Sentiment Analysis Model with Baseline Correlation for Financial Market Prediction with Improved Efficiency”, In Proceedings of 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp. 472-477, IEEE, October 2019.

[43]Ruan, Y., Durresi, A., and Alfantoukh, L., “Using Twitter trust network for stock market analysis”, In Knowledge-Based Systems, 145, 207-218, 2018.

[44]Kingma, D. P., and Ba, J., “Adam: A method for stochastic optimization”, In Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.

[45]Hochreiter, S., and Schmidhuber, J., “Long short-term memory” In Journal of Neural Computation, Vol. 9, Issue 8, pp. 1735-1780, 1997.

[46]Chung, J. et al., “Empirical evaluation of gated recurrent neural networks on sequence modeling”, In NIPS 2014 Workshop on Deep Learning, December 2014.

[47]Cho, K. et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation”, In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), October 2014.

[48]Venugopalan, S. et al., “Sequence to sequence-video to text”, In Proceedings of the IEEE international conference on computer vision, pp. 4534-4542, 2015.

[49]Mnih, V., Heess, N., and Graves, A., “Recurrent models of visual attention”, In Proceedings of Advances in neural information processing systems 27, pp. 2204-2212, 2014.

[50]Bahdanau, D., Cho, K., and Bengio, Y., “Neural machine translation by jointly learning to align and translate”, In Proceedings of the 3rd International Conference on Learning Representations(ICLR), January 2015.

[51]Vaswani, A. et al., “Attention is all you need” In Proceedings of Advances in Neural Information Processing Systems, pp. 5998-6008, 2017.

[52]Dai, A. M., and Le, Q. V., “Semi-supervised sequence learning”, In Proceedings of Advances in Neural Information Processing Systems, pp. 3079-3087, 2015.

[53]Howard, J., and Ruder, S., “Universal language model fine-tuning for text classification”, In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 328-339, July 2018.

[54]He, K. et al., “Deep residual learning for image recognition”, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.

[55]Taylor, W. L., “Cloze procedure : A new tool for measuring readability”, In Journal of Journalism Quarterly, Vol. 30, Issue 4, pp. 415-433, 1953.

[56]MacQueen, J., “Some methods for classification and analysis of multivariate observations”, In Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, Issue 14, pp. 281-297, June 1967.

[57]Ester, M. et al., “A density-based algorithm for discovering clusters in large spatial databases with noise”, In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD’96), Vol. 96, Issue 34, pp. 226-231, August 1996.

[58]Misra, R., “News category dataset”, 2018.

[59]Smilkov, D. et al., “Embedding projector: Interactive visualization and interpretation of embeddings”, In arXiv preprint arXiv:1611.05469, 2016.

[60]Go, A., Bhayani, R., and Huang, L., “Twitter sentiment classification using distant supervision”, In CS224N project report, Stanford, Vol. 1, Issue 12, 2009.

[61]Hutto, C., and Gilbert, E., “Vader: A parsimonious rule-based model for sentiment analysis of social media text”, In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 8, May 2014.

[62]Ioffe, S., and Szegedy, C., “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”, In Proceedings of the 32nd International Conference on Machine Learning, pp. 448-456, June 2015.