臺灣博碩士論文加值系統

近年來社群媒體的蓬勃發展，帶來了許多便利的服務和應用，同時也引發了一些負面現象，例如惡意言論、網絡霸凌和其他形式的網絡暴力行為。面對這一現象，我們迫切需要針對社群媒體安全進行深入研究，本研究透過情感分析技術，幫助社群媒體平台識別負面言論。以往方法中，大多使用了大型的語言模型進行解析，例如BERT、ALBERT等，這些模型在處理大量資料時，需要耗費大量的時間和計算資源，就無法即時對貼文進行分析。因此，針對大型語言模型的時間成本問題，本研究提出了一種新的方法，研究採用了參數較少的ALBERT-tiny模型，並結合了多個卷積層和注意力，從語言模型中擷取出更良好的情感特徵。實驗結果顯示，本研究在自製的社群媒體情感資料集中，達到了87.36%的準確率，並且比ALBERT-base快了4.37倍，在準確率和計算成本之間取得了良好的平衡。

In recent years, the flourishing development of social media has brought about numerous convenient services and applications, while concurrently sparking negative phenomena, such as malicious content, cyberbullying, and other forms of online violent behavior. In the face of these phenomena, there is an urgent need for profound research into the security of social media platforms. This study employs sentiment analysis techniques to assist in identifying adverse content within social media.Previous methods have predominantly employed large language models for parsing, such as BERT and ALBERT. However, these models demand significant time and computational resources when processing substantial amounts of data, thereby impeding real-time post analysis. Addressing the time-cost issue associated with large-scale language models, this research introduces a novel approach. It adopts the ALBERT-tiny model, characterized by fewer parameters, and amalgamates multiple CNN and attention layers to extract superior sentiment features from the language model. Experimental results demonstrate that this study achieves an accuracy of 88.36% within a self-constructed social media sentiment dataset. Furthermore, it outperforms ALBERT-base by 4.37 times in terms of speed, achieving a harmonious equilibrium between accuracy and computational cost.

摘要 i
ABSTRACT ii
致謝 iii
目錄 iv
表目錄 vi
圖目錄 vii
第一章 序論 1
1.1 研究動機 1
1.2 論文架構 2
第二章 文獻探討 3
2.1 資料蒐集 3
2.2 資料前處理 4
2.3 詞嵌入 5
2.3.1 靜態詞嵌入 5
2.3.2 動態詞嵌入 9
2.4 情感分析 11
2.4.1 淺層學習 12
2.4.2 深度學習 13
第三章 研究方法 16
3.1 資料蒐集 16
3.1.1 Facebook API 16
3.1.2 Labeling 18
3.2 系統架構 19
3.2.1 前處理模組 20
3.2.2 特徵擷取模組 21
3.2.3 分類模組 23
3.3 損失函數 24
3.4 訓練技巧 25
第四章 實驗結果 27
4.1 實驗環境 27
4.2 實驗資料集 27
4.3 模型訓練 28
4.4 評估方法 29
4.5 實驗結果 30
4.5.1 資料集實驗 30
4.5.2 消融實驗 31
第五章 結論 32
第六章 參考文獻 33

[1] Bing Liu. Sentiment analysis and opinion mining. Synthesis lectures
on human language technologies, 5(1):1–167, 2012.
[2] Marouane Birjali, Mohammed Kasri, and Abderrahim Beni-Hssane.
A comprehensive survey on sentiment analysis: Approaches, chal-
lenges and trends. Knowledge-Based Systems, 226:107134, 2021.
[3] Y. Zhang and Flora S. Tsai. Chinese novelty mining. In Conference
on Empirical Methods in Natural Language Processing, 2009.
[4] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Ef-
ficient estimation of word representations in vector space. arXiv
preprint arXiv:1301.3781, 2013.
[5] Jeffrey Pennington, Richard Socher, and Christopher D. Manning.
Glove: Global vectors for word representation. In Conference on
Empirical Methods in Natural Language Processing, 2014.
[6] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas
Mikolov. Enriching word vectors with subword information. Trans-
actions of the Association for Computational Linguistics, 5:135–
146, 2016.
[7] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner,
Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep con-
textualized word representations. In North American Chapter of the
Association for Computational Linguistics, 2018.
[8] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina
Toutanova. Bert: Pre-training of deep bidirectional transformers
for language understanding. ArXiv, abs/1810.04805, 2019.
[9] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gim-
pel, Piyush Sharma, and Radu Soricut. Albert: A lite bert
for self-supervised learning of language representations. ArXiv,
abs/1909.11942, 2019.
[10] Andrea Gasparetto, Matteo Marcuzzo, Alessandro Zangari, and An-
drea Albarelli. A survey on text classification algorithms: From text
to predictions. Inf., 13:83, 2022.
[11] Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. Thumbs up?
sentiment classification using machine learning techniques. ArXiv,
cs.CL/0205070, 2002.
[12] Untari Novia Wisesty, Rita Rismala, Wira Munggana, and Ayu Pur-
warianti. Comparative study of covid-19 tweets sentiment classifi-
cation methods. 2021 9th International Conference on Information
and Communication Technology (ICoICT), pages 588–593, 2021.
[13] Yoon Kim. Convolutional neural networks for sentence classifica-
tion. In Conference on Empirical Methods in Natural Language
Processing, 2014.
[14] Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Ko-
ray Kavukcuoglu, and Pavel P. Kuksa. Natural language processing
(almost) from scratch. ArXiv, abs/1103.0398, 2011.
[15] Keliang Jia. Sentiment classification of microblog: A framework
based on bert and cnn with attention mechanism. Comput. Electr.
Eng., 101:108032, 2022.
[16] Hyunjin Choi, Judong Kim, Seongho Joe, and Youngjune Gwon.
Evaluation of bert and albert sentence embedding performance on
downstream nlp tasks. 2020 25th International Conference on Pat-
tern Recognition (ICPR), pages 5482–5487, 2021.
[17] Shikhar Vashishth, Shyam Upadhyay, Gaurav Singh Tomar, and
Manaal Faruqui. Attention interpretability across nlp tasks. ArXiv,
abs/1909.11218, 2019.
[18] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit,
Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention is all you need. ArXiv, abs/1706.03762, 2017.