臺灣博碩士論文加值系統

在傳統紙本年代，透過閱讀，可以將少量有限的文本資料標示重點，進而找到關鍵字。由於近代網路科技的快速發展，我們在生活當中會接觸到大量的網路資訊，尤其是各式各樣的文本。對使用者來說，不論是在搜尋引擎或是在各大媒體大量產生的文本，使用者很難在短時間擷取文本中的主題關鍵字，所以對現代人而言，文本分析顯得格外重要。因此本研究動機是嘗試以主題建模(Topic Model-ing)對文本進行擷取關鍵字，讓使用者知道在文本當中的主題關鍵字，也可知道關鍵字在文本分析當中的分群結果。
本研究會先對文本的字句透過Gensim與Spacy等相關自然語言進行相關的斷詞的預處理，再透過Topic Modeling裡的54這三種模型裡的Topics num整理出Topics keywords，最後會透過clustering裡的K-means得知分群前的結果，在LDA (latent Dirichlet Allocation) Model、LSI ( Latent Semantic Indexing) Model、HDP ( Hierarchical Dirichlet Process) Model這三種模型當中，以LDA (latent Dirichlet Allocation) Model分群的最好。
本研究是針對一篇長篇的文本進行分析，所以在研究當中的關鍵字都是從同一篇的文本裡整理出來的，可以達到輔助使用者釐清文本當中的關鍵字。研究貢獻主要是使用者能夠快速地釐清文本當中的關鍵字，透過Topic Modeling及分群的相關技術整理出關鍵字及分群結果，能夠讓使用者有快速釐清文本當中的關鍵字及關鍵字在文本當中的重要性。

In the traditional paper age, through reading, a small amount of limited text information can be highlighted and key words can be found. Due to the rapid development of modern Internet technology, we will come into contact with a large amount of Internet information in our daily life, especially various texts. For users, it is difficult for users to extract the subject keywords in the text in a short time, no matter in the search engine or in the mass media, so
for modern people, text analysis is particularly important . Therefore, the motivation of this
research is to try to extract keywords from the text by Topic Modeling, so that users can
know the topic keywords in the text, and also know the grouping results of the keywords in the text analysis.
This research will first preprocess the words and sentences of the text through related natural languages such as Gensim and Spacy, and then sort out the Topics keywords through the Topics num in the three models of LDA (latent Dirichlet Allocation) Model, LSI ( Latent Semantic Indexing) Model, and HDP ( Hierarchical Dirichlet Process) Model in Topic M-odeling , and finally the results before clustering will be obtained through K-means in cluster-ing. Among the three models of LDA (latent Dirichlet Allocation) Model, LSI ( Latent Semantic Indexing) Model and HDP ( Hierarchical Dirichlet Process) Model, LDA (latent Dirichlet Allocation) Model grou-ping is the best.
This research is based on the analysis of a long text, so the keywords in the research are sorted out from the same text, which can help users to clarify the keywords in the text.
The research contribution is mainly that users can quickly clarify the keywords in the text, and sort out the keywords and grouping results through Topic Modeling and clustering related technologies, allowing users to quickly clarify the keywords in the text and the keywords in the
text importance.

目 錄
摘要 -----------------i
Abstract-------------ii
致謝-----------------iii
目錄-----------------iv
表目錄----------------v
圖目錄---------------vi
壹、緒論------------- 1
一、 研究背景與動機--- 1
二、研究目的--------- 2
三、研究貢獻--------- 2
貳、文獻探----------- 3
一、文本分析--------- 3
二、 主題建模(Topic Modeling)----------------- 4
三、Gensim---------- 5
四、 Spacy---------- 6
五、K-means--------- 6
參、研究方法--------- 8
一、研究架構--------- 8
二、參數及範圍設定----12
三、LDA Model、LSI Model、HDP Model模型--------13
四、系統環境-------- 14
肆、研究結果-------- 15
一、Topic Modeling keywords結果---------------- 15
二、K-means結果-----29
三、研究結果探討--- 50
四、研究限制------- 52
伍、結論與未來展望-- 53
陸、參考文獻------- 54

[1]‘文本分析解决方案’, Megaputer Intelligence <https://www.megaputer.com/zh/solutions/text-analytics/> [accessed 20 April 2023].
[2]‘內容分析法’, 維基百科，自由的百科全書, 2022 <https://zh.wikipedia.org/w/index.php?title=%E5%85%A7%E5%AE%B9%E5%88%86%E6%9E%90%E6%B3%95&oldid=75232087> [accessed 23 March 2023].
[3]‘什麼是文本分析 - 德鴻科技 Grandsys’ <https://www.grandsys.com.tw/news/innovative-applications/886-textual-analysis> [accessed 23 March 2023].
[4] ‘主题模型’, 維基百科，自由的百科全書, 2022 <https://zh.wikipedia.org/w/index.php?title=%E4%B8%BB%E9%A2%98%E6%A8%A1%E5%9E%8B&oldid=70205618> [accessed 8 March 2023].
[5]‘What Is Topic Modeling? A Beginner’s Guide’ <https://levity.ai/blog/what-is-topic-modeling> [accessed 24 March 2023].
[6]‘Latent Dirichlet Allocation(LDA (LATENT DIRICHLET ALLOCATION))’, HackMD <https://hackmd.io/@WangJengYun/LDA (LATENT DIRICHLET ALLOCATION)> [accessed 24 March 2023].
[7]Chawla, Indu, and Sandeep K. Singh. "Performance evaluation of vsm and LSI ( Latent Semantic Indexing) models to determine bug reports similarity." 2013 Sixth International Conference on Contemporary Computing (IC3). IEEE, 2013.
[8] ‘Hierarchical Dirichlet Process’, Wikipedia, 2022 <https://en.wikipedia.org/w/index.php?title=Hierarchical_Dirichlet_process&oldid=1100589533> [accessed 24 March 2023].
[9]‘Gensim’, 維基百科, 2022 <https://en.wikipedia.org/w/index.php?title=Gensim&oldid=1088908886#References> [accessed 27 March 2023].
[10] ‘介紹· Gensim 中文文檔· 看雲’ <https://www.kancloud.cn/apachecn/gensim-doc-zh/1949218> [accessed 27 March 2023].
[11] ‘Gensim介紹 - 台部落’ <https://www.twblogs.net/a/5c3b4979bd9eee35b21dd224> [accessed 27 March 2023].
[12] ‘强大的 NLP 库 Gensim 进行文本分析’, 知乎专栏 <https://zhuanlan.zhihu.com/p/566790161> [accessed 27 March 2023].
[13] ‘spaCy’, 維基百科, 2023 <https://en.wikipedia.org/w/index.php?title=SpaCy&oldid=1133910462> [accessed 30 March 2023].
[14]‘自然語言處理函式庫spaCy 2.2釋出，加入資料增強系統、提升字詞配對速度’, iThome <https://www.ithome.com.tw/news/133448> [accessed 30 March 2023].
[15] iThome, ‘[Day 6] 非監督式學習 K-means 分群’, iT 邦幫忙::一起幫忙解決難題，拯救 IT 人的一天 <https://ithelp.ithome.com.tw/articles/10266672> [accessed 31 March 2023].
[16] ‘K 平均法 (K Means)’ <https://rstudio-pubs-static.s3.amazonaws.com/378455_ddbefe5075b941d1a1f6a1bf9cf1e85f.html> [accessed 31 March 2023].
[17] PyInvest, ‘[機器學習首部曲] 聚類分析 K-Means / K-Medoids’, PyInvest, 2020 <https://pyecontech.com/2020/05/19/k-means_k-medoids/> [accessed 31 March 2023].
[18]‘k-平均演算法’, 維基百科，自由的百科全書, 2023 <https://zh.wikipedia.org/w/index.php?title=K-%E5%B9%B3%E5%9D%87%E7%AE%97%E6%B3%95&oldid=76044067> [accessed 31 March 2023].
[19]Jelodar, Hamed, et al. "Latent Dirichlet allocation (LDA (LATENT DIRICHLET ALLOCATION)) and topic modeling: models, applications, a survey." Multimedia Tools and Applications 78 (2019): 15169-15211.
[20]Wolfhagen, Jesse. "Re-examining the use of the LSI ( LATENT SEMANTIC INDEXING) technique in zooarchaeology." Journal of Archaeological Science 123 (2020): 105254.
[21]Ding, Wanying, et al. "A novel hybrid HDP ( HIERARCHICAL DIRICHLET PROCESS)-LDA (LATENT DIRICHLET ALLOCATION) model for sentiment analysis." 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT). Vol. 1. IEEE, 2013.
[22] ‘KMeans Silhouette Score Python 示例 - 數據分析’ <https://vitalflux.com/kmeans-silhouette-score-explained-with-python-example/> [accessed 26 June 2023].