臺灣博碩士論文加值系統

在人工智慧自然語言處理逐漸興起的環境下，機器的知識或智慧形成傾向於從網路上各
式各樣的短文、語料庫及知識庫中獲取其知識或智慧內容來源。一般的語料庫與知識庫
在內容上因用字的不同特性，及語料庫與知識庫中相似字之間過於零散，導致機器在使
用一般的語料庫及知識庫去做相似字的預測之知識或智慧之學習結果差異很大。因此，
語料庫與知識庫的選擇與製作就變得在建構機器的智慧之高低尤為重要。因此，在訓練
前將語料庫或短文本及知識庫進行選擇與分析是非常重要。本論文提出三種基於語料庫
與知識庫之改良中文 Word2Set 的開發。首先本論文提出了在 WordNet 基礎上加入 EHowNet 進行單詞語意分析的中文 Word2Set 的改良。單辭的語意訊息和單詞關聯詞對於
識別單詞非常重要，單辭語意訊息可由 WordNet 加上 E-HowNet 提供，以單詞的同義詞
和反義詞所組成的訊息。同時改良中文 Word2Set 的 SVR，使用 LSTM 解決訓練過程中
的梯度消失。其次本論文提出改善中文 Word2Set 中使用 Word2Vec 學習經由改良的語
料庫以及修改 Word2Vec 隱藏層的權重，使得訓練更精準及在 LDA 的應用。再其次本論
文提出用深度學習方法重建知識庫，以提升中文的 Word2Set 的功能，所提出的方法利
IV
用深度學習去學改良的知識庫，使得中文的 Word2Set 之應用更加精準。實驗結果驗證，
經過改良過後的語料庫與知識庫可以更加有效的訓練機器在相似字的預測上更加精準。

In an environment where artificial intelligence and natural language processing are gradually
emerging, the knowledge or intelligence of machines tends to obtain their knowledge or
intelligence content sources from various short essays, corpus, and knowledge-base on the
Internet. The general corpus and knowledge-base have different characteristics of the words
used in the content, and the similar words in the corpus and the knowledge-base are too scattered,
which causes the machine to use the general corpus and the knowledge-base to predict for the
knowledge or intelligence of similar words. The results of learning vary greatly. Therefore, the
selection and production of corpus and knowledge-base has become particularly important in
constructing the level of intelligence of the machine. Therefore, it is very important to select
and analyze the corpus or short text and knowledge-base before training. This thesis proposes
the development of three improved Chinese Word2Set based on corpus and knowledge-base.
Firstly, this thesis proposes the improvement of Chinese Word2Set that adds E-HowNet to
analyze word semantics on the basis of WordNet. Single-word semantic information and word-
VI
related words are very important for recognizing words. Single-word semantic information can
be provided by WordNet and E-HowNet. It is a message composed of synonyms and antonyms
of words. At the same time, improve the SVR of Chinese Word2Set, and use LSTM to solve
the gradient disappearance during training. Secondly, this thesis proposes to improve the use of
Word2Vec learning in Chinese Word2Set through the improved corpus and modify the weight
of Word2Vec hidden layer, so that the training is more accurate and the application in LDA.
Thirdly, this thesis proposes to use deep learning to rebuild the knowledge-base to improve the
function of Chinese Word2Set. The proposed method uses deep learning to learn the improved
knowledge-base, making the application of Chinese Word2Set more accurate. The experimental
results verify that the improved corpus and knowledge-base can train the machine more
effectively to predict similar words more accurately.

摘要......i
Abstract......iii
誌謝......v
Table of Contents......vi
List of Figures......viii
Chapter 1 Introduction......1
1.1 Artificial Intelligence......1
1.2 Natural Language Processing......1
1.3 Corpus and Knowledge-Base......2
1.4 Motivation and Contribution......3
1.4 Organization of This Thesis......4
Chapter 2 Preliminary......6
2.1 WordNet......6
2.2 Word2Set......6
2.3 Corpus......11
2.4 Knowledge-Base......11
2.5 Word2Vec......12
Chapter 3 Improved Knowledge-Base with E-HowNet and LSTM for Chinese Word2Set under Corpus with Word2Vec......14
3.1 The Difference between E-HowNet and WordNet Related Words......14
3.2 Proposed Method......16
3.3 Training Data......20
3.4 Experimental Results......25
3.5 Concluding Remarks......32
Chapter 4 Improved Knowledge-Base with E-HowNet and Improved Corpus with Word2Vec for Chinese Word2Set under Corpus with Word2Vec and Its application on LDA......33
4.1 Word Preprocessing......33
4.2 Neural Word Embedding......35
4.3 Proposed Method......36
4.4 Training Data......40
4.5 Experimental Results......41
4.6 Discussion and Concluding Remarks......47
Chapter 5 Specific Purpose under Word2Vec Knowledge-Base for Improved Chinese Word2Set under Corpus with Word2Vec......48
5.1 Feature Extraction of Word......48
5.2 Proposed Method......48
5.3 Training Data......50
5.4 Experimental Results......52
5.5 Concluding Remarks......59
Chapter 6 Conclusions......60
References......61
Extended Abstract......67

[1] 鄭耀騰。中文Word2Set的開發與在字詞與短文的相似度應用。國立虎尾科技大學資訊工程系碩士班碩士論文，雲林縣（2020）。
[2] S. Jimenez, F. A. Gonzalez, A. Gelbukh and G. Duenas, "word2set: WordNet-Based Word Representation Rivaling Neural Word Embedding for Lexical Similarity and Sentiment Analysis," in IEEE Computational Intelligence Magazine, vol. 14, no. 2, pp. 41-53, May (2019)
[3] T. Ergen and S. S. Kozat, "Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences," in IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 10, pp. 5159-5165, Oct. (2018).
[4] M. Zhao, Z. Zhang, T. W. S. Chow and Z. Wu, "On the Theoretical and Computational Analysis between Trace Ratio LDA and null-space LDA," The 2012 International Joint Conference on Neural Networks (IJCNN), (2012).
[5] S. Singh and A. Mahmood, "The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures," in IEEE Access, vol. 9, pp. 68675-68702, (2021).
[6] 機器之心, "Word2Vec Algorithm Combing ," September 4, (2019). (https://www.gushiciku.cn/pl/29R2/zh-tw)
[7] AINLP, "Explain the Principle of Word2Vec in Simple Terms," March 31, (2020). (https://www.gushiciku.cn/pl/pVET/zh-tw)
[8] Youngmi huang, "NLTK 初學指南(二)：由外而內，從語料庫到字詞拆解—上手篇," Jul 30, (2018). ( https://medium.com/pyladies-taiwan/nltk-%E5%88%9D%E5%AD%B8%E6%8C%87%E5%8D%97-%E4%BA%8C-%E7%94%B1%E5%A4%96%E8%80%8C%E5%85%A7-%E5%BE%9E%E8%AA%9E%E6%96%99%E5%BA%AB%E5%88%B0%E5%AD%97%E8%A9%9E%E6%8B%86%E8%A7%A3-%E4%B8%8A%E6%89%8B%E7%AF%87-e9c632d2b16a)
[9] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory", Neural Comput., vol. 9, no. 8, pp. 1735-1780, (1997).
[10] S. Song, J. C. K. Lam, Y. Han and V. O. K. Li, "ResNet-LSTM for Real-Time PM2.5 and PM₁₀ Estimation Using Sequential Smartphone Images," in IEEE Access, vol. 8, pp. 220069-220082, (2020).
[11] X. Shu, L. Zhang, Y. Sun and J. Tang, "Host–Parasite: Graph LSTM-in-LSTM for Group Activity Recognition," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 2, pp. 663-674, Feb. (2021).
[12] W. Zheng, B. Ge and C. Wang, "Building a TIN-LDA Model for Mining Microblog Users’ Interest," in IEEE Access, vol. 7, pp. 21795-21806, (2019).
[13] M. Zhao, Z. Zhang, T. W. S. Chow and Z. Wu, "On the Theoretical and Computational Analysis between Trace Ratio LDA and Null-space LDA," The 2012 International Joint Conference on Neural Networks (IJCNN), 2012, pp. 1-7.
[14] X. Xu, H. Li, W. Xu, Z. Liu, L. Yao and F. Dai, "Artificial Intelligence for Edge Service Optimization in Internet of Vehicles: A survey," in Tsinghua Science and Technology, vol. 27, no. 2, pp. 270-287, April (2022).
[15] K. -L. A. Yau et al., "Augmented Intelligence: Surveys of Literature and Expert Opinion to Understand Relations Between Human Intelligence and Artificial Intelligence," in IEEE Access, vol. 9, pp. 136744-136761, (2021).
[16] Y. -S. Ong and A. Gupta, "AIR5: Five Pillars of Artificial Intelligence Research," in IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 3, no. 5, pp. 411-415, Oct. (2019).
[17] Y. Azan Basallo, V. Estrada Senti and N. Martinez Sanchez, "Artificial Intelligence Techniques for Information Security Risk Assessment," in IEEE Latin America Transactions, vol. 16, no. 3, pp. 897-901, March (2018).
[18] T. Patten and P. Jacobs, "Natural-Language Processing," in IEEE Expert, vol. 9, no. 1, pp. 35-, Feb. (1994).
[19] J. L. Verdu-Mas, R. C. Carrasco and J. Calera-Rubio, "Parsing with Probabilistic Strictly Locally Testable Tree Languages," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 7, pp. 1040-1050, July (2005).
[20] O. G. Fragoso-Diaz, V. López-Caballero, J. C. Rojas-Pérez, R. Santaolaya-Salgado and J. G. González-Serna, "On the Generation of E-Learning Resources Using Business Process, Natural Language Processing, and Web Services," in IT Professional, vol. 23, no. 2, pp. 40-44, 1 March-April (2021).
[21] M. T. Mills and N. G. Bourbakis, "Graph-Based Methods for Natural Language Processing and Understanding—A Survey and Analysis," in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 44, no. 1, pp. 59-71, Jan. (2014).
[22] M. -H. Chu, W. -Y. Chen and S. -D. Lin, "A Learning-Based Framework to Utilize E-HowNet Ontology and Wikipedia Sources to Generate Multiple-Choice Factual Questions," 2012 Conference on Technologies and Applications of Artificial Intelligence, 2012, pp. 125-130.
[23] P. Bello López and G. De Ita Luna, "An Algorithm to Belief Revision and to Verify Consistency of a Knowledge-Base," in IEEE Latin America Transactions, vol. 19, no. 11, pp. 1867-1874, Nov. ( 2021).
[24] K. Niwa, "Toward Successful Implementation of Knowledge-Based Systems: Expert Systems versus Knowledge Sharing Systems," in IEEE Transactions on Engineering Management, vol. 37, no. 4, pp. 277-283, Nov. (1990).
[25] K. Liu, J. Zhao, S. He and Y. Zhang, "Question Answering over Knowledge-Bases," in IEEE Intelligent Systems, vol. 30, no. 5, pp. 26-35, Sept.-Oct. (2015).
[26] Z. Bi, T. Zhang, P. Zhou and Y. Li, "Knowledge Transfer for Out-of-Knowledge-Base Entities: Improving Graph-Neural-Network-Based Embedding Using Convolutional Layers," in IEEE Access, vol. 8, pp. 159039-159049, (2020).
[27] P. Bello López and G. De Ita Luna, "An Algorithm to Belief Revision and to Verify Consistency of a Knowledge-Base," in IEEE Latin America Transactions, vol. 19, no. 11, pp. 1867-1874, Nov. (2021).
[28] C. Qiu, G. Zhou, Z. Cai and A. Søgaard, "A Global–Local Attentive Relation Detection Model for Knowledge-Based Question Answering," in IEEE Transactions on Artificial Intelligence, vol. 2, no. 2, pp. 200-212, April (2021).
[29] M. Gnjatović and D. Rösner, "Inducing Genuine Emotions in Simulated Speech-Based Human-Machine Interaction: The NIMITEK Corpus," in IEEE Transactions on Affective Computing, vol. 1, no. 2, pp. 132-144, July-Dec. (2010).
[30] H. Su and C. Wu, "Improving Structural Statistical Machine Translation for Sign Language with Small Corpus Using Thematic Role Templates as Translation Memory," in IEEE Transactions on Audio, Speech, and Language Processing, vol. 17, no. 7, pp. 1305-1315, Sept. (2009).
[31] T. Nagata and H. Mori, "Defining Laughter Context for Laughter Synthesis with Spontaneous Speech Corpus," in IEEE Transactions on Affective Computing, vol. 11, no. 3, pp. 553-559, 1 July-Sept. (2020).
[32] R. K. Boettger and S. Wulff, "Gender Effects in Student Technical and Scientific Writing—A Corpus-Based Study," in IEEE Transactions on Professional Communication, vol. 62, no. 3, pp. 239-252, Sept. (2019).
[33] A. Lamurias, D. Sousa and F. M. Couto, "Generating Biomedical Question Answering Corpora from Q&A Forums," in IEEE Access, vol. 8, pp. 161042-161051, (2020).
[34] C. C. Hayes and M. I. Parzen, "QUEM: an Achievement Test for Knowledge-Based Systems," in IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 6, pp. 838-847, Nov.-Dec. (1997).
[35] C. Qiu, G. Zhou, Z. Cai and A. Søgaard, "A Global–Local Attentive Relation Detection Model for Knowledge-Based Question Answering," in IEEE Transactions on Artificial Intelligence, vol. 2, no. 2, pp. 200-212, April (2021).
[36] S. Ji, N. Satish, S. Li and P. K. Dubey, "Parallelizing Word2Vec in Shared and Distributed Memory," in IEEE Transactions on Parallel and Distributed Systems, vol. 30, no. 9, pp. 2090-2100, (2019).
[37] J. Li, H. Zhang and Z. Wei, "The Weighted Word2vec Paragraph Vectors for Anomaly Detection Over HTTP Traffic," in IEEE Access, vol. 8, pp. 141787-141798, (2020).
[38] Y. Chen, S. Huang, H. Lee, Y. Wang and C. Shen, "Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 9, pp. 1481-1493, Sept. (2019).
[39] X. Chen, C. Chen, D. Zhang and Z. Xing, "SEthesaurus: WordNet in Software Engineering," in IEEE Transactions on Software Engineering, vol. 47, no. 9, pp. 1960-1979, 1 Sept. (2021).
[40] C. Panayiotou, "Extraction of Poetic and Non-Poetic Relations From of-Prepositions Using WordNet," in IEEE Access, vol. 10, pp. 3469-3494, (2022).
[41] C. Shih, Y. Hsieh and W. Hsu, "Sense decomposition from E-HowNet for word similarity measurement," Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014), 2014, pp. 613-618.
[42] Y. Ding, F. Teng, P. Zhang, X. Huo, Q. Sun and Y. Qi, "Research on Text Information Mining Technology of Substation Inspection Based on Improved Jieba," 2021 International Conference on Wireless Communications and Smart Grid (ICWCSG), 2021, pp. 561-564.
[43] S. Lei, "Research on the Improved Word2Vec Optimization Strategy Based on Statistical Language Model," (2020) International Conference on Information Science, Parallel and Distributed Systems (ISPDS), 2020, pp. 356-359.
[44] H. Tian and L. Wu, "Microblog Emotional Analysis Based on TF-IWF Weighted Word2vec Model," 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), 2018, pp. 893-896.
[45] J. Gao, Y. He, X. Zhang and Y. Xia, "Duplicate short text detection based on Word2vec," 2017 8th IEEE International Conference on Software Engineering and Service Science (ICSESS), 2017, pp. 33-37.
[46] L. Xiang, J. Yu, C. Yang, D. Zeng and X. Shen, "A Word-Embedding-Based Steganalysis Method for Linguistic Steganography via Synonym Substitution," in IEEE Access, vol. 6, pp. 64131-64141, (2018).
[47] A. Mathur and G. M. Foody, "Multiclass and Binary SVM Classification: Implications for Training and Classification Users," in IEEE Geoscience and Remote Sensing Letters, vol. 5, no. 2, pp. 241-245, April (2008).
[48] K. -A. Toh and H. -L. Eng, "Between Classification-Error Approximation and Weighted Least-Squares Learning," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 658-669, April (2008).
[49] N. Almusallam, Z. Tari, J. Chan, A. Fahad, A. Alabdulatif and M. Al-Naeem, "Towards an Unsupervised Feature Selection Method for Effective Dynamic Features," in IEEE Access, vol. 9, pp. 77149-77163, (2021).
[50] D. Embick, "On the Structure of Resultative Participles in English," in Linguistic Inquiry, vol. 35, no. 3, pp. 355-392, July (2004).