臺灣博碩士論文加值系統

在互聯網興起的環境下，機器的訊息和知識組成形式傾向於由網際網路中各式各樣的短文本或短篇章的文件上獲取內容。然而，由於短文本在內容的同義字或近義詞的用字不同的特性，使得短文本的內容在預測或分類的落差過大，導致於機器在預測文本時會受學習時的既有文字資料之用字的差異，產生短文本的預測與分類之誤判。因此，在短文本處理時需將短文本中的單詞內容進行語意分析是非常重要的，因此本論文提出的以中文WordNet為基礎的中文Word2Set的開發與整合Word2Vec到中文Word2Set的改進，透過這兩種方法的整合來做為單詞比較的語意分析。基於單詞比較的語意分析合併至文本內容進行分析，本論文提出的整合Word2Set與Word2Vec對潛在狄利克雷分配的改進的方法做為文本比較的語意分析，降低機器在預測時受到用詞的差異而產生的誤判。
在單詞比較的語意分析的任務需求，以人工智慧中自然語言處理的領域是最為合適。在單詞的辨識上，單詞的語意訊息和單詞的關聯詞對於識別單詞非常重要。單詞的語意訊息由具有語意訊息的語料庫中文WordNet提供，以單詞的同義詞和反義詞所組成的訊息。根據語言體系上的漢語與英文的表意差異，開發出適合於漢語體系的中文Word2Set。單詞的關聯詞訊息由詞嵌入中的Word2Vec模型透過外部的語料庫訓練得到的關聯詞所組成。因此提出整合Word2Vec到中文Word2Set的改進，透過這兩種方法的整合來做為單詞比較的語意分析。最後，透過同義詞、反義詞及近義詞的測試來評估所提出方法的有效性。
在短文本的語意比較分析應用上，使用潛在狄利克雷分配找出文本的主題組成來比對未知文本的相似度。在主題的單詞組成上是基於訓練集的詞彙表和句子中的詞頻矩陣所組成。透過以中文WordNet為基礎的中文Word2Set的開發、整合Word2Vec到中文Word2Set的改進，透過這兩種方法的整合來改善詞頻矩陣中的詞頻統計，並且可以 擴展LDA的詞彙內容。 實驗結果表明，該方法可以有效地區分短文中具有相似含義的不同單詞，從而可以區分同義詞和近義詞，從而比較未知短文的相似性。

About the context on the rise of the Internet using, machine information and knowledge forms tend to be obtained from a variety of short text or short chapter documents on the Internet. However, due to the different characteristics of words in short text for the content of synonyms or near-synonyms, the content of short text has a large gap in prediction or classification between two short texts. The results in the machine's prediction of the text data will be affected by the existing text information and use of the synonyms or near-synonyms text. The difference of the words causes misjudgment of prediction and classification of short text. Therefore, it is very important to analyze the semantic of words in short texts when processing short texts. Therefore, the development of Chinese Word2Set based on Chinese WordNet and the improvement of integrated Word2Vec to Chinese Word2Set are proposed in this thesis. This method is integrated for semantic analysis of word comparison. The semantic analysis based on word comparison is merged into the content of text for analysis. This thesis also proposes an improved method that integrates Word2Set and Word2Vec for Latent Dirichlet Allocation (LDA) as a semantic analysis for comparison of short text, which reduces the machine's exposure to the synonyms or near-synonyms word differences in prediction misjudged.
In the task of semantic analysis of word comparison, the field of natural language processing (NLP) in artificial intelligence is the most suitable tool. In word recognition, the semantic information of words and the related words of words are very important on recognize words. The semantic information of a word is provided by the corpus Chinese WordNet with semantic information. It can provide a message composed of synonyms and antonyms of the word. Based on the ideographic differences between Chinese and English on the language system, a Chinese Word2Set suitable for the Chinese system was developed in this thesis. Besides, the related information of the word is composed of related words obtained by the Word2Vec model that embedded in the word through external corpus training. Therefore, the improvement of integrated Word2Vec into Chinese Word2Set is proposed, and the integration of these two methods is also used as a semantic analysis of word comparison. Finally, the effectiveness of the proposed method is evaluated by testing synonyms, antonyms, and near-synonyms in this thesis.
In the application of semantic comparison analysis for short texts, LDA is used to find the subject composition of short texts and to compare the similarity of unknown short texts. The topic's word composition is based on the training set vocabulary and the word frequency matrix in exist short texts. Through the development of Chinese Word2Set based on Chinese WordNet and the improvement of integrated Word2Vec into Chinese Word2Set, the integration of these two methods can improve the term frequency statistics in the word frequency matrix under the addition with synonyms and near-synonyms of words and can expand the LDA in content of the vocabulary. For an effective comparison of different words with similar meanings on short texts, the experimental results show that the proposed method can make the machine that distinguish synonyms and near-synonyms to compare the similarity of unknown short texts.

摘要.....i
Abstract.....iii
誌謝.....v
Table of Contents.....vi
List of Tables.....viii
List of Figures.....x
Chapter 1 Introduction.....1
1.1 Natural Language Processing.....1
1.2 Words Representation.....2
1.3 Information Retrieval.....2
1.4 Motivation and Contribution.....3
1.5 Organization of This Thesis.....5
Chapter 2 Introduction to Chinese WordNet and WordNet.....6
2.1 WordNet.....6
2.2 Chinese WordNet.....6
2.3 WordNet of Word Representation.....9
2.4 Semantic Analysis.....9
Chapter 3 Development of Chinese Word2Set based on Chinese WordNet.....11
3.1 Morphemic differences between Chinese characters and Words.....11
3.2 Proposed Method.....12
3.3 Training Data.....18
3.4 Experimental Results.....23
3.5 Discussion and Conclude Remarks.....43
Chapter 4 Improvement of Proposed Chinese Word2Set with Word2Vec.....44
4.1 Word Preprocessing.....44
4.2 Neural Word Embedding.....45
4.3 Proposed Method.....50
4.4 Experimental Results.....53
4.5 Discussion and Conclude Remarks.....68
Chapter 5 Improvement of Latent Dirichlet Allocations with Proposed Chinese Word2Set with Word2Vec.....69
5.1 Latent Dirichlet Allocation.....69
5.2 Proposed Method.....69
5.3 Training Data.....76
5.4 Experimental Results.....76
5.5 Discussion and Conclude Remarks.....79
Chapter 6 Conclusions.....80
References.....82
Extended Abstract.....88

[1] Gruzd, Anatoliy, Barry Wellman, and Yuri Takhteyev. "Imagining twitter as an imagined community." American Behavioral Scientist 55.10 (2011): 1294-1318.
[2] Dahlgren, Kathleen, and Joyce McDowell. "Knowledge representation for commonsense reasoning with text." Computational linguistics 15.3 (1989): 149-170.
[3] Chowdhury, Gobinda G. "Natural language processing." Annual review of information science and technology 37.1 (2003): 51-89.
[4] Hua, Wen, et al. "Short text understanding through lexical-semantic analysis." 2015 IEEE 31st International Conference on Data Engineering. IEEE, 2015.
[5] Fellbaum, Christiane. "WordNet." Theory and applications of ontology: computer applications. Springer, Dordrecht, 2010. 231-243.
[6] Loi, Chek Kim, and Moyra Sweetnam Evans. "Cultural differences in the organization of research article introductions from the field of educational psychology: English and Chinese." Journal of Pragmatics 42.10 (2010): 2814-2825.
[7] Schmitt, Bernd H., Yigang Pan, and Nader T. Tavassoli. "Language and consumer memory: The impact of linguistic differences between chinese and english." Journal of Consumer Research 21.3 (1994): 419-431.
[8] Chen, Jenn-Yeu, and Train-Min Chen. "Form encoding in chinese word production does not involve morphemes." Language and Cognitive Processes 22.7 (2007): 1001-1020.
[9] Huang, Chu-Ren, et al. "Chinese WordNet: Design, implementation, and application of an infrastructure for cross-lingual knowledge processing." Journal of Chinese Information Processing 24.2 (2010): 14-23.
[10] Metzler, Donald, Susan Dumais, and Christopher Meek. "Similarity measures for short segments of text." European conference on information retrieval. Springer, Berlin, Heidelberg, 2007.
[11] Abowd, Gregory D., et al. "Towards a better understanding of context and context-awareness." International symposium on handheld and ubiquitous computing. Springer, Berlin, Heidelberg, 1999.
[12] Dey, Anind K. "Understanding and using context." Personal and ubiquitous computing 5.1 (2001): 4-7.
[13] Tweedy, James R., Robert H. Lapinski, and Roger W. Schvaneveldt. "Semantic-context effects on word recognition: Influence of varying the proportion of items presented in an appropriate context." Memory & Cognition 5.1 (1977): 84-89.
[14] MacDonald, Maryellen C., Neal J. Pearlmutter, and Mark S. Seidenberg. "The lexical nature of syntactic ambiguity resolution." Psychological review 101.4 (1994): 676.
[15] Altmann, Gerry TM. "Ambiguity in sentence processing." Trends in cognitive sciences 2.4 (1998): 146-152.
[16] Li, Yang, and Tao Yang. "Word embedding for understanding natural language: a survey." Guide to Big Data Applications. Springer, Cham, 2018. 83-104.
[17] Zamani, Hamed, and W. Bruce Croft. "Relevance-based word embedding." Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017.
[18] Blei, David M., Andrew Y. Ng, and Michael I. Jordan. "Latent dirichlet allocation." Journal of machine Learning research 3.Jan (2003): 993-1022.
[19] Hofmann, Thomas. "Unsupervised learning by probabilistic latent semantic analysis." Machine learning 42.1-2 (2001): 177-196.
[20] Wiemer-Hastings, Peter. "How latent is latent semantic analysis?." Proceedings of the 16th international joint conference on Artificial intelligence-Volume 2. 1999.
[21] Landauer, Thomas K., Peter W. Foltz, and Darrell Laham. "An introduction to latent semantic analysis." Discourse processes 25.2-3 (1998): 259-284.
[22] Louwerse, Max, et al. "Cognitively inspired NLP-based knowledge representations: Further explorations of latent semantic analysis." International Journal on Artificial Intelligence Tools 15.06 (2006): 1021-1039.
[23] Jimenez, Sergio, et al. "word2set: WordNet-based word representation rivaling neural word embedding for lexical similarity and sentiment analysis." IEEE Computational Intelligence Magazine 14.2 (2019): 41-53.
[24] Androutsopoulos, Ion, and Prodromos Malakasiotis. "A survey of paraphrasing and textual entailment methods." Journal of Artificial Intelligence Research 38 (2010): 135-187.
[25] Jimenez, Sergio, Claudia Becerra, and Alexander Gelbukh. "Soft cardinality+ ML: learning adaptive similarity functions for cross-lingual textual entailment." Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2012.
[26] Jimenez, Sergio, Claudia Becerra, and Alexander Gelbukh. "Softcardinality-core: Improving text overlap with distributional measures for semantic textual similarity." Second Joint Conference on Lexical and Computational Semantics (* SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity. 2013.
[27] De Baets, Bernard, Hans De Meyer, and Helga Naessens. "A class of rational cardinality-based similarity measures." Journal of Computational and Applied Mathematics 132.1 (2001): 51-69.
[28] Sidorov, Grigori, et al. "Soft similarity and soft cosine measure: Similarity of features in vector space model." Computación y Sistemas 18.3 (2014): 491-504.
[29] Shen, Kai-Quan, et al. "Feature selection via sensitivity analysis of SVM probabilistic outputs." Machine Learning 70.1 (2008): 1-20.
[30] Chandrashekar, Girish, and Ferat Sahin. "A survey on feature selection methods." Computers & Electrical Engineering 40.1 (2014): 16-28.
[31] Pedregosa, Fabian, et al. "Scikit-learn: Machine learning in python." Journal of machine learning research 12.Oct (2011): 2825-2830.
[32] Picard, Richard R., and R. Dennis Cook. "Cross-validation of regression models." Journal of the American Statistical Association 79.387 (1984): 575-583.
[33] Luor, Dah-Chin. "A comparative assessment of data standardization on support vector machine for classification problems." Intelligent Data Analysis 19.3 (2015): 529-546.
[34] Bisong, Ekaba. "Introduction to Scikit-learn." Building Machine Learning and Deep Learning Models on Google Cloud Platform. Apress, Berkeley, CA, 2019. 215-229.
[35] Al-anazi, Ammal F., and Ian D. Gates. "Support-vector regression for permeability prediction in a heterogeneous reservoir: A comparative study." SPE Reservoir Evaluation & Engineering 13.03 (2010): 485-495.
[36] Nelson, T. M., and C. J. Ladan. "Chinese ideograms as intermediaries to reading." Journal of Reading Behavior 9.2 (1977): 167-174.
[37] Liu, Xue. "Contrastive study on similarities and differences between chinese and english characters." 2017 International Conference on Culture, Education and Financial Development of Modern Society (ICCESE 2017). Atlantis Press, 2017.
[38] Hua, L. I. U. "The theoretical and experimental justifications of phrase advantage in chinese information processing [J]." Applied Linguistics 4 (2007): 129-135.
[39] Yang, Heng-Li, and August FY Chao. "Sentiment analysis for chinese reviews of movies in multi-genre based on morpheme-based features and collocations." Information Systems Frontiers 17.6 (2015): 1335-1352.
[40] Arcodia, Giorgio Francesco. "Chinese: a language of compound words." Selected proceedings of the 5th Décembrettes: Morphology in Toulouse (2007): 79-90.
[41] Chunyan, A. N. "Definition and discrimination of synonyms in modern Chinese [J]." Journal of Liaoning Technical University (Social Science Edition) 5 (2011).
[42] Zhang, Zheng-Sheng. "Visualizing stylistic differences in chinese synonyms." Computational and Corpus Approaches to Chinese Language Learning. Springer, Singapore, 2019. 145-172.
[43] Tanaka-Ishii, Kumiko, and Zhihui Jin. "From phoneme to morpheme: Another verification using a corpus." International Conference on Computer Processing of Oriental Languages. Springer, Berlin, Heidelberg, 2006.
[44] Hoosain, Rumjahn. "Psychological reality of the word in chinese." Advances in psychology. Vol. 90. North-Holland, 1992. 111-130.
[45] Handler, Abram. "An empirical study of semantic similarity in WordNet and Word2vec." (2014).
[46] Goldberg, Yoav, and Omer Levy. "Word2vec explained: Deriving Mikolov et al.'s negative-sampling word-embedding method." arXiv preprint arXiv:1402.3722 (2014).
[47] McCormick, Chris. "Word2vec tutorial-the skip-gram model." (2016).
[48] Inkpen, Diana Zaiu, and Graeme Hirst. "Acquiring collocations for lexical choice between near-synonyms." Proceedings of the ACL-02 workshop on Unsupervised lexical acquisition-Volume 9. Association for Computational Linguistics, 2002.
[49] Reiter, Ehud, and Somayajulu Sripada. "Contextual influences on near-synonym choice." International Conference on Natural Language Generation. Springer, Berlin, Heidelberg, 2004.
[50] Pigada, Maria, and Norbert Schmitt. "Vocabulary acquisition from extensive reading: A case study." Reading in a foreign language 18.1 (2006): 1-28.
[51] Wang, Yile, Leyang Cui, and Yue Zhang. "Using dynamic embeddings to Improve Static embeddings." arXiv preprint arXiv:1911.02929 (2019).
[52] Liang, Wenxin, et al. "GLTM: A global and local word embedding-based topic model for short texts." IEEE access 6 (2018): 43612-43621.
[53] Li, Chao, et al. "Mining dynamics of research topics based on the combined LDA and WordNet." IEEE Access 7 (2018): 6386-6399.
[54] Fuhr, Norbert. "Probabilistic models in information retrieval." The computer journal 35.3 (1992): 243-255.
[55] Zhai, ChengXiang. "Probabilistic topic models for text data retrieval and analysis." Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. 2017.
[56] Yan, Xiaohui, et al. "A biterm topic model for short texts." Proceedings of the 22nd international conference on World Wide Web. 2013.
[57] Al-Rfou, Rami, Bryan Perozzi, and Steven Skiena. "Polyglot: Distributed word representations for multilingual NLP." arXiv preprint arXiv:1307.1662 (2013).
[58] Bai, Jing, et al. "Using query contexts in information retrieval." Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. 2007.
[59] Ganguly, Debasis, et al. "Word embedding based generalized language model for information retrieval." Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 2015.
[60] Salton, Gerard, and Christopher Buckley. "Term-weighting approaches in automatic text retrieval." Information processing & management 24.5 (1988): 513-523.
[61] Mikolov, Tomas, et al. "Efficient estimation of word representations in vector space." arXiv preprint arXiv:1301.3781 (2013).
[62] Green, Robin AR. "Document retrieval system and search method using word set and character look-up tables." U.S. Patent No. 6,741,985. 25 May 2004.
[63] Chen, Xinxiong, Zhiyuan Liu, and Maosong Sun. "A unified model for word sense representation and disambiguation." Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014.
[64] Gong, Chengyue, et al. "Frage: Frequency-agnostic word representation." Advances in neural information processing systems. 2018.
[65] Řehůřek, Radim, and Petr Sojka. "Gensim—statistical semantics in python." Retrieved from genism. org (2011).
[66] Wan, Shen, and Rafal A. Angryk. "Measuring semantic similarity using wordnet-based context vectors." 2007 IEEE International Conference on Systems, Man and Cybernetics. IEEE, 2007.
[67] Liu, Hongzhe, Hong Bao, and De Xu. "Concept vector for semantic similarity and relatedness based on WordNet structure." Journal of Systems and software 85.2 (2012): 370-381.
[68] Malakasiotis, Prodromos, and Ion Androutsopoulos. "Learning textual entailment using SVMs and string similarity measures." Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing. 2007.
[69] Inkpen, Diana, Darren Kipp, and Vivi Nastase. "Machine learning experiments for textual entailment." Proceedings of the Second Challenge Workshop Recognising Textual Entailment. 2006.
[70] Fortson IV, Benjamin W. "An approach to semantic change." The handbook of historical linguistics (2017): 648-666.
[71] Buitinck, Lars, et al. "API design for machine learning software: experiences from the scikit-learn project." arXiv preprint arXiv:1309.0238 (2013).
[72] Srinivasa-Desikan, Bhargav. Natural Language Processing and Computational Linguistics: A practical guide to text analysis with Python, Gensim, spaCy, and Keras. Packt Publishing Ltd, 2018.
[73] Perkins, Jacob. Python 3 text processing with NLTK 3 cookbook. Packt Publishing Ltd, 2014.
[74] Junyi, Sun. "Jieba python library." (2015).
[75] Lehnert, Wendy G., and Martin H. Ringle, eds. Strategies for natural language processing. Psychology Press, 2014.
[76] Bullinaria, John A., and Joseph P. Levy. "Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD." Behavior research methods 44.3 (2012): 890-907.
[77] Meng, Lingling, Runqing Huang, and Junzhong Gu. "A review of semantic similarity measures in wordnet." International Journal of Hybrid Information Technology 6.1 (2013): 1-12.
[78] Faruqui, Manaal, and Chris Dyer. "Non-distributional word vector representations." arXiv preprint arXiv:1506.05230 (2015).