臺灣博碩士論文加值系統

在現今社會中，有許多重大決策在分析時存在著許多主觀因素，因此在決策時，會因為個人的定義不同而產生分歧。傳統的模糊集只能透過歸屬度函數，表達了資料的模糊情況。因此在1986年，Krassimir T. Atanassov提出了直覺模糊集(Intuitionistic Fuzzy Sets，IFS)，IFS透過歸屬度函數、非歸屬度函數以及直覺函數，可對支持與反對的二種資料表達了資料的模糊情況。但是IFS最大的問題在於分類與回歸的依據完全依靠專家的結論和過往經驗，無法透過分析輸入資料和已知知識去推論出其模型，因此後續又提出了基於自適應網路的直覺模糊推論系統(Adaptive Network-Based Fuzzy Inference System, ANFIS)，利用已知知識以及輸入輸出數據建立神經網路模型。本研究透過同時分析歸屬度函數以及非歸屬度函數，改善了ANFIS在歸屬度較低時，無法精準判斷的問題。同時加入了直覺函數，使資料集能夠更接近實際情況，也讓本研究能夠在實際情況能擁有同樣精準的判斷能力。本篇論文於第三章先討論若是在TSK模糊系統(Takagi-Sugeno-Kang fuzzy system, TSK fuzzy system)內加入β−dropout，將參數過低的規則拋棄的話，能否有效地降低整體模型的複雜度，並加快整體的訓練時間。接著在第四章的部分，由於目前的IFS ANFIS多數都使用一種模糊化與解模糊化方法訓練，於是本篇論文改良兩種常見的模糊化與解模糊化方法，並另外與一種模糊化與解模糊化方法進行比較，嘗試將結果的準確度提高，同時確認是否能夠加快訓練速度。最後則在第五章的部分，將提出的IFS ANFIS架構與β−dropout結合，嘗試是否能夠擁有更快的訓練速度與更優秀的表現。

At present, there are many subjective factors in the analysis of many major decisions, so when making decisions, there will be differences due to different definitions of individuals. Traditional fuzzy sets can only express the fuzzy situation of data through the membership function. Therefore, in 1986, Krassimir T. Atanassov proposed intuitionistic fuzzy sets (Intuitionistic Fuzzy Sets, IFS), IFS through the membership function, the non- membership function and the intuitionistic function, can express the support and objection of the two kinds of data fuzzy situation. However, the biggest problem with IFS is that the basis of classification and regression depends entirely on the conclusions and past experiences of experts, and it is impossible to infer its model by analyzing input data and known knowledge. Therefore, an Adaptive-Network-Based Fuzzy Inference System (ANFIS) which uses known knowledge and input and output data to establish a neural network model was proposed. By analyzing the membership function and non- membership function at the same time, this study has improved the problem that ANFIS cannot accurately judge when the attribution is low. In Chapter 3 of this thesis discusses whether the introduction of β−dropout into the Takagi-Sugeno-Kang fuzzy system (TSK fuzzy system), by discarding rules with overly low parameters that can effectively reduce the overall model complexity and accelerate the training time. In Chapter 4, this thesis discuss that the current IFS ANFIS primarily use one type of fuzzification and defuzzification method for training. Therefore, this thesis introduces two common methods of fuzzification and defuzzification and proposes an additional method. Its aims are to effectively improve regression results while speeding up the training process. Finally, in Chapter 5 of this thesis, it incorporates the improved IFS ANFIS under β−dropout. The thesis strives to shorten the proposed ANFIS training time and enhance its precision in regression effects, by achieving equal accuracy and faster training speed.

摘要...i
Abstract...iii
Acknowledgements...v
Table of Contents...vi
List of Tables...ix
List of Figures...x
Chapter 1 Introduction...1
1.1 Review of Fuzzy Sets...1
1.2 Review of Intuitionistic Fuzzy Sets...2
1.3 Review of β-Dropout...3
1.4 Review of Takagi-Sugeno Fuzzy System...7
1.5 Review of Fuzzy Inference System...8
1.6 Review of Adaptive-Network-Based Fuzzy Inference System...10
1.7 Motivations and Contributions...11
1.8 Organization of This Thesis...14
Chapter 2 Mathematical Background...15
2.1 Fuzzy Sets...15
2.2 Intuitionistic Fuzzy Sets...16
2.3 Takagi-Sugeno-Kang Fuzzy Rule and DropRule...17
2.4 β-dropout...20
2.5 Mini-Batch Gradient Descent with Regularization...24
2.6 Fuzzy Inference System...26
2.7 Takagi-Sugeno Fuzzy System...29
2.8 Adaptive-Network-Based Fuzzy Inference System...32
Chapter 3 If-Then Rule and β-Dropout...45
3.1 Dropout and β-Dropout...45
3.2 System Architecture...48
3.3 Simulation Results...53
3.4 Conclude Remarks...55
Chapter 4 Fuzzification and Defuzzification for IFS ANFIS...56
4.1 Detail of ANFIS: Adaptive-Network-Based Fuzzy Inference System...56
4.2 Detail Comparison of Others Fuzzification and Defuzzification of ANFIS Structures...57
4.3 Proposed Fuzzification and Defuzzification for IFS ANFIS...61
4.4 Details of the Proposed IFS ANFIS Structure...67
4.5 Simulation Results...74
4.6 Conclude Remarks...95
Chapter 5 Proposed IFS ANFIS with β-Dropout...96
5.1 β-Dropout in the Benefit and Weakness of IFS ANFIS...96
5.2 Proposed IFS ANFIS with β-Dropout...98
5.3 Simulation Results...104
5.4 Conclude Remarks...122
Chapter 6 Conclusion...123
References...125
Extended Abstract...131

[1]Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, vol. 8, no. 3, pp. 338-353.
[2]Atanassov, K. (1986). Intuitionistic Fuzzy Sets. Fuzzy Sets and Systems, vol. 20, no. 1, pp. 87-96.
[3]Nguyen, A. & Taniguchi, T. & Eciolaza, L.& Campos, V.& Palhares, R. and Sugeno, M. (2019). Fuzzy Control Systems: Past present and Future. IEEE Comput. Intell. Mag., vol. 14, no. 1, pp. 56-68.
[4]L. Liu, Y. Luo, X. Shen, M. Sun and B. Li, (2019) β-Dropout: A Unified Dropout. IEEE Access, vol. 7, pp. 36140-36153, doi: 10.1109/ACCESS.2019.2904881.
[5]Mamdani, E. H. (1974). Application of Fuzzy Algorithms for Control of Simple Dynamic Plant. Proceedings of the Institution of Electrical Engineers, vol. 121, no. 12, pp. 1585-1588.
[6]Mendel, J. M. (1995). Fuzzy Logic Systems for Engineering: A Tutorial. Proceedings of the IEEE, vol. 83, no. 3, pp. 345-377.
[7]Klir, G. J. & Yuan, B. (1995). Fuzzy Sets and Fuzzy Logic: Theory and Applications. Prentice Hall.
[8]Bezdek, J. C. & Pal, N. R. (1992). Fuzzy Models for Pattern Recognition: Methods that Search for Structures in Data. IEEE Press selected reprint series.
[9]Takagi, T. and Sugeno, M. (1985). Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-15, no. 1, pp. 116-132, Jan.-Feb. doi: 10.1109/TSMC.1985.6313399.
[10]Terziyska, M. Todorov, Y. Doneva, M. et al. (2019). Distributed Adaptive Neuro Intuitionistic Fuzzy Architecture for prediction of the dose in gamma irradiated milk products. IFAC-PapersOnLine. Vol. 52, No. 25. pp. 75-80.
[11]Hajek, P. & Olej, V. (2014). Defuzzification Methods in Intuitionistic Fuzzy Inference Systems of Takagi-Sugeno Type: The Case of Corporate Bankruptcy Prediction, 2014 11th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Xiamen, China, pp. 232-236, doi: 10.1109/FSKD.2014.6980838.
[12]Hájek, P. & Olej, V. (2017). Intuitionistic Neuro-Fuzzy Network with Evolutionary Adaptation. Evolving Systems, vol. 8, pp. 35–47
[13]D. Wu & Y. Yuan & J. Huang and Y. Tan, (2020). Optimize TSK Fuzzy Systems for Regression Problems: Minibatch Gradient Descent with Regularization, DropRule, and AdaBound (MBGD-RDA), in IEEE Transactions on Fuzzy Systems, vol. 28, no. 5, pp. 1003-1015, doi: 10.1109/TFUZZ.2019.2958559.
[14]Hinton, G. E. & Srivastava, N. & Krizhevsky, A. & Sutskever, I. and Salakhutdinov. R. R. (2012). Improving Neural Networks by Preventing Co-adaptation of Feature Detectors. Department of Computer Science, University of Toronto, Canada.
[15]Srivastava, N. & Hinton, G. & Krizhevsky, A. & Sutskever, I. and. Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting, J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958
[16]Wan, L. & Zeiler, M. & Zhang, S. & Le Cun, Y. and Fergus, R. (2013). Regularization of Neural Networks using Dropconnect, in Proc. Int. Conf. Mach. Learn, pp. 1058–1066.
[17]Wang, S. and Manning, C. (2013). Fast Dropout Training, in Proc. Int. Conf. Mach. Learn., pp. 118–126.
[18]Ba, J. and Frey, B. (2013). Adaptive Dropout for Training Deep Neural Networks, in Proc. Proc. Adv. Neural Inf. Process. Syst., pp. 3084–3092.
[19]Rennie, S. J. & Goel, V. and Thomas, S. (2014). Annealed Dropout Training of Deep Networks, in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), pp. 159–164.
[20]Kingma, D. P. & Salimans, T. and Welling, M. (2015) Variational Dropout and The Local Reparameterization Trick, in Proc. Adv. Neural Inf. Process. Syst., pp. 2575–2583.
[21]Morerio, P. & Cavazza, J. & Volpi, R. & Vidal, R. and Murino, V. (2017). Curriculum Dropout.
[22]Li, Z. & Gong, B. and Yang, T. (2016) Improved Dropout for Shallow and Deep Learning, in Proc. Adv. Neural Inf. Process. Syst., pp. 2523–2531.
[23]Li, Y. & Xu, R. and Liu. F. (2016). Whiteout: Gaussian Adaptive Noise Regularization in Deep Neural Networks. arXiv:1612.01490.
[24]Srinivas, S. and Babu, R. V. (2016). Generalized Dropout. arXiv:1611.06791.
[25]Tsukamoto, Y. (1979). An Approach to Fuzzy Reasoning Method, in M. M. Gupta, R. K. Ragade, and R. R. Yager, Eds., Advances in Fuzzy Set Theory and Applications. Amsterdam: North-Holland, pp. 137-149.
[26]Lee, C.C. (1990). Fuzzy Logic in Control Systems: Fuzzy Logic Controller-Part I, IEEE Trans. Syst., Man, Cybern., vol. 20, pp. 404418.
[27]Cybern, Man. (1990). Fuzzy Logic in Control Systems: Fuzzy Logic Controller-Part I, IEEE Trans. Syst., vol. 20, pp. 419435.
[28]Takagi, T. and Sugeno, M. (1983). Derivation of Fuzzy Control Rules from Human Operator’s Control Actions, in Proc. IFAC Symp. Fuzzy Inform., Knowledge Representation and Decision Analysis, pp. 55-60.
[29]Baldi, P. and Sadowski, P. (2014). The Dropout Learning Algorithm, Artif. Intell., vol. 210, pp. 78–122.
[30]Jang, J-SR. (1993). ANFIS: Adaptive-Network-Based Fuzzy Inference System. IEEE transactions on systems, man, and cybernetics, vol. 23, no. 3, pp. 665-685.
[31] Zadeh, L. A. (1973) “Outline of a new approach to the analysis of complex systems and decision processes,” IEEE Trans. Syst., Man, Cybern., vol. 3, pp. 28-44.
[32]Wang, L.-X. (1992). Fuzzy Systems Are Universal Approximators, in Proc. IEEE Int. Conf Fuzzy Systems, San Diego, CA.
[33]Haldane, J. B. S. (1932). A Note on Inverse Probability, in Mathematical Proceedings of the Cambridge Philosophical Society, vol. 28, no. 1. Cambridge, U.K.: Cambridge Univ. Press, pp. 55–61.
[34]Bayes, T. and Price, R. (1763). An Essay Towards Solving a Problem in The Doctrine of Chances. By the late rev. Mr. Bayes, communicated by Mr. Price, in a letter to John Canton, M. A. and F. R. S, Philosoph. Trans. Roy. Soc. London, vol. 53, pp. 370–418.
[35]Kingma, D. P. and Ba, J. (2015). Adam: A Method for Stochastic Optimization in Proc. 3rd Int’l Conf. on Learning Representations, San Diego, CA.
[36]Duchi, J. & Hazan, E. and Singer, Y. (2011). Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, Journal of Machine Learning Research, vol. 12, pp. 2121–2159.
[37]Tieleman, T. and Hinton, G. (2012). Lecture 6.5–RMSProp: Divide the Gradient by A Running Average of Its Recent Magnitude, COURSERA: Neural Networks for Machine Learning, vol. 4, no. 2, pp. 26–31.
[38]Luo, L. & Xiong, Y. & Liu, Y. & and Sun, X. (2019). Adaptive Gradient Methods with Dynamic Bound of Learning Rate, arXiv preprint arXiv:1902.09843.
[39]Birkhioff, G. (1948). Lattice Theory, Am. Math. Soc. Colloq. Publ., vol. 25, New York.
[40]Nakasima-Lpez, S. & Castro, J. R. & Sanchez, M. A. & Mendoza, O. and Rodrguez-Daz, A. (2019). An Approach on the Implementation of Full Batch, Online and Mini-Batch Learning On a Mamdani Based Neuro-Fuzzy System with Center-of-Sets Defuzzification: Analysis and Evaluation About Its Functionality, Performance, and Behavior, PLOS ONE, vol. 14, no. 9, pp. 1–40.
[41]Wang, L. & Ye, J. (1998). Extracting fuzzy rules for system modeling using a hybrid of genetic algorithms and Kalman filter. Fuzzy Set Syst., vol. 101, pp. 353-362.
[42]Lughofer, E. and Kindermann, S. (2008). Improving The Robustness of Data-driven Fuzzy Systems with Regularization, in Proc. IEEE Int’l Conf. on Fuzzy Systems, Hong Kong, China, pp. 703–709.
[43]Johansen, T. A. (1996). Robust Identification of Takagi-Sugeno-Kang Fuzzy Models Using Regularization, in Proc. 5th IEEE Int’l Conf. on Fuzzy Systems, New Orleans, LA, pp. 180–193.
[44]Jin, Y. (2000). Fuzzy Modeling of High-Dimensional Systems: Complexity Reduction and Interpretability Improvement, IEEE Trans. on Fuzzy Systems, vol. 8, no. 2, pp. 212–221.
[45]Lughofer, E. and Kindermann, S. (2010). SparseFIS: Data-Driven Learning of Fuzzy Systems with Sparsity Constraints, IEEE Trans. on Fuzzy Systems, vol. 18, no. 2, pp. 396–411.
[46]Luo, M. & Sun, F. and Liu, H. (2015). Dynamic T-S Fuzzy Systems Identification Based on Sparse Regularization, Asian Journal of Control, vol. 17, no. 1, pp. 274–283.
[47]Wang, L.-X. and Mendel, J. M. (1992). Fuzzy Basis Function, Universal Approximation, And Orthogonal Least Squares Larning, IEEE Trans. Neural Networks, vol. 3 no. 5, pp. 807-814.
[48]Angelov, P. (1995). Crispification: Defuzzification Over Intuitionistic Fuzzy Sets, vol. 64, pp. 51–55.
[49]Akram MS, Habib S, Javed I (2014). Intuitionistic Fuzzy Logic Control for Washing Machines. Indian J. Sci. Technol., vol. 7, no. 5, pp. 654–661.
[50]Barrenechea, E (2009). Generalized Atanassov’s Intuitionistic Fuzzy Index. Construction Method. IFSA-EUSFLAT, Lisbon, pp. 478–482.