臺灣博碩士論文加值系統

一般來說，提到深度學習，最直觀的方法就是使用類神經網路，但是類神經網路有許多缺點，例如不容易理解，當遇到需要實作類神經網路時往往需要大量的相關研究，而且不容易追蹤其變數變化也是一個缺點，本篇論文企圖使用高階模糊派翠網路(HLFPN)來實現深度學習，並達成結構簡單與容易追蹤變數的目的。方法為先瞭解問題，列出問題的邏輯式，根據問題畫出HLFPN結構。這種針對問題性的結構可以使構造較為精簡，而精簡的構造也可以使深度學習的效能有所提升

In recent years, artificial intelligence (AI) is one of the hottest research topics in computer science. In general, when it comes to the needs to use deep learning, the most intuitive and unique implementation method is to use neural networks (NN), but there are many shortcomings in NN. First, it is not easy to understand. When encountering the needs for implementation, it often requires a lot of relevant research efforts to start the implementation. Second, the structure is complex. When building a perfect learning structure, in order to achieve the fully defined connection between the nodes, the overall structure becomes very complicated. It is not easy for developers to track the parameter changes inside. Therefore, the goal of this thesis is to provide a more streamlined method to perform deep learning. High level fuzzy Petri nets (HLFPN) are used to achieve deep learning, also called deep learning Petri nets (DLPN), in an attempt to propose a simple and easy structure, to track parameter changes, with speed faster than the deep learning neural network (DLNN). DLPN approach is to solve such a problem. First, write the predicate logic of the problem, then draw the structure according to predicate logic, use this drawn structure to perform targeted deep learning, because the design process makes the problem formulation have the advantages of the structure. DLPN makes the structure simpler, does not require redundant nodes to increase the complexity of the structure, and the simpler structure can also effectively improve the computing speed. The experimental results have shown that the DLPN performs better than the DLNN.

Acknowledgements I
Abstract (Chinese) II
Abstract (English) III
Table of Contents V
List of Figures VII
List of Tables VIII

Chapter 1 Introduction 1
1-1 Research Motivation and Purpose 1
1-2 Problem Statement 1
1-3 Organization of Thesis 3

Chapter 2 LITERATURE REVIEW 4
2-1 Deep Learning 4
2-2 NN Categories 4
2-2-1 Deep Neural Network 4
2-2-2 Recurrent Neural Network 5
2-2-3 Convolutional Neural Network 5
2-2-4 Deep Autoencoder 5
2-3 High Level Fuzzy Petri Net 6

Chapter 3 Unsupervised Deep Learning Algorithm 10

Chapter 4 Supervised Deep Learning Algorithm 18

Chapter 5 Main Results 27
5-1 Research Tools 27
5-2 Benchmarks 27
5-3 Experimental Results 36
5-3-1 Part 1 36
5-3-2 Part 2 37
5-4 Functional Comparison 38

Chapter 6 Conclusion and Future Work 39

References 40

Appendix - A 45
Publication 45

Appendix – B: DLPN C++ Partial Program Code 47
Fuzzy Relational Matrix Operation 47

Appendix – C: SONY’s Neural Network Cloud 50
SONY’s Neural Network Cloud Structure 50
SONY’s Neural Network Cloud Results Graph 50
Partial Program Code 51

[1] Victor R. L. Shen, Yue-Shan Chang, and Tony Tong-Ying Juang, “Supervised and unsupervised learning by using Petri nets, “IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, vol. 40, no. 2, pp. 363-375, Mar. 2010.
[2] Mufti Mahmud, Mohammed Shamim Kaiser, Amir Hussain, and Stefano Vassanelli, “Applications of deep learning and reinforcement learning to biological data ,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2063-2079, Jun. 2018.
[3] Dietmar Heinke and Fred H. Hamker, “Comparing neural networks: a benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP ,” IEEE Transactions on Neural Networks, vol. 9, no. 6, pp.1279-1291, Nov. 1998.
[4] Michael Margaliot and Gideon Langholz, “Fuzzy control of a benchmark problem: a computing with words approach,” IEEE Transactions on Fuzzy Systems, vol. 12, no. 2, pp.230-235, Apr. 2004.
[5] Ling Shao, Di Wu, and Xuelong Li, “Learning deep and wide: a spectral method for learning deep networks ,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 12, pp. 2303-2308, Dec. 2014.
[6] C. L. Philip Chen, Chun-Yang Zhang, Long Chen, and Min Gan, “Fuzzy restricted boltzmann machine for the enhancement of deep learning,” IEEE Transactions on Fuzzy Systems, vol. 23, no. 6, pp.2163-2173, Dec. 2015.
[7] Zuohua Ding, Yuan Zhou, and Mengchu Zhou, “Modeling self-adaptive software systems with learning Petri Nets
,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 46, no. 4, pp. 483-498, Apr. 2016.
[8] Chih-Hung Chang, “Deep and shallow architecture of multilayer neural networks ,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 10, pp. 2477-2486, Oct. 2015.
[9] S. Mehdi Vahidipour, Mohammad Reza Meybodi, and Mehdi Esnaashari , “Learning automata-based adaptive petri net and its application to priority assignment in queuing systems with unknown parameters, “IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 45, no. 10, pp. 1373-1384, Oct. 2015.
[10] Maoguo Gong, Jia Liu, Hao Li, Qing Cai, and Linzhi Su, “A multiobjective sparse feature learning model for deep neural networks,” IEEE Transactions On Neural Networks and Learning Systems, vol. 26, no. 12, pp. 3263-3277, Dec. 2015.
[11] Qingchen Zhang, Laurence T. Yang, and Zhikui Chen, “Deep computation model for unsupervised feature learning on big data,” IEEE Transactions on Services Computing, vol. 9, no. 1, pp. 161-171, Jan./Feb. 2016.

[12] Tong Zhang, Wenming Zheng, Zhen Cui, Yuan Zong, Jingwei Yan, and Keyu Yan, “A deep neural network-driven feature learning method for multi-view facial expression recognition,” IEEE Transactions on Multimedia, vol. 18, no. 12, pp. 2528-2536, Dec. 2016.
[13] Mark Pfeiffer , Samarth Shukla , Matteo Turchetta , Cesar Cadena , Andreas Krause, Roland Siegwart , and Juan Nieto, “Reinforced imitation: sample efficient deep reinforcement learning for mapless navigation by leveraging prior demonstrations ,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 4423-4430, Oct. 2018.
[14] V. R. L. Shen, “Knowledge representation using high-level fuzzy Petri nets,” IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 36, no. 6, pp. 2120–2127, Nov. 2006.
[15] Jia-Lin Chen and Jyh-Yeong Chang, “Fuzzy perceptron neural networks for classifiers with numerical data and linguistic rules as inputs,” IEEE Transactions on Fuzzy Systems, vol. 8, no. 6, pp. 730-745, Dec. 2000.
[16] Ji-Cheng Duan and Fu-Lai Chung,“Cascaded fuzzy neural network model based on syllogistic fuzzy reasoning, ” IEEE Transactions on Fuzzy Systems, vol. 9, no. 2, pp. 293-306, Apr. 2001.
[17] Chia-Feng Juang, and Ksuan-Chun Ku, “A recurrent fuzzy network for fuzzy temporal sequence processing and gesture recognition,” IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 35, no. 4, pp. 646-658, Aug. 2005.
[18] Hao Ying, “Deriving analytical input–output relationship for fuzzy controllers using arbitrary input fuzzy sets and Zadeh fuzzy and operator,” IEEE Transactions on Fuzzy Systems, vol. 14, no. 5, pp. 654-662, Oct. 2006.
[19] Haibo Zhou, and Hao Ying, Fellow, IEEE, “A method for deriving the analytical structure of a broad class of typical interval type-2 mamdani fuzzy controllers,” IEEE Transactions on Fuzzy Systems, vol. 21, no. 3, pp.447-458, Jun. 2013.
[20] Weilin Deng and Daowen Qiu, “Supervisory control of fuzzy discrete-event systems for simulation equivalence,” IEEE Transactions on Fuzzy Systems, vol. 23, no. 1, pp.178-192, Feb. 2015.
[21] Yuanpeng Zhang, Hisao Ishibuchi, and Shitong Wang, “Deep takagi–sugeno–kang fuzzy classifier with shared linguistic fuzzy rules,” IEEE Transactions on Fuzzy Systems, vol. 26, no. 3, pp.1535-1549, Jun. 2018.
[22] Hani Hagras, “Comments on dynamical optimal training for interval type-2 fuzzy neural network (T2FNN),” IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics, vol. 36, no. 5, pp. 1206-1209, Oct. 2006.
[23] Chia-Feng Juang, Shih-Hsuan Chiu, and Shu-Wew Chang, “A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems, ” IEEE Transactions On Fuzzy Systems, vol. 15, no. 5, pp. 998-1008, Oct. 2007.
[24] Chia-Feng Juang, and Yu-Wei Tsao, “A self-evolving interval type-2 fuzzy neural network with online structure and parameter learning,” IEEE Transactions On Fuzzy Systems, vol. 16, no. 6, pp. 1411-1424, Dec. 2008.
[25] Rong-Jong Wai, Zhi-Wei Yang, “Adaptive fuzzy neural network control design via a t–s fuzzy model for a robot manipulator including actuator dynamics, ” IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics, vol. 38, no. 5, pp. 1326-1346, Oct. 2008.
[26] Chia-Feng Juang, Ren-Bo Huang, and Yang-Yin Lin, “A recurrent self-evolving interval type-2 fuzzy neural network for dynamic system processing,” IEEE Transactions On Fuzzy Systems, vol. 17, no. 5, pp. 1092-1105, Oct. 2009.
[27] Sung-Suk Kim and Keun-Chang Kwak, “Development of quantum-based adaptive neuro-fuzzy networks,” IEEE Transactions On Systems, Man, And Cybernetics—Part B: Cybernetics, vol. 40, no. 1, pp. 91-100, Feb. 2010.
[28] Mohammad Mehdi Ebadzadeh, Armin Salimi-Badr, “IC-FNN: A Novel Fuzzy Neural Network With Interpretable, Intuitive, and Correlated-Contours Fuzzy Rules for Function Approximation,” IEEE Transactions On Fuzzy Systems, vol. 26, no. 3, pp. 1288-1302, Jun. 2018.