臺灣博碩士論文加值系統

隨著網路的發展，使用者們上網時間愈來愈長，拜訪網站的數量及種類也變得愈來愈多，然而對於網路使用者而言惡意網站仍是個極為嚴重的威脅。惡意網站在使用者不知情的情況下利用路過式下載技術(Drive-by-download)注入間諜軟體、病毒或是惡意程式到使用者的電腦之中，從事惡意的活動。例如:身分盜用、勒索敲詐、病毒或蠕蟲的散播…等等，網路安全已經成為非常重要的議題，也是值得深入研究的領域。
傳統的方法主要利用黑名單及機器學習來辨識惡意網站，然而這些方法經常使用了數百個甚至數千個特徵來進行計算，這些特徵中，有許多並非顯著特徵，這除了增加儲存資料及運算所消耗的資源，更有可能影響其評估的結果。本研究則利用特徵選取的方法挑選出較顯著的特徵，不僅降低特徵的數量來加速運算，更能提高惡意網站辨識的準確率。
本研究將提出一個惡意網站辨識套件，讓使用者在拜訪網站時，先行判斷該網站是否為惡意網站。研究分為二個步驟:(1)透過網路爬蟲的方式蒐集惡意網站並建立惡意網站黑名單與良好網站白名單。(2)針對惡意網站的相關特徵，進行特徵的選取及分類，以找出顯著的特徵並且辨識出潛在的惡意網站。本研究所提出的方法將透過網站的特徵來預判未知或潛在的惡意網站，並且在使用者拜訪該網站前先行阻擋，讓使用者免於受到傷害。

With the popularity of the networks, people spend more and more time surfing the Internet and visiting various websites. Although it is very convenient to obtain information from the websites, malicious websites are still a significant threat to the Internet users. Malicious websites implant malwares into users’ computers, without their knowledge, through the drive-by-downloads technology. And then the infected computers are ordered to do such illegal activities as identity theft, blackmail and extortion, virus or worm spreading, and so on. Hence, network security becomes an important issue, which is also an active research topic in the academia.
In order to distinguish between normal and malicious websites, common methods are establishing a blacklist through machine learning. However, such methods usually use hundreds or even thousands of URL features as training data in the machine learning process, which results in a large amount of computing resources. Furthermore, some of the URL features (called noise features) tend to decrease the accuracy of malicious website identification. They should be identified and excluded from the training data.
In this research, we propose methods to identify important features in order to improve the accuracy. First, we collect malicious and normal websites via web crawlers and then create a blacklist and whitelist, respectively. Second, the common URL features are processed one by one with the Lasso method to determine if they are significant. And finally, the significant features are used to evaluate if a website is normal or malicious through the SVM method. The techniques developed in this research can improve the quality of malicious website identification and therefore can protect users from be harmed by malicious websites.

目錄
中文摘要 I
Abstract III
致謝 IV
目錄 V
圖目錄 VI
表目錄 VII
第1章 緒論 1
1.1 研究動機與目的 1
1.2 論文架構 4
第2章 文獻探討 5
2.1 路過式下載攻擊 5
2.2 黑名單與機器學習 6
2.3 網路爬蟲 8
第3章 研究方法 10
3.1 爬蟲技術運用 11
3.2 網站判斷 12
3.2.1 資料蒐集 14
3.2.2 特徵提取 14
3.2.3 資料預處理 16
3.2.4 選擇顯著特徵 21
3.2.5 利用SVM進行訓練及分類 21
第4章 實驗結果分析 24
第5章 結論 29
參考文獻 30

[1]Aldwairi, M. & Alsalman, R., “MALURLS: A Lightweight Malicious Website Classification Based on URL Features.” Journal of Emerging Technologies in Web Intelligence (JETWI) Vol. 4, pp. 128-133 (2012)
[2]Anti-Phishing Working Group http://www.antiphishing.org (online)
[3]AVAST https://www.avast.com/ (online)
[4]Breiman, L., “Better Subset Regression Using the Nonnegative Garrote.” Technometrics No.37, pp. 373-384 (1995).
[5]Bright Cloud https://www.brightcloud.com/ (online)
[6]Chen, C. M., Huang, J. J., Ou, Y. H., “Efficient suspicious URL filtering based on reputation.” Journal of Information Security and Applications Vol. 20, pp. 26-36 (2015)
[7]Choi, H., Zhu, B. B., and Lee, H., “Detecting Malicious Web Links and Identifying Their Attack Types.” WebApps'11 Proceedings of the 2nd USENIX conference on Web application development, pp. 11-11, Berkeley, USA (2011)
[8]Cortes, C. & Vapnik V., “Support-vector networks.” Machine Learning No. 20, pp. 273-297 (1995).
[9]Darling, M., Heileman, G., Gressel, G., Ashok, A., and Poornachandran, P., “A Lexical Approach for Classifying Malicious URLs.” High Performance Computing & Simulation (HPCS), Alexandria,USA, pp. 195-202 (2015).
[10]DMOZ - The Directory of the Web. https://www.dmoz.org/ (online)
[11]Egele, M., Kirda, E., and Kruegel, C., “Mitigating driveby download attacks: Challenges and open problems.” In Proceedings of Open Research Problems in Network Security Workshop (iNetSec 2009), Zurich, Switzerland (2009).
[12]Fukushima, Y., Hori, Y., and Sakurai, K., “Proactive Blacklisting for Malicious Web Sites by Reputation Evaluation Based on Domain and IP Address Registration.” 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications, Changsha, China, pp. 352-361 (2011).
[13]Guyon, I., & Elisseeff, A., "An introduction to variable and feature selection." Journal of machine learning research", pp. 1157-1182.(2003)
[14]Kazemian, H. B. & Ahmed, S., “Comparisons of machine learning techniques for detecting malicious webpages.” Expert Systems with Applications, pp. 1166-1177 (2015).
[15]Ma, J., Saul, L. K., Savage, S., and Voelker, G. M., “Beyond Blacklists: Learning to Detect Malicious Web Sites from Suspicious URLs.” In KDD '09 Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, New York, USA, pp. 1245-1254 (2009).
[16]Malware - Clean MX.
http://support.clean-mx.de/clean-mx/viruses.php (online)
[17]Malware domain list
https://www.malwaredomainlist.com/mdl.php (online)
[18]MBA智库百科-網路爬蟲 http://wiki.mbalib.com/zh-tw/%E7%BD%91%E7%BB%9C%E7%88%AC%E8%99%AB (online)
[19]McAfee https://www.mcafee.com/ (online)
[20]McGrath, D. K. & Gupta, M., “Behind Phishing: An Examination of Phisher Modi Operandi.” In Proceedings of the 1st Usenix Workshop on Large-Scale Exploits and Emergent Threats (LEET), San Francisco, CA, (2008).
[21]PhishTank http://www.phishtank.com/phish_archive.php (online)
[22]Powers, D. M. W., “Evaluation: from precision, recall and f-measure to roc, informedness, markedness & correlation.” Journal of Machine Learning Technologies (2011).
[23]Provos, N., McNamee, D., Mavrommatis, P., Wang, K., and Modadug, N., "The ghost in the browser analysis of web-based malware.” In: Proceedings of the first workshop on hot topics in understanding botnets, Cambridge (2007).
[24]Salzberg, S., “On comparing classifiers: pitfalls to avoid and a recommended approach.” Data Mining and Knowledge Discovery, Boston, USA, pp. 317-328 (1997).
[25]Saranya, C., Manikandan, G. “A study on normalization techniques for privacy preserving data mining.” International Journal of Engineering and Technology (IJET), pp. 2701-2704 (2013).
[26]Security:
https://sucuri.net/website-security/google-blacklisted-my-website (online)
[27]Stehman, S. V., “Selecting and interpreting measures of thematic classification accuracy.” Remote sensing of Environment, pp. 77-89. (1997)
[28]Thomas, K., Grier, C., Ma, J., Paxson, V., and Song, D., “Design and Evaluation of a Real-Time URL Spam Filtering Service.” 2011 IEEE Symposium on Security and Privacy, California, USA, pp. 447-462 (2011).
[29]Tibshirani, R., “Regression Shrinkage and Selection via the lasso.” Journal of the Royal Statistical Society Vol.58, pp. 267-288 (1996).
[30]TrendLabs 2016年資訊安全總評 http://www.trendmicro.tw/cloud-content/tw/pdfs/security-intelligence/reports/trendlabs_2016_annual_information_security_review.pdf (online)
[31]Trend Web Reputation Policies
http://docs.trendmicro.com/all/ent/officescan/v10.6/en-us/osce_10.6_olhsrv/ohelp/web/wreppo.htm
[32]Web of Trust(WOT) https://www.mywot.com/ (online)
[33]Xu, H., Caramanis, C., and Mannor, S., “Robust Regression and Lasso.” IEEE Transactions on Information Theory, pp. 3561-3574 (2010).
[34]Zhou, Q., Song, S., Huang, G. and Wu, C., “Efficient Lasso training from a geometrical perspective.” Neurocomputing, Vol. 168, pp. 234-239 (2015).