使用非監督式機器學習之僵屍網路偵測｜國立清華大學博碩士論文庫

簡易檢索 / 詳目顯示

回結果列表

研究生：	安傑米 Jaime Alvarez
論文名稱：	使用非監督式機器學習之僵屍網路偵測 Botnet Detection Using Unsupervised Machine Learning
指導教授：	孫宏民 Sun, Hung-Min
口試委員:	葉志浩顏嵩銘洪國寶陳建銘
學位類別：	碩士 Master
系所名稱：	電機資訊學院 - 資訊系統與應用研究所 Institute of Information Systems and Applications
論文出版年：	2015
畢業學年度：	103
語文別：	英文
論文頁數：	39
中文關鍵詞：	殭屍網路、偵測、機器學習、演算法、相似度
外文關鍵詞：	Botnet, Detection, Machine Learning, Clustering, Similarity
相關次數：	點閱：3 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

This research focuses on botnet detection through implementation of techniques such as traffic analysis, unsupervised machine learning, and similarity analysis between benign traffic data and bot traffic data. In this study, we tested and experimented with different clustering algorithms and recorded their accuracy with our prepared datasets. Later, the best clustering algorithm was used to proceed with the next steps of the methodology such as determination of majority clusters (cluster with most flows), removal of duplicate flows, and calculation of similarity analysis. Results were recorded for the removal of duplicate flows stage, the results indicate how many flows each majority cluster contains and how many duplicate flows were removed from this majority cluster. Next, results for similarity analysis indicate the value of the similarity coefficient for the comparisons between all datasets (bot datasets and benign dataset.) With these results we can conclude and present some concluding heuristics for determining possible bot infection in a certain host.

本研究使用了不同的技術來偵測殭屍網路, 我們使用了網路流量分析, 非監督式學習, 以及分析正常網路與殭屍網路之間的相似性等技術來實踐。
研究中, 我們測試了不同的分群演算法並比較它們的表現, 下一步,我們選擇表現最好的分群演算法,去決定主群體還有移除多餘且相同的網路資料, 並分析其相似度。
藉由計算出的網路相似度結果, 我們設計出了啟發式的方法來偵測殭屍網路

Contents
Declaration of Authorship i
Abstract ii
Acknowledgements iii
List of Figures vi
List of Tables vii
Introduction 1
Background 2
1 Defining and Understanding a Botnet . . . . . . . . . . . . . . . . . . . . 2
1.1 Botnet Life-Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.1 Infection Phase . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2 Communication Phase . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Attack Phase . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Command and Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Centralized Botnets . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Distributed Botnets . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Hybrid Botnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Unsupervised Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Comparison Between Unsupervised and Supervised Learning . . . 7
Botnet Detection and Unsupervised Machine Learning 8
1 Traffic Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1 Simple K-Means Clustering . . . . . . . . . . . . . . . . . . . . . . 9
2.2 EM Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Clustering & Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4 Clustering Evaluation in Weka . . . . . . . . . . . . . . . . . . . . . . . . 11
Botnet Detection Datasets 12
1 Benign P2P, Zeus, and Waledac Botnet Traces . . . . . . . . . . . . . . . 12
iv
Contents v
1.1 Dataset Preparations . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 Storm Bot Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1 Storm Dataset 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Storm Dataset 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Methodology 15
1 Packet Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2 Flow Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1 Flow Features and Statistics . . . . . . . . . . . . . . . . . . . . . . 16
3 Data Pre-processing: Data Labeling . . . . . . . . . . . . . . . . . . . . . 17
4 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1 Information Gain Attribute Evaluation . . . . . . . . . . . . . . . . 19
5 Importing Data Into Weka . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6 Flow Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
6.1 Classes to Clusters Evaluation . . . . . . . . . . . . . . . . . . . . . 20
7 Determining Majority Clusters . . . . . . . . . . . . . . . . . . . . . . . . 22
8 Removal of Duplicate Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 22
9 Similarity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Experiments and Results 24
1 Feature Selection & Attribute Evaluation Results . . . . . . . . . . . . . . 24
2 Classes to Clusters Evaluation Results . . . . . . . . . . . . . . . . . . . . 26
2.1 Performance and Accuracy Results . . . . . . . . . . . . . . . . . . 27
3 Duplicate Flow Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4 Similarity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Conclusions 30
1 Concluding Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.1 Similarity Analysis and Unknown Botnets . . . . . . . . . . . . . . 30
1.2 Possible Improvements for Future Work . . . . . . . . . . . . . . . 31
Related Work 32
1 BotOnus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2 Botminer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3 Peershark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
A Complete List of Features 35
Bibliography
                                

[1] Matija Stevanovic and Myrup Pedersen. An efficient flow-based botnet detection
using supervised machine learning. Networking and Security Section, Department
of Electronic Systems, .
[2] Shang-Chiuan Su. Detecting p2p botnet in software defined network. Institute of
Network Engineering College of Computer Science National Chiao Tung University.
[3] Sharath Chandra Guntuku, Pratik Narang, and Chittaranjan Hota. Real-time peerto-
peer botnet detection framework based on bayesian regularized neural network.
Institute of Network Engineering College of Computer Science National Chiao Tung
University.
[4] Pratik Narang1, Chittaranjan Hota1, and VN Venkatakrishnan. Peershark: flowclustering
and conversation-generation for malicious peer-to-peer traffic identification.
EURASIP Journal on Information Security, 2014.
[5] Charles A. Fowler and Robert J. Hammell II. Converting pcaps into weka mineable
data. Department of Computer and Information Sciences Towson University.
[6] John Brozycki. Capturing and analyzing packets with perl. SANS Institute InfoSec
Reading Room.
[7] Daniel Arndt. How to: Calculating flow statistics using netmate. URL http:
//dan.arndt.ca/nims/calculating-flow-statistics-using-netmate/.
[8] Anthony McGregor, Mark Hall, Perry Lorier, and James Brunskill. Flow clustering
using machine learning techniques. The University of Waikato, New Zealand.
[9] Sherif Saad, Issa Traore, Ali A. Ghorbani, Bassam Sayed, David Zhao, Wei Lu, John
Felix, and Payman Hakimian. Detecting p2p botnets through network behavior
analysis and machine learning. Proceedings of 9th Annual Conference on Privacy,
Security and Trust (PST2011), 2011.
[10] Khalid Huseynov, Kwangjo Kim, and Paul D. Yoo. Semi-supervised botnet detection
using ant colony clustering. Cryptography and Information Security, 2014.
39
Bibliography 40
[11] David Zhao, Issa Traore, Bassam Sayed, and ... Botnet detection based on traffic
behavior analysis and flow intervals. Elsevier, 2013.
[12] Tao Cai and Futai Zou. Detecting http botnet with clustering network traffic.
[13] Chittaranjan Hota Pratik Narang, Jagan Mohan Reddy. Feature selection for detection
of peer-to-peer botnet traffic.
[14] Pijush Barthakur, Manoj Dahal, and Mrinal Kanti Ghose. Clusibothealer: Botnet
detection through similarity analysis of clusters. Journal of Advances in Computer
Networks, Vol. 3, No. 1, March 2015.
[15] Nicolás García-Pedrajas, Aida de Haro-García, and Javier Pérez-Rodríguez. A scalable
approach to simultaneous evolutionary instance and feature selection. Elsevier.
[16] Matija Stevanovic and Jens Myrup Pedersen. An efficient flow-based botnet detection
using supervised machine learning. Networking and Security Section, Depart-
ment of Electronic Systems, Aalborg University, .
[17] Paul Barford and Vinod Yegneswaran. An inside look at botnets. Computer Sciences
Department University of Wisconsin, Madison.
[18] Kamaldeep Singh, Sharath Chandra Guntuku, Abhishek Thakur, and Chittaranjan
Hota. Big data analytics framework for peer-to-peer botnet detection. Elsevier.
[19] Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee. Botminer: Clustering
analysis of network traffic for protocol- and structure-independent botnet detection.
College of Computing, Georgia Institute of Technology.
[20] . Botnet detection and collapse based on traffic analysis. , .
[21] David Zhao. Botnet detection based on traffic behavior analysis and flow intervals.
Elsevier.
[22] Mosa Yahyazadeh and Mahdi Abadi. Botonus: An online unsupervised method for
botnet detection. The ISC Int’l Journal of Information Security.
[23] Wei Lu, Goaletsa Rammidi, and Ali A. Ghorbani. Clustering botnet communication
traffic based on n-gram feature selection. Elsevier.
[24] Norbert Pohlmanna Christian J. Dietricha, Christian Rossowa. Cocospot: Clustering
and recognizing botnet command and control channels using traffic analysis.
Elsevier.
[25] Esraa Alomari and Selvakumar Manickama. Design, deployment and use of httpbased
botnet testbed. National Advanced IPv6 Centre (NAv6), Universiti Sains
Malaysia, Malaysia.
Bibliography 41
[26] Pratik Narang, Jagan Mohan Reddy, and Chittaranjan Hota. Feature selection
for detection of peer-to-peer botnet traffic. Department of Computer Science &
Engineering Birla Institute of Technology and Science-Pilani.
[27] Stevanovic Matija and Pedersen Jens Myrup. Machine learning for identifying botnet
network traffic. Department of Computer Science & Engineering Birla Institute of
Technology and Science-Pilani.
[28] Vaibhav Nivargi. Machine learning based botnet detection.
[29] Babak Rahbarinia and Roberto Perdisci. Peerrush: Mining for unwanted p2p traffic.
Dept. of Computer Science, University of Georgia.
[30] Khalid Huseynov and Kwangjo Kim. Unsupervised hadoop-based p2p botnet detection
with threshold setting. Department of Computer Science, Korea Advanced
Institute of Science and Technology.
[31] Zoubin Ghahramani. Unsupervised learning. Gatsby Computational Neuroscience
Unit University College London, UK.
[32] Carl Livadas. Using machine learning techniques to identify botnet traffic. Inter-
network Research Department BBN Technologies.
[33] Lasse Trolle Borup. Peer-to-peer botnets: A case study on waledac. Kongens Lyngby
2009.

全文公開日期本全文未授權公開 (校內網路)
全文公開日期本全文未授權公開 (校外網路)

簡易檢索 / 詳目顯示

相關論文