Extracting Cyber Threat Intelligence with Test Automation Tools from Open Sources

Öz

Cyber threat intelligence is a concept used for cyber threat-oriented, structured and analyzed information. In this study, we propose a model to transform raw data obtained from various open sources such as social media and cybersecurity sites through test automation tools into structured data and to obtain cybersecurity intelligence ready for data-driven threat analysis. The collected and unstructured raw data is converted into structured data by our model and standardized into forms that can feed cyber security softwares.

Anahtar Kelimeler

• Open-source intelligence
• cyber threat intelligence
• test automation
• data harvesting

Açık Kaynaklardan Test Otomasyon Araçlarıyla Siber Tehdit İstihbaratı Çıkarılması

Öz

Siber tehdit istihbaratı siber tehdit odaklı, yapılandırılmış ve analiz edilmiş bilgiler için kullanılan bir kavramdır. Bu çalışmada sosyal medya ve siber güvenlik siteleri gibi çeşitli açık kaynaklardan, test otomasyon araçları aracılığıyla elde edilen ham verilerin yapısal verilere dönüştürülmesini ve veriye dayalı tehdit analizine hazır, siber güvenlik istihbaratı elde edilmesini sağlayacak bir model önerilmiştir. Toplanan ve yapılandırılmamış olan ham veriler, modelimiz tarafından yapılandırılmış verilere dönüştürülmekte ve siber güvenlik yazılımlarını besleyebilecek standart formlara getirilmektedir.

Anahtar Kelimeler

• açık kaynak istihbaratı
• siber tehdit istihbaratı
• test otomasyon
• veri toplama

Kaynakça

1. F. Neri and P. Geraci, "Mining Textual Data to Boost Information Access in OSINT," in International Conference Information Visualisation, 2009.
2. P. Maciolek and G. Dobrowolski, "Cluo: Web-Scale Text Mining System For Open Source Intelligence Purposes," Computer, vol. 14, no. 1, pp. 45-62, 2013.
3. B. M. Thuraisingham, M. Kantarcıoğlu, K. W. Hamlen and L. Khan, "A Data Driven Approach for the Science of Cyber Security: Challenges and Directions," in International Conference on Information Reuse and Integration (IRI), 2016.
4. S. Yamada, K. Utsu and O. Uchida, "An Analysis of Tweets During the 2018 Osaka North Earthquake in Japan -A Brief Report," in International Conference on Information and Communication Technologies for Disaster Management (ICT-DM), 2018.
5. B. Shah, V. Agarwal, U. Dubey and S. Correia, "Twitter Analysis for Disaster Management," in International Conference on Computing Communication Control and Automation (ICCUBEA), 2018.
6. P. Garg, H. Garg and V. Ranga, "Sentiment analysis of the Uri terror attack using Twitter," in International Conference on Computing, Communication and Automation (ICCCA), 2017.
7. J. Wang and J. Q. Gan, "Prediction of the 2017 French election based on Twitter data analysis," in Computer Science and Electronic Engineering (CEEC), 2017.
8. D. S. A. Fernandes, M. G. C. Fernandes, G. A. Borges and F. A. A. M. N. Soares, "Decision-Making Simulator for Buying and Selling Stock Market Shares Based on Twitter Indicators and Technical Analysis," in International Conference on Systems, Man and Cybernetics (SMC), 2019.

9. N. Dionísio, F. Alves, P. M. Ferreira and A. N. Bessani, "Cyberthreat Detection from Twitter using Deep Neural Networks," in International Joint Conference on Neural Networks (IJCNN), 2019.
10. F. Alves, A. Bettini, P. Ferreira and A. N. Bessani, "Processing tweets for cybersecurity threat awareness," Information Systems, vol. 95, 2021.
11. C. Sabottke, O. Suciu and T. Dumitraş, "Vulnerability disclosure in the age of social media: exploiting twitter for predicting real-world exploits," in USENIX Conference on Security Symposium, 2015.
12. Q. L. Sceller, E. B. Karbab, M. Debbabi and F. Iqbal, "SONAR: Automatic Detection of Cyber Security Events over the Twitter Stream," in International Conference on Availability, Reliability and Security, 2017.
13. S. Lee, H. Cho, N. Kim, B. Kim and J. Park, "Managing Cyber Threat Intelligence in a Graph Database: Methods of Analyzing Intrusion Sets, Threat Actors, and Campaigns," in International Conference on Platform Technology and Service (PlatCon), 2018.
14. M. Kim, S. Lee, B. Cho, -I. Kim and M. Jun, "Design of a Cyber Threat Information Collection System for Cyber Attack Correlation," in International Conference on Platform Technology and Service (PlatCon), 2018.
15. Y. Ghazi, Z. Anwar, R. Mumtaz, S. Saleem and A. Tahir, "A Supervised Machine Learning Based Approach for Automatically Extracting High-Level Threat Intelligence from Unstructured Sources," in International Conference on Frontiers of Information Technology (FIT), 2018.
16. P. Zhang, J. Ya, T. Liu and J. Shi, "Mining Open-Source Cyber Threat Intelligence with Distant Supervision from the Web," in International Conference on Data Science in Cyberspace (DSC), 2021.
17. S. M. Arıkan and S. Acar, "A Data Mining Based System for Automating Creation of Cyber Threat Intelligence," in International Symposium on Digital Forensics and Security (ISDFS), 2021.
18. Y. Kawano and E. Nunohiro, "A Proposal of Distributed Autonomous Cooperative System about Exclusive Web Crawling for Cyber Security," in International Conference on Network-Based Information Systems (NBiS), 2016.
19. M. H. Mohd Pakhari, N. Jamil, M. E. Rusli and A. A. Abdul Rahim, "Implementation of Token Parsing Technique for Regex Based Classification of Unstructured Data for Cyber Threat Analysis," in International Conference on Information Technology and Multimedia (ICIMU), 2020.
20. P. Koloveas, T. Chantzios, C. Tryfonopoulos and S. Skiadopoulos, "A Crawler Architecture for Harvesting the Clear, Social, and Dark Web for IoT-Related Cyber-Threat Intelligence," in IEEE World Congress on Services (SERVICES), 2019.
21. R. Williams, S. Samtani, M. Patton and H. Chen, "Incremental Hacker Forum Exploit Collection and Classification for Proactive Cyber Threat Intelligence: An Exploratory Study," in IEEE International Conference on Intelligence and Security Informatics (ISI), 2018.
22. V. Mavroeidis, R. Hohimer, T. Casey and A. Jesang, "Threat Actor Type Inference and Characterization within Cyber Threat Intelligence," 2021 13th International Conference on Cyber Conflict (CyCon), 2021, pp. 327-352.