Abstract
Gelişen teknoloji, iletişim ve ulaşımdaki
kolaylıklar sınırların yavaş yavaş ortadan kalkmasına yol açmıştır. Bu sebeple
farklı coğrafyalarda gerçekleşen olaylar, sınırların ötesine geçerek etkisini
büyük kitlelere hissettirmektedir. Sınırları denetlemek, korunmak, kaçak
geçişlere karşı hızlı ve kapsamlı çözümler bulmak birçok ülkenin ana problemi
haline gelmiştir. Sınır güvenliğinin kapsamı düşünüldüğünde; farklı zamanlarda,
farklı kaynaklardan ve farklı tiplerde gelen verinin, gerçek zamanlı olarak
incelenmesi gerektiği anlaşılacaktır. Bu da farklı prensiplerin bir arada
çalışması ile mümkündür. Bunların yanı sıra yapılandırılmamış büyük ölçekli
veri analizi de göz önüne alındığında, kapsamlı bir çözüm için Büyük Veri (Big
Data) teknolojilerinden yararlanarak boru hattı (pipeline) geliştirmek akılcı
bir yöntem olacaktır. Bu çalışmada, sınır güvenliği için gerçek zamanlı akan
veri ve yığın veri üzerinde analiz yapmayı sağlayan Lambda Mimarisi (Lambda
Architecture) yetenekleri
araştırılmaktadır. Sınır güvenliği
problemi çözümüne yönelik geliştirilen uygulama açıklanarak sistemin çalışması
hakkında bilgi verilmektedir.
Keywords
Apache Spark Lambda Architecture Apache Kafka Cassandra Gerçek zamanlı veri işleme Yığın Veri işleme
References
- Aksu, M. and Turhan, F. “New Threats, Expansion of Security Dimensions and Human Security”, International Journal of Alanya Faculty of Business, Vol. 4, 69-80, 2012
- Eker, G. and Yılmaz, G. “Providing Environmental Security Using Wireless Sensor Networks”, TBV BBMD, 64-71, 2013
- Demchenko, Y., de Laat, C. and Membrey, P, “Defining architecture components of the Big Data Ecosystem”, Collaboration Technologies and Systems (CTS), 2014, 104-112
- Engel, Y., and Opher, E., “Towards proactive event-driven computing”, 5th ACM international conference on Distributed event-based system (DEBS '11), 2014, 125-136
- Kreps, J., Narkhede, N. and Rao, J., “Kafka: A distributed messaging system for log processing”, Proceedings of 6th International Workshop on Networking Meets Databases, 2011
- Zaharia, M., Das, T., Li, H., Shenker, S. and Stoica, I., “Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters”, 4th USENIX conference on Hot Topics in Cloud Computing, 2012
- Chebotko Kashlev, A. and Shiyong, L., “A Big Data Modeling Methodology for Apache Cassandra”, Big Data (BigData Congress), IEEE International Congress, 2015, 238-245
- Nodejs, https://nodejs.org/en/about/
- Cesiumjs, https://cesiumjs.org/
- The Reactive Manifesto, http://www.reactivemanifesto.org/
- Nathan Marz and James Warren, Principles and best practices of scalable realtime data systems, Manning, 2015, 328 pages
- Spark Streaming + Kafka Integration, http://spark.apache.org/docs/latest/streaming-kafka-integration.html
- Kiran, M., Murphy, P., Monga, I., Dugan, J. and Baveja, S., “Lambda architecture for cost-effective batch and speed big data processing.” Big Data (Big Data Congress), IEEE International Conference, 2015, 2785-2792
Abstract
Developing technology, communication and transportation facilities have led borders to the gradual disappearance. The incidents taking place in different countries go beyond the limits and being felt by large audiences. Solutions to rapidly increasing terrorist attacks and internal turmoil caused security problems in recent years which have become very complicated. Protecting and controlling the borders, finding rapid and comprehensive solutions to illegal border passing problems have become a major problem in many countries. When considering the scope of the border security, it would be understood to examine the data which comes at different times from different sources and types in real time. This can be achieved by a collaboration of different principles. In addition, when large-scale unstructured data analysis also considered for a comprehensive solution, it would be a rational method to develop a pipeline using the Big Data technologies. In this study, we aim to achieve cost-effective, robust, scalable and flexible system to solve the border security problems. The use of Lambda Architecture which provides real time data processing and batch processing capabilities was investigated in border security applications. System development was explained and all information on its principles was provided.
Keywords
Apache Spark Lambda Architecture Apache Kafka Cassandra Real-time Data Processing Batch Data Processing
References
- Aksu, M. and Turhan, F. “New Threats, Expansion of Security Dimensions and Human Security”, International Journal of Alanya Faculty of Business, Vol. 4, 69-80, 2012
- Eker, G. and Yılmaz, G. “Providing Environmental Security Using Wireless Sensor Networks”, TBV BBMD, 64-71, 2013
- Demchenko, Y., de Laat, C. and Membrey, P, “Defining architecture components of the Big Data Ecosystem”, Collaboration Technologies and Systems (CTS), 2014, 104-112
- Engel, Y., and Opher, E., “Towards proactive event-driven computing”, 5th ACM international conference on Distributed event-based system (DEBS '11), 2014, 125-136
- Kreps, J., Narkhede, N. and Rao, J., “Kafka: A distributed messaging system for log processing”, Proceedings of 6th International Workshop on Networking Meets Databases, 2011
- Zaharia, M., Das, T., Li, H., Shenker, S. and Stoica, I., “Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters”, 4th USENIX conference on Hot Topics in Cloud Computing, 2012
- Chebotko Kashlev, A. and Shiyong, L., “A Big Data Modeling Methodology for Apache Cassandra”, Big Data (BigData Congress), IEEE International Congress, 2015, 238-245
- Nodejs, https://nodejs.org/en/about/
- Cesiumjs, https://cesiumjs.org/
- The Reactive Manifesto, http://www.reactivemanifesto.org/
- Nathan Marz and James Warren, Principles and best practices of scalable realtime data systems, Manning, 2015, 328 pages
- Spark Streaming + Kafka Integration, http://spark.apache.org/docs/latest/streaming-kafka-integration.html
- Kiran, M., Murphy, P., Monga, I., Dugan, J. and Baveja, S., “Lambda architecture for cost-effective batch and speed big data processing.” Big Data (Big Data Congress), IEEE International Conference, 2015, 2785-2792
Details
| Subjects | Engineering |
|---|---|
| Journal Section | Journals |
| Authors | |
| Publication Date | June 9, 2016 |
| Published in Issue | Year 2016 |
Cite
Cited By
Migration of a Vehicle Tracking System Running on Relational Database to Big Data Environment
Turkish Journal of Science and Technology
https://doi.org/10.55525/tjst.1364046
