TY  - JOUR
T1  - Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques
TT  - Microsoft SQL Server’da Büyük Veri Yönetiminin Optimizasyonu: Normalizasyon ve İleri Analitik Teknikler ile Performansının Artırılması
AU  - Bilgin, Turgay Tugay
AU  - Altinişik, Süleyman Burak
PY  - 2025
DA  - June
Y2  - 2025
DO  - 10.46460/ijiea.1563777
JF  - International Journal of Innovative Engineering Applications
JO  - ijiea, IJIEA
PB  - Niyazi ÖZDEMİR
WT  - DergiPark
SN  - 2587-1943
SP  - 23
EP  - 36
VL  - 9
IS  - 1
LA  - en
AB  - This study investigates Big Data management challenges and solutions in cable manufacturing using Microsoft SQL Server (MSSQL), focusing on performance optimization, normalization, and advanced analytical techniques. Addressing the 4Vs of Big Data, our case study collects data from 45 TAGs at one-minute intervals, generating approximately 56 million daily records. We employ OPC technology for data acquisition, strategic normalization processes, and advanced MSSQL optimization techniques. Normalization significantly reduced data redundancy, decreasing the dataset from 56 million to 283 rows per day and improving query execution times from over 40 minutes to less than 0.1 seconds for complex analytical queries. We also propose a database-independent software development approach to balance cost and performance. This research contributes practical insights into performance optimization, scalability, and cost-effective solutions for organizations managing large-scale data processing challenges in industrial settings, offering a blueprint for efficient Big Data management that balances technical performance with economic considerations.
KW  - Big Data Management
KW  - Data Normalization
KW  - Cable Manufacturing
KW  - Performance Optimization
N2  - Bu çalışma, kablo üretim sektöründe Microsoft SQL Server (MSSQL) kullanarak Büyük Veri yönetimi zorluklarını ve çözümlerini incelemekte, performans optimizasyonu, normalizasyon ve ileri düzey analitik tekniklere odaklanmaktadır. Büyük Veri'nin 4V'sini ele alan örnek olay incelemesinde, 45 TAG'dan bir dakikalık aralıklarla veri toplanmakta ve günlük yaklaşık 56 milyon kayıt oluşturulmaktadır. Veri toplamak için OPC teknolojisini, stratejik normalizasyon süreçlerini ve ileri düzey MSSQL optimizasyon tekniklerini kullanmaktayız. Normalizasyon, veri tekrarını önemli ölçüde azaltmış, veri setini günde 56 milyondan 283 satıra düşürmüş ve karmaşık analitik sorgular için sorgu yürütme sürelerini 40 dakikadan 0.1 saniyenin altına indirmiştir. Ayrıca, maliyet ve performans dengesini sağlamak için veritabanından bağımsız yazılım geliştirme yaklaşımı önermekteyiz. Bu araştırma, endüstriyel ortamlarda büyük ölçekli veri işleme zorluklarıyla karşılaşan organizasyonlar için performans optimizasyonu, ölçeklenebilirlik ve maliyet etkin çözümler konusunda pratik bilgiler sunmakta, teknik performans ile ekonomik hususlar arasında denge kuran etkili bir Büyük Veri yönetimi için bir yol haritası sunmaktadır.
CR  - Malik, P. K., Sharma, R., Singh, R., Gehlot, A., Satapathy, S. C., Alnumay, W. S., Pelusi, D., Ghosh, U., & Nayak, J. (2021). Industrial internet of things and its applications in industry 4.0: State of the art. Computer Communications, 166, 125–139.
CR  - Ghasemaghaei, M. (2021). Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics. International Journal of Information Management, 57, 102055
CR  - Fan, C., Yan, D., Xiao, F., Li, A., An, J., & Kang, X. (2021). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14(1), 3–24.
CR  - Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., De-la Hoz-Franco, E., & De-La-Hoz-Valdiris, E. (2022). Trends and future perspective challenges in big data. In Advances in Intelligent Data Analysis and Applications (pp. 309–325). Springer.
CR  - Ranjan, J., & Foropon, C. (2021). Big data analytics in building the competitive intelligence of organizations. International Journal of Information Management, 56, 102231.
CR  - Larrea, M. L., & Urribarri, D. K. (2021). Visualization technique for comparison of time-based large data sets. In Conference on Cloud Computing, Big Data & Emerging Topics (pp. 179–187). Springer.
CR  - Dinneen, J. D., & Brauner, C. (2017). Information-not-thing: Further problems with and alternatives to the belief that information is physical.
CR  - Vaitis, M., Feidas, H., Symeonidis, P., Kopsachilis, V., Dalaperas, D., Koukourouvli, N., Simos, D., & Taskaris, S. (2019). Development of a spatial database and web-GIS for the climate of Greece. Earth Science Informatics, 12(1), 97–115.
CR  - Amin, M., Romney, G. W., Dey, P., & Sinha, B. (2019). Teaching relational database normalization in an innovative way. Journal of Computing Sciences in Colleges, 35(2), 48–56.
CR  - Alqithami, S. (2021). A serious-gamification blueprint towards a normalized attention. Brain Informatics, 8(1), 1–13.
CR  - Oditis, I., Bicevska, Z., Bicevskis, J., & Karnitis, G. (2018). Implementation of NoSQL-based data warehouse. Baltic Journal of Modern Computing, 6(1), 45–55.
CR  - Hrubaru, I., Talabă, G., & Fotache, M. (2019). A basic testbed for JSON data processing in SQL data servers. In Proceedings of the 20th International Conference on Computer Systems and Technologies (pp. 278–283).
CR  - Chung, Y. G., Haldoupis, E., Bucior, B. J., Haranczyk, M., Lee, S., Zhang, H., Vogiatzis, K. D., Milisavljevic, M., Ling, S., Camp, J. S., et al. (2019). Advances, updates, and analytics for the computation-ready, experimental metal–organic framework database: Core MOF 2019. Journal of Chemical & Engineering Data, 64(12), 5985–5998.
CR  - Bouros, P., & Mamoulis, N. (2019). Spatial joins: What’s next? SIGSPATIAL Special, 11(1), 13–21.
CR  - Myalapalli, V. K., Totakura, T. P., & Geloth, S. (2015). Augmenting database performance via SQL tuning. In 2015 International Conference on Energy Systems and Applications (pp. 13–18). IEEE.
CR  - Pedrozo, W. G., & Vaz, M. S. M. G. (2014). A tool for automatic index selection in database management systems. In 2014 International Symposium on Computer, Consumer and Control (pp. 1061–1064). IEEE.
CR  - Correia, J., Santos, M. Y., Costa, C., & Andrade, C. (2018). Fast online analytical processing for big data warehousing. In 2018 International Conference on Intelligent Systems (IS) (pp. 435–442). IEEE.
CR  - Sulistiani, H., Setiawansyah, S., & Darwis, D. (2020). Penerapan metode agile untuk pengembangan online analytical processing (OLAP) pada data penjualan (Studi kasus: CV Adilia Lestari). Jurnal CoreIT: Jurnal Hasil Penelitian Ilmu Komputer dan Teknologi Informasi, 6(1), 50–56.
CR  - Erdinç, H. N., Buluş, N., & Erdoğan, C. (2018). Veritabanı tasarımının yazılım performansına etkisi: Normalizasyona karşı denormalizasyon. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 22(2), 887–895.
CR  - Alshemaimri, B., Elmasri, R., Alsahfi, T., & Almotairi, M. (2021). A survey of problematic database code fragments in software systems. Engineering Reports, 3(10), e12441.
CR  - Milicev, D. (2021). Hyper-relations: A model for denormalization of transactional relational databases. IEEE Transactions on Knowledge and Data Engineering.
CR  - Chaparro-Cruz, I. N., & Montoya-Zegarra, J. A. (2021). Borde: Boundary and sub-region denormalization for semantic brain image synthesis. In 2021 34th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (pp. 81–88). IEEE.
CR  - Costa, R. L. D. C., Moreira, J., Pintor, P., dos Santos, V., & Lifschitz, S. (2021). A survey on data-driven performance tuning for big data analytics platforms. Big Data Research, 25, 100206.
CR  - Chillón, A. H., Ruiz, D. S., & Molina, J. G. (2021). Towards a taxonomy of schema changes for NoSQL databases: The Orion language. In International Conference on Conceptual Modeling (pp. 176–185). Springer.
CR  - Gupta, E., Sural, S., Vaidya, J., & Atluri, V. (2021). Attribute-based access control for NoSQL databases. In Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy (pp. 317–319).
CR  - Santos, M. Y., Costa, C., Galvão, J., Andrade, C., Martinho, B. A., Lima, F. V., & Costa, E. (2017, July). Evaluating SQL-on-Hadoop for big data warehousing on not-so-good hardware. In Proceedings of the 21st International Database Engineering & Applications Symposium (pp. 242-252).
CR  - Yang, F., Tschetter, E., Léauté, X., Ray, N., Merlino, G., & Ganguli, D. (2014, June). Druid: A real-time analytical data store. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data (pp. 157-168).
CR  - Silva, Y. N., Almeida, I., & Queiroz, M. (2016, February). SQL: From traditional databases to big data. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (pp. 413-418).
CR  - Rahman, M. M., Islam, S., Kamruzzaman, M., & Joy, Z. H. (2024). Advanced Query Optimization in SQL Databases For Real-Time Big Data Analytics. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 1-14.
CR  - Uzzaman, A., Jim, M. M. I., Nishat, N., & Nahar, J. (2024). Optimizing SQL databases for big data workloads: techniques and best practices. Academic Journal on Business Administration, Innovation & Sustainability, 4(3), 15-29.
CR  - Pirzadeh, P., Carey, M., & Westmann, T. (2017, December). A performance study of big data analytics platforms. In 2017 IEEE international conference on big data (big data) (pp. 2911-2920). IEEE.
CR  - Panwar, V. (2024). Optimizing Big Data Processing in SQL Server through Advanced Utilization of Stored Procedures. Journal Homepage: http://www. ijmra. us, 14(02).
CR  - Ordonez, C. (2013, October). Can we analyze big data inside a DBMS?. In Proceedings of the sixteenth international workshop on Data warehousing and OLAP (pp. 85-92)
UR  - https://doi.org/10.46460/ijiea.1563777
L1  - https://dergipark.org.tr/tr/download/article-file/4273819
ER  -