Optimizing Big Data Management on Microsoft SQL Server: Enhancing Performance through Normalization and Advanced Analytical Techniques
Öz
This study investigates Big Data management challenges and solutions in cable manufacturing using Microsoft SQL Server (MSSQL), focusing on performance optimization, normalization, and advanced analytical techniques. Addressing the 4Vs of Big Data, our case study collects data from 45 TAGs at one-minute intervals, generating approximately 56 million daily records. We employ OPC technology for data acquisition, strategic normalization processes, and advanced MSSQL optimization techniques. Normalization significantly reduced data redundancy, decreasing the dataset from 56 million to 283 rows per day and improving query execution times from over 40 minutes to less than 0.1 seconds for complex analytical queries. We also propose a database-independent software development approach to balance cost and performance. This research contributes practical insights into performance optimization, scalability, and cost-effective solutions for organizations managing large-scale data processing challenges in industrial settings, offering a blueprint for efficient Big Data management that balances technical performance with economic considerations.
Anahtar Kelimeler
Big Data Management, Data Normalization, Cable Manufacturing, Performance Optimization
Kaynakça
- Malik, P. K., Sharma, R., Singh, R., Gehlot, A., Satapathy, S. C., Alnumay, W. S., Pelusi, D., Ghosh, U., & Nayak, J. (2021). Industrial internet of things and its applications in industry 4.0: State of the art. Computer Communications, 166, 125–139.
- Ghasemaghaei, M. (2021). Understanding the impact of big data on firm performance: The necessity of conceptually differentiating among big data characteristics. International Journal of Information Management, 57, 102055
- Fan, C., Yan, D., Xiao, F., Li, A., An, J., & Kang, X. (2021). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14(1), 3–24.
- Naeem, M., Jamal, T., Diaz-Martinez, J., Butt, S. A., Montesano, N., Tariq, M. I., De-la Hoz-Franco, E., & De-La-Hoz-Valdiris, E. (2022). Trends and future perspective challenges in big data. In Advances in Intelligent Data Analysis and Applications (pp. 309–325). Springer.
- Ranjan, J., & Foropon, C. (2021). Big data analytics in building the competitive intelligence of organizations. International Journal of Information Management, 56, 102231.
- Larrea, M. L., & Urribarri, D. K. (2021). Visualization technique for comparison of time-based large data sets. In Conference on Cloud Computing, Big Data & Emerging Topics (pp. 179–187). Springer.
- Dinneen, J. D., & Brauner, C. (2017). Information-not-thing: Further problems with and alternatives to the belief that information is physical.
- Vaitis, M., Feidas, H., Symeonidis, P., Kopsachilis, V., Dalaperas, D., Koukourouvli, N., Simos, D., & Taskaris, S. (2019). Development of a spatial database and web-GIS for the climate of Greece. Earth Science Informatics, 12(1), 97–115.
- Amin, M., Romney, G. W., Dey, P., & Sinha, B. (2019). Teaching relational database normalization in an innovative way. Journal of Computing Sciences in Colleges, 35(2), 48–56.
- Alqithami, S. (2021). A serious-gamification blueprint towards a normalized attention. Brain Informatics, 8(1), 1–13.