Performance Analysis of Compression Algorithms on Matrix Data: Data Transfer Optimization in Microservices Architectures

Yıl 2024, Sayı: 1, 49 - 60, 01.10.2024

Faruk Atasoy , Alper Akkaya , Nadir Kocakır , Önder Karademir

https://doi.org/10.46810/tdfd.1425959

Öz

With the rapid proliferation of microservices architectures these days, the efficient and fast transfer of large matrix data between services has become a significant challenge. This study presents an analysis aimed at finding solutions to this challenge. The analysis addresses the compression and decompression of large matrix data, focusing on lossless compression algorithms to optimize data transfer without data loss. The study is implemented on an example scenario. This scenario is taken from a project with a microservice architecture. In the example scenario, an image processing service developed in Python programming language generates 640x480 matrix data. After going through a compression algorithm, this data is periodically transferred to a backend service developed in C# programming language. This data is then stored in a database. In the final stage, decompression operations are performed so that this data can be used for reporting. The performance of various compression algorithms in the data compression, database storage and report generation stages is extensively tested. Within the scope of the study, tests were performed using five different compression algorithms (Gzip, Zlib, Deflate, Brotli and Bz2). The results are obtained through performance tests aimed at determining the most optimized end-to-end solution. Analyzing the performance of the compression algorithms on the example scenario, the Brotli algorithm gives the most optimal result in terms of both speed and compression size. This work makes an important contribution to data transfer optimization in microservice architectures and provides a reference for research in this area by presenting the performance analysis of various compression algorithms.

Anahtar Kelimeler

Data Compression Algorithm, Data Transfer, Gzip, Deflate, Deflate, Bz2

Kaynakça

• Enliçay M, Şahin Ö, Ülger İ, Balçiçek ÖE, Baydarman MV, Taşdemir Ş. Veri Sıkıştırma Algoritmalarının Karşılaştırılması: Katılım Bankası Örneği. Konya: Selçuk Üniversitesi; 2014.
• Öztürk E, Mesut A, Diri B. The performance analysis of data compression algorithms used in NoSQL databases. In: Proceedings of the International Conference on Computer Science and Engineering (UBMK 2016); 2016 Oct; Tekirdağ.
• Deorowicz S. Universal lossless data compression algorithms [dissertation]. Gliwice: Silesian University of Technology; 2003.
• Ramu V. Performance impact of microservices architecture. Rev Contemp Sci Acad Stud. 2023;3.
• Kodituwakku SR, Amarasinghe US. Comparison of lossless data compression algorithms for text data. Indian J Comput Sci Eng. 2010;1(4):416-25.
• Tapia F, et al. From monolithic systems to microservices: a comparative study of performance. Appl Sci. 2020;10(17):5797.
• Somashekar G. Performance management of large-scale microservices applications [dissertation]. Stony Brook (NY): Stony Brook University; 2023.
• Semunigus W, Balachandra P. Analysis for lossless data compression algorithms for low bandwidth networks. J Phys Conf Ser. 2021;1964(4).
• Alakuijala J, et al. Brotli: A general-purpose data compressor. ACM Trans Inf Syst. 2018;37(1):1-30.
• Bulut F. Huffman algoritmasıyla kayıpsız hızlı metin sıkıştırma. El-Cezeri. 2016;3(2).
• Gailly J-l, Adler M. gzip [Internet]. Available from: http://ftp.gnu.org/gnu/gzip/gzip-1.6.tar.gz .
• Zlib Compression Library [Internet]. Available from: http://www.zlib.net/ [cited 2024 Jan 10].
• Deutsch P. Deflate compressed data format specification version 1.3. RFC 1951 (Informational). IETF; 1996.