Optimization of GPU-Based Deployment Strategies for Large Language Models in E-Commerce Platforms

Abstract

In e-commerce platforms, millions of product–offer matchings are performed daily, which requires scalable and efficient solutions beyond traditional methods. This study aims to improve the deployment performance of Large Language Models (LLMs) in high-volume data matching processes. In this context, the Turkish BERT model was converted into the ONNX format, and the model size was reduced from 1.2 GB to 200–300 MB, thereby enhancing deployment efficiency. The performance of the model was comparatively evaluated across five different deployment infrastructures: NVIDIA Triton Inference Server, BentoML, Jina AI, LightningLite, and FastAPI (GPU). The results demonstrate that Triton Inference Server provides superior performance compared to other solutions, with its high throughput capacity and low latency. Furthermore, migrating from CPU to GPU achieved an approximately 70% improvement in response times and a reduction in operational costs. Future work will focus on automating model updates and orchestrating multiple models.

Keywords

• Large Language Models
• LLM Deployment
• Model Optimization
• GPU Performance Analysis

E-Ticaret Platformlarında Büyük Dil Modelleri için GPU Tabanlı Dağıtım Stratejilerinin Optimizasyonu

Abstract

E-ticaret platformlarında her gün milyonlarca ürün-teklif eşleştirmesi gerçekleştirilmekte ve bu durum geleneksel yöntemlerin ötesinde ölçeklenebilir ve verimli çözümler gerektirmektedir. Bu çalışma, yüksek hacimli veri eşleştirme süreçlerinde Büyük Dil Modellerinin (LLM) dağıtım performansını iyileştirmeyi amaçlamaktadır. Bu bağlamda, Türkçe BERT modeli ONNX formatına dönüştürülmüş ve model boyutu 1,2 GB'tan 200-300 MB'a düşürülerek dağıtım verimliliği artırılmıştır. Modelin performansı, NVIDIA Triton Inference Server, BentoML, Jina AI, LightningLite ve FastAPI (GPU) olmak üzere beş farklı dağıtım altyapısında karşılaştırmalı olarak değerlendirilmiştir. Sonuçlar, Triton Inference Server'ın yüksek işlem hacmi kapasitesi ve düşük gecikme süresiyle diğer çözümlere kıyasla üstün performans sağladığını göstermektedir. Ayrıca, CPU'dan GPU'ya geçiş, yanıt sürelerinde yaklaşık %70'lik bir iyileştirme ve operasyonel maliyetlerde bir düşüş sağlamıştır. Gelecekteki çalışmalar, model güncellemelerinin otomatikleştirilmesine ve birden fazla modelin düzenlenmesine odaklanılacaktır.

Keywords

• Büyük Dil Modelleri
• LLM Dağıtımı
• Model Optimizasyonu
• GPU Performans Analizi

References

1. [1] Z. Zhou et al., “A Survey on Efficient Inference for Large Language Models,” Jul. 2024, [Online]. Available: http://arxiv.org/abs/2404.14294
2. [2] S. Liu et al., “Optimizing LLM Queries in Relational Data Analytics Workloads,” Apr. 2025, [Online]. Available: http://arxiv.org/abs/2403.05821
3. [3] R. Ao, G. Luo, D. Simchi-Levi, and X. Wang, “Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints,” Apr. 2025, [Online]. Available: http://arxiv.org/abs/2504.11320
4. [4] N. Louloudakis and A. Rajan, “Selective Quantization Tuning for ONNX Models,” Jul. 2025, [Online]. Available: http://arxiv.org/abs/2507.12196
5. [5] M. Someki, Y. Higuchi, T. Hayashi, and S. Watanabe, “ESPnet-ONNX: Bridging a Gap Between Research and Production,” in 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, Nov. 2022, pp. 420–427. doi: 10.23919/APSIPAASC55919.2022.9979824.
6. [6] D. Ren et al., “ONNXPruner: ONNX-Based General Model Pruning Adapter,” Apr. 2024, [Online]. Available: http://arxiv.org/abs/2404.08016
7. [7] Z. Ye and R. Ying, “An AI-aware Orchestration Framework for Cloud-based LLM Workloads,” in 2024 IEEE 10th International Conference on Edge Computing and Scalable Cloud (EdgeCom), IEEE, Jun. 2024, pp. 22–24. doi: 10.1109/EdgeCom62867.2024.00011.
8. [8] P. P. Ray and M. P. Pradhan, “LLMEdge: A Novel Framework for Localized LLM Inferencing at Resource Constrained Edge,” in 2024 International Conference on IoT Based Control Networks and Intelligent Systems (ICICNIS), IEEE, Dec. 2024, pp. 1–8. doi: 10.1109/ICICNIS64247.2024.10823332.

9. [9] J. Li et al., “TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators,” Feb. 2025, [Online]. Available: http://arxiv.org/abs/2502.14752
10. [10] Y. He, M. Xu, J. Wu, W. Zheng, K. Ye, and C. Xu, “UELLM: A Unified and Efficient Approach for Large Language Model Inference Serving,” 2025, pp. 218–235. doi: 10.1007/978-981-96-0805-8_16.
11. [11] F. Aamir, R. Sherafgan, T. Arbab, A. Jamil, F. N. Bhatti, and A. A. Hameed, “Deep Learning-based Semantic Search Techniques for Enhancing Product Matching in E-commerce,” in 2024 IEEE 3rd International Conference on Computing and Machine Intelligence (ICMI), IEEE, Apr. 2024, pp. 1–9. doi: 10.1109/ICMI60790.2024.10586148.
12. [12] O. Cherednichenko, O. Ivashchenko, Ľ. Cibák, and M. Lincenyi, “Item Matching Model in E-Commerce: How Users Benefit,” Economics and Culture, vol. 20, no. 1, pp. 77–90, Jun. 2023, doi: 10.2478/jec-2023-0007.
13. [13] S. Li, F. Lv, R. Zhang, D. Ou, Z. Zhang, and M. de Rijke, “Text Matching Indexers in Taobao Search,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, New York, NY, USA: ACM, Aug. 2024, pp. 5339–5350. doi: 10.1145/3637528.3671654.
14. [14] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), Minneapolis, MN, USA: ACL, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
15. [15] S. Schweter, “BERTurk – BERT models for Turkish,” Apr. 2020. [Online]. Available: https://huggingface.co/dbmdz/bert-base-turkish-uncased. doi:10.5281/zenodo.3770924.