Data-Driven Insights for Enhancing Retail Sustainability: An Empirical Dimension Reduction Apriori Algorithm for Association Rule Mining

Abstract

In the era of digital transformation and big data, organizations generate vast volumes of raw data from diverse channels such as IoT devices, cyber systems, and e-commerce platforms. Extracting meaningful insights from this data is essential, particularly in identifying interesting item sets through association rule mining. These patterns reveal strong product relationships and can drive more sustainable strategies for marketing and sales in retail and online platforms. However, when working with large-scale datasets containing thousands of transactions and items, many frequently occurring items do not necessarily result in meaningful or interesting item sets. This leads to inefficiencies in the mining process and unnecessary computational overhead. To address this issue, we propose a novel algorithm called Dimension Reduction Apriori (DR-Apriori), which enhances the performance of traditional association rule mining by incorporating a dimension reduction step. This approach streamlines the dataset, enabling the extraction of more relevant and interesting item sets. We evaluated DR-Apriori on two benchmark datasets—15,000 transactions with 4,089 features, and 7,500 transactions with 121 features—under varying thresholds for dimension reduction, support, and confidence. Experimental results show that DR-Apriori outperforms traditional Apriori and Hybrid-Apriori algorithms, achieving up to 51% faster runtime, reducing memory usage by up to 31%, and maintaining the number of interesting item sets discovered. This study highlights the potential of DR-Apriori in enhancing the efficiency and scalability of association rule mining, ultimately supporting more intelligent, data-driven decisions for sustainable retail practices.

Keywords

• Association rule mining
• Big data analytics
• Apriori algorithm
• Market basket analysis
• Dimension reduction techniques

References

1. [1] Alasow, A. and Perkowski, M., “Quantum algorithm for mining frequent patterns for association rule mining”, Journal of Quantum Information Science, 13(1): 1–23, (2023). DOI: https://doi.org/10.4236/jqis.2023.131001
2. [2] Alawadh, M. and Barnawi, A., “A consumer behavior analysis framework toward improving mar-ket performance indicators: Saudi’s retail sector as a case study”, Journal of Theoretical and Ap-plied Electronic Commerce Research, 19(1): 152–171, (2024). DOI: https://doi.org/10.3390/jtaer19010009
3. [3] Baroto, W. A., “Advancing digital forensic through machine learning: an integrated framework for fraud investigation”, Asia Pacific Fraud Journal, 9(1): 1–16, (2024). DOI: https://doi.org/10.21532/apfjournal.v9i1.346
4. [4] Chowdhury, J. H., Hossain, M. B., Kaiser, M. S. and Arefin, M. S., “Performance comparisons in association rule mining over public datasets”, Lecture Notes on Data Engineering and Communica-tions Technologies, 95(1): 761–775, (2022). DOI: https://doi.org/10.1007/978-981-16-6636-0_57
5. [5] Datta, S. and Mali, K., “Significant association rule mining with mms and efficient correlation framework”, Lecture Notes in Networks and Systems, 288(1): 755–769, (2022). DOI: https://doi.org/10.1007/978-981-16-5120-5_57
6. [6] Datu, N. H., “Road traffic accidents analysis using association rule mining and descriptive analyt-ics”, AIP Conference Proceedings, 2508(1): 1–10, (2023). DOI: https://doi.org/10.1063/5.0117371
7. [7] Fang, F., “A study on the application of data mining techniques in the management of sustainable education for employment”, Data Science Journal, 22(1): 1–23, (2023). DOI: https://doi.org/10.5334/dsj-2023-023
8. [8] Genga, L., Allodi and L., Zannone, N., “Association rule mining meets regression analysis: an automated approach to unveil systematic biases in decision-making processes”, Journal of Cyber-security and Privacy, 2(1): 191–219, (2022). DOI: https://doi.org/10.3390/jcp2010011

9. [9] Guan, Y., “An algorithm for data management of higher education based on fuzzy set theory – association rule mining algorithm”, Informatica, 47(9): 157–164, (2023). DOI: https://doi.org/10.31449/inf.v47i9.5222
10. [10] Çaliskan, D., Kazım, Y., Buket, D. and Abdulsamet, A., “Crime data analysis with association rule mining”, International Periodical of Recent Technologies in Applied Engineering, 2(1): 42–50, (2021). DOI: https://doi.org/10.29228/porta.1
11. [11] Hamdad, L. and Benatchba, K., “Association rules mining”, SN Computer Science, 2(6): 449–476, (2021). DOI: https://doi.org/10.1007/s42979-021-00819-x
12. [12] Jia, W., Sun, M., Lian, J. and Hou, S., “Feature dimensionality reduction: a review”, Complex and Intelligent Systems, 8(1): 2663–2693, (2022). DOI: https://doi.org/10.1007/s40747-021-00637-x
13. [13] Kishor, P. and Porika, S., “A novel association rule mining model for generating positive and neg-ative association rules with hybridized meta-heuristic development”, International Journal of Intel-ligent Engineering and Systems, 16(2): 125–141, (2023). DOI: https://doi.org/10.22266/ijies2023.0430.11
14. [14] Krishnan, M. S., Nair, A. S. and Sebastian, J., “Comparative analysis of apriori and eclat algo-rithm for frequent itemset data mining”, Smart Innovation, Systems and Technologies, 243(1): 489–497, (2022). DOI: https://doi.org/10.1007/978-981-16-3675-2_37
15. [15] Kruse, R., Lokukatagoda, T. and Alkhushayni, S., “A framework for association rule learning with social media networks”, IOP SciNotes, 3(1): 015001–015007, (2022). DOI https://doi.org/10.1088/2633-1357/abe9be
16. [16] Liu, X., Niu, X. and Fournier-Viger, P., “Fast top-k association rule mining using rule generation property pruning”, Applied Intelligence, 51(4): 2077–2093, (2021). DOI https://doi.org/10.1007/s10489-020-01994-9
17. [17] Liu, Y., Wang, L., Miao, R. and Ren, H., “A data mining algorithm for association rules with chronic disease constraints”, Computational Intelligence and Neuroscience, 1: 1–8, (2022). DOI: https://doi.org/10.1155/2022/8526256
18. [18] Luna, J. M. and Ventura, S., “Frequent itemset mining: a 25 years review”, WIREs Data Mining and Knowledge Discovery, 9(5): 1–15, (2019). DOI: https://doi.org/10.1002/widm.1329m
19. [19] Moghe, B., Pamnani, H. K. and Verma, N. K., “A comparative review of various data mining algorithms for customer behaviour identification using market basket analysis”, Journal of Data Mining and Management, 6(1): 9–16, (2021). DOI: https://doi.org/10.46610/jodmm.2021.v06i01.003
20. [20] Mudumba, B. and Kabir, M. F., “Mine-first association rule mining: an integration of independent frequent patterns in distributed environments”, Decision Analytics Journal, 10(1): 1–8, (2024). DOI: https://doi.org/10.1016/j.dajour.2024.100434
21. [21] Pamnani, H. K., Raja, L. and Ives, T., “Developing a novel h-apriori algorithm using support-leverage matrix for association rule mining”, International Journal of Information Technology, 16(1): 5395–5405, (2024). DOI: https://doi.org/10.1007/s41870-024-02214-0
22. [22] Pamnani, H. K., Raja, L. and Ives, T., “An impact of feature selection in association rule mining for market basket analysis”, ICAICR 2023, 130–140, (2025). DOI: https://doi.org/10.1007/978-3-031-84059-3_11
23. [23] Raad, H. and Hamad, M. M., “A hybrid integrated model for big data applications based on asso-ciation rules and fuzzy logic: a review”, Iraqi Journal for Computer Science and Mathematics, 4(2): 171–178, (2023). DOI: https://doi.org/10.52866/ijcsm.2023.02.02.015
24. [24] Rahayu, S. A., “Market basket analysis using fp-growth algorithm to design marketing strategy by determining consumer purchasing patterns”, Journal of Applied Data Sciences, 4(1): 38–49, (2023). DOI: https://doi.org/10.47738/jads.v4i1.83
25. [25] Ray, P., Reddy, S. S. and Banerjee, T., “Various dimension reduction techniques for high dimen-sional data analysis: a review”, Artificial Intelligence Review, 54(1): 3473–3515, (2021). DOI: https://doi.org/10.1007/s10462-020-09928-0
26. [26] Shrestha, A., Zikos, D. and Fegaras, L., “An annotated association mining approach for extracting and visualizing interesting clinical events”, International Journal of Medical Informatics, 148(1): 104366–104378, (2021). DOI: https://doi.org/10.1016/J.IJMEDINF.2020.104366
27. [27] Siswanto, B., Soeparno, H., Sianipar, N. F. and Budiharto, W., “SDFP-growth algorithm as a nov-elty of association rule mining optimization”, IEEE Access, 12(1): 21491–21502, (2024). DOI: https://doi.org/10.1109/ACCESS.2024.3361667
28. [28] Sokhangoee, Z. F. and Rezapour, A., “A novel approach for spam detection based on association rule mining and genetic algorithm”, Computers and Electrical Engineering, 97(1): 107655, (2022). DOI: https://doi.org/10.1016/j.compeleceng.2021.107655
29. [29] Srivastava, T., Mullick, I. and Bedi, J., “Association mining based deep learning approach for financial time-series forecasting”, Applied Soft Computing, 155(1): 111469, (2024). DOI: https://doi.org/10.1016/J.ASOC.2024.111469
30. [30] Sun, X., Ngueilbaye, A., Luo, K., Cai, Y., Wu, D. and Huang, J. Z., “A scalable and flexible bas-ket analysis system for big transaction data in spark”, Information Processing and Management, 61(2): 1–22, (2024). DOI: https://doi.org/10.1016/j.ipm.2023.103577
31. [31] Verma, N. K. and Singh, V., “A pragmatic approach to recover access time of apriori algorithm by applying intersection on css for redefining fis through matrix implementation in textual data”, Journal of Discrete Mathematical Sciences and Cryptography, 25(4): 871–878, (2022). DOI: https://doi.org/10.1080/09720529.2022.2068601
32. [32] Wang, T., Xiao, B. and Ma, W., “Student behavior data analysis based on association rule min-ing”, International Journal of Computational Intelligence Systems, 15(1): (2022). DOI: https://doi.org/10.1007/s44196-022-00087-4
33. [33] Wu, X., Wen, Q. and Zhu, J., “Association rule mining with a special rule coding and dynamic genetic algorithm for air quality impact factors in Beijing, China”, PLoS ONE, 19(3): 1–23, (2024). DOI: https://doi.org/10.1371/journal.pone.0299865
34. [34] Yang, W., “Research on weighted association rules mining algorithm for big data”, IEEE Interna-tional Conference on Electrical, Automation and Computer Engineering, 1644–1652, (2023). DOI: https://doi.org/10.1109/ICEACE60673.2023.10441983
35. [35] Yannam, V. R., Kumar, J., Vankayala, T. and Babu, K. S., “Hybrid approach for next basket rec-ommendation system”, International Journal of Information Technology, 15(3): 1733–1740, (2023). DOI: https://doi.org/10.1007/s41870-023-01220-y
36. [36] Ye, Z., Cai, W., Wang, M., Zhang, A., Zhou, W., Deng, N., Wei, Z. and Zhu, D., “Association rule mining based on hybrid whale optimization algorithm”, International Journal of Data Ware-housing and Mining, 18(1): 1–22, (2023). DOI: https://doi.org/10.4018/IJDWM.308817
37. [37] Yildiz, K., Camurcu, Y. and Dogan, B., “Comparison of dimension reduction techniques on high dimensional datasets”, The International Arab Journal of Information Technology, 15(2): 256–262, (2018).
38. [38] Yoset, A., Roth, I., Shanider, E., Baranes, A. and Schneider, M., “Horizontal learning approach to discover association rules”, Computer, 13(3): 1–18, (2024). DOI: https://doi.org/10.3390/computers13030062
39. [39] Zhang, J., Feng, T., Timmermans, H. J. P. and Lin, Z., “Association rules and prediction of trans-portation mode choice: application to national travel survey data”, Transportation Research Part C: Emerging Technologies, 150(1): 1–12, (2023). DOI: https://doi.org/10.1016/J.TRC.2023.104086
40. [40] Zhou, S., Bi, S. and Qi, G., “Rule mining trends from 1987 to 2022: a bibliometric analysis and visualization”, Data Intelligence, 1: 1–44, (2023). DOI: https://doi.org/10.1162/dint_a_00239