A proposed classification method approach for binary variable data using Boolean algebra and an application to digital advertising

Abstract

In this paper, Boolean decision table (BDT) approach is proposed as a new classification technique for binary variables using Boolean algebra. Since the proposed BDT approach is similar to the decision tree methods used in classification analysis, the performance of the BDT approach is compared with the widely used decision tree methods in the literature: classification and regression tree (CART), random forest (RF), and extreme gradient boost (XGBoost) algorithms. While making the comparison, attention was paid to the classification performance of the models (classification accuracy, ROC, and PR curve) as well as the interpretability of the results obtained. The benefits and drawbacks of the proposed BDT approach were analyzed using real data from digital ads of an e-commerce company. The results of the analysis show that the BDT approach outperforms RF and CART algorithms in classification and is close to the XGBoost algorithm. The BDT approach has demonstrated greater validity in the digital advertising industry because, in comparison to the XGBoost algorithm, its results are more interpretable. Furthermore, classification performance was also compared using a future dataset from the same e-commerce company that is not included in the training or test datasets. Important target audiences were identified in addition to classification performance because target audiences are crucial to digital advertising. A multi-criteria decision-making technique called TOPSIS was used to ascertain the relative importance of the target audiences. Both the proposal of the BDT approach and the evaluation of the results of the classification algorithms using the TOPSIS method are considered to contribute to the literature in this field.

Keywords

• Boolean algebra
• classification and regression tree
• random forest
• extreme gradient boost

Thanks

The authors would like to thank the editor and anonymous reviewers for their constructive comments which led to the improvement of the paper

References

1. Aarthi, G., Karthikha, R., Sankar, S., Priya, S. S., Jamal, D. N., Banu, W. A., Application of Machine Learning in Customer Services and E-commerce. Data Management, Data Management, Analytics and Innovation, Springer, Singapore, 2023.
2. Aruldoss, M., Lakshmi, T. M., Venkatesan, V. P., A Survey on multi criteria decision making methods and its applications, American Journal of Information Systems, 1(1) (2013), 31-43.
3. Becker, B. G., Visualizing decision table classifiers, In Proceedings IEEE Symposium on Information Visualization, (1998).
4. Becker, C., Rigamonti, R., Lepetit, V., Fua, P., Supervised Feature Learning for Curvilinear Structure Segmentation, In Medical Image Computing and Computer-Assisted Intervention - MICCAI, Springer Berlin, 2013.
5. Bianchi, F., Garnett, E., Dorsel, C., Aveyard, P., Jebb, S. A., Restructuring physical micro-environments to reduce the demand for meat: a systematic review and qualitative comparative analysis, The Lancet Planetary Health, 2(9) (2018), e384–e397.
6. Blackman, T., Dunstan, K., Qualitative comparative analysis and health inequalities: investigating reasons for differential progress with narrowing local gaps in mortality, Journal of Social Policy, 39(3) (2010), 359–373.
7. Boole, G., The Calculus of Logic, (1848).
8. Breiman, L., Bagging Predictors, Machine Learning, 24 (1996), 123-140.

9. Breiman, L., Friedman, J. H., Olshen, R. A., Stone, C. J., Classification and Regression Trees, Routledge, New York, 1984.
10. Breiman, L., Random Forests, Machine Learning, 45(1) (2001), 5-32.
11. Chen, T., Guestrin, C., XGBoost: a scalable tree boosting system, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, (2016), 785–794.
12. Chen, T., He, T., Benesty, M., Khotilovich, V., Tang, Y., Cho, H., Chen, K., Mitchell, R., Cano, I., Zhou, T., Li, M., Junyuan Xie, Lin, M., Geng, Y., Li, Y., Xgboost: Extreme Gradient Boosting, R package version 1.4.1.1, (2021).
13. Chiu, C., Ku, Y., Lie, T., Chen, Y., Internet auction fraud detection using social network analysis and classification tree approaches, International Journal of Electronic Commerce, 15(3) (2011), 123–147.
14. Cramer, J. S., Logit Models from Economics and Other Fields, Cambridge University Press, 2003.
15. Davis, J., Goadrich, M., The relationship between precision-recall and ROC curves, Proceedings of the 23rd International Conference on Machine learning, (2006), 233–240.
16. Ekelik, H., Emir, Ş., A comparison of machine learning classifiers for evaluation of remarketing audiences in e-commerce, Eskişehir Osmangazi University Journal of Economics and Administrative Sciences, 16(2) (2021), 341–359.
17. Fawcett, T., An introduction to ROC analysis, Pattern Recognition Letters, 27(8) (2006), 861–874.
18. Fernández, A., García, S., Galar, M., Prati, R. C., Krawczyk, B., Herrera, F., Learning from Imbalanced Data Sets, Springer Nature, Switzerland, 2018.
19. Fratta, L., Montanari, U., A Boolean algebra method for computing the terminal reliability in a communication network, IEEE Transactions on Circuit Theory, 20(3) (1973), 203–211.
20. Freeman, E. A., Moisen, G. G., A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and kappa, Ecological Modelling, 217(1) (2008), 48-58.
21. Freitas, A. A., Comprehensible classification models: a position paper, ACM SIGKDD Explorations Newsletter, 15(1) (2014), 1–10.
22. Friedman, J. H., Greedy function approximation: a gradient boosting machine, Annals of Statistics, (2001), 1189–1232.
23. Friedman, J. H., Stochastic gradient boosting, Computational Statistics & Data Analysis, 38(4) (2002), 367–378.
24. Gran, B., Aliberti, D., The office of the children’s ombudsperson: children’s rights and social-policy innovation, International Journal of the Sociology of Law, 31(2) (2003), 89–106.
25. Hailperin, T., Boole’s algebra isn’t Boolean algebra, Mathematics Magazine, 54(4) (1981), 173–184.
26. Han, J., Kamber, M., Pei, J., Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann, 2012.
27. Hastie, T., Tisbshirani, R., Friedman, J., The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, 2018.
28. Ho, T. K., The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8) (1998), 832–844.
29. Hosmer, D. W., Lemeshow, S., Sturdivant, R. X., Applied Logistic Regression, 3rd ed., John Wiley & Sons, 2013.
30. Huang, S. H., Supervised feature selection: A tutorial, Artificial Intelligence Research, 4(2) (2015), 22-37.
31. James, G., Witten, D., Hastie, T., Tibshirani, R., An Introduction to Statistical Learning: with Applications in R, Springer, New York, 2013.
32. Jiménez‐Hernández, E. M., Oktaba, H., Díaz-Barriga, F., Piattini, M., Using web-based gamified software to learn Boolean algebra simplification in a blended learning setting, Computer Applications in Engineering Education, 28(6) (2020), 1591–1611.
33. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., Liu, T. Y., LightGBM: a highly efficient gradient boosting decision tree, Proceedings of the 31st International Conference on Neural Information Processing Systems, (2017), 3149–3157.
34. Kim, J. W., Lee, B. H., Shaw, M. J., Chang, H. L., Nelson, M., Application of decision-tree induction techniques to personalized advertisements on internet storefronts, International Journal of Electronic Commerce, 5(3) (2001), 45–62.
35. Kleinbaum, D. G., Klein, M., Logistic Regression A Self-Learning Text, 3rd ed., Springer New York, 2010.
36. Kumar, P., A Study on interconnection between Boolean algebra and binary tree, Globus an International Journal of Management & IT, 9(2) (2018), 1–2.
37. Kumar, R., Lawrance, R., Boolean Rule Based Classification for Microarray Gene Expression Data, International Journal of Recent Technology and Engineering, 2019.
38. Larose, D. T., Larose, C. D., Discovering Knowledge in Data: an Introduction to Data Mining, 2nd ed., John Wiley & Sons, 2014.
39. Liaw, A., Wiener, M., Classification and regression by randomForest, R News, 2(3) (2002), 18-22.
40. Lima, E., Mues, C., Baesens, B., Domain knowledge integration in data mining using decision tables: case studies in churn prediction, Journal of the Operational Research Society, 60(8) (2009), 1096–1106.
41. Lindaman, R., A theorem for deriving majority-logic networks within an augmented Boolean algebra, IRE Transactions on Electronic Computers, 3 (1960), 338–342.
42. Liu, C. J., Huang, T. S., Ho, P. T., Huang, J. C., Hsieh, C. T., Machine learning-based e-commerce platform repurchase customer prediction model, PLOS ONE, 15(12) (2020), e0243105.
43. Lu, H., Liu, H., Decision tables: scalable classification exploring RDBMS capabilities, Proceedings of the 26th Inter- national Conference on Very Large Databases, Cairo, Egypt, 2000.
44. Maimon, O., Rokach, L., Data Mining and Knowledge Discovery Handbook, 2nd ed., Springer, 2010.
45. Mitchell, R., Frank, E., Accelerating the XGBoost algorithm using GPU computing, PeerJ Computer Science, 3 (2017), e127.
46. Muller, D. E., Application of Boolean algebra to switching circuit design and to error detection, Transactions of the IRE Professional Group on Electronic Computers, 3 (1954), 6–12.
47. Ogihara, H., Fujita, Y., Hamamoto, Y., Iizuka, N., Oka, M., Classification based on boolean algebra and its application to the prediction of recurrence of liver cancer, 2nd IAPR Asian Conference on Pattern, 2013.
48. Özcan, T., Çelebi, N., Esnaf, Ş., Comparative analysis of multi-criteria decision making methodologies and implementation of a warehouse location selection problem, Expert Systems with Applications, 38(8) (2011), 9773-9779.
49. Phiffer, P. E., Concepts of Probability Theory, Second Revised Edition, Dover Publications, New York, 1978.
50. Provost, F., Fawcett, T., Analysis and visualization of classifier performance: comparison under imprecise class and cost distributions, Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, (1997), 43–48.
51. Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N. D., Dataset Shift in Machine Learning, The MIT Press, 2008.
52. Ragin, C. C., The Comparative Method, University of California Press, 2014.
53. Rokach, L., Maimon, O., Data Mining with Decision Trees: Theory and Applications, 2nd ed., World Scientific Publishing, 2015.
54. Rokach, L., Pattern Classification Using Ensemble Methods, Singapore, World Scientific Publishing, 2010.
55. Rushdi, A. M., Zagzoog, S. S., Balamesh, A. S., Design of a hardware circuit for integer factorization using a big Boolean algebra, Journal of Advances in Mathematics and Computer Science, (2018), 1–25.
56. Son, J., Jung, I., Park, K., Han, B., Tracking-by-Segmentation with Online Gradient Boosting Decision Tree, IEEE International Conference on Computer Vision (ICCV), 2015.
57. Tekin, M., Calculation of Probabilities of Some Statistical Events with the Help of Boolean Algebra, Unpublished Master’s Thesis, Istanbul University, Institute of Social Sciences, 1989.
58. Therneau, T., Atkinson, B., rpart: Recursive Partitioning and Regression Trees, R package version 4.1-15, (2019).
59. Thomas, R., Boolean formalization of genetic control circuits, Journal of Theoretical Biology, 42(3) (1973), 563–585.
60. Vis, B., Under which conditions does spending on active labor market policies increase? An fsQCA analysis of 53 governments between 1985 and 2003, European Political Science Review, 3(2) (2011), 229–252.
61. Wang, R. S., Saadatpour, A., Albert, R., Boolean modeling in systems biology: an overview of methodology and applications, Physical Biology, 9(5) (2012), 055001.
62. Xiao, Y., Mehrotra, K. G., Mohan, C. K., Efficient classification of binary data stream with concept drifting using conjunction rule based boolean classifier, In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Springer, 2015.
63. Zhang, Q., Li, Z., Boolean algebra of two-dimensional continua with arbitrarily complex topology, Mathematics of Computation, 89(325) (2020), 2333–2364.
64. Zheng, A., Casari, A., Feature Engineering for Machine Learning, O’Reilly Media, California, 2018.