Evaluating SISA-Based Machine Unlearning Across Diverse Modalities: Tabular, Visual and Auditory Data

Abstract

Retraining a model entirely from scratch to delete specific information is highly inefficient in terms of both time and cost. This study comprehensively evaluates the training framework known as Sharded, Isolated, Sliced, and Aggregated (SISA) as a practical solution to this problem. Researchers tested SISA’s unlearning performance across a wide range of data types. These included tabular data from the Purchase and Adult datasets, visual data from the DR Grading dataset, and auditory data from the ESC fifty dataset. The core principle of the method involves partitioning data into small, independent sections. This allows for the retraining of only the relevant component when a deletion request is made. The study’s key findings indicate that SISA’s success is largely dependent on the characteristics of the data. The approach demonstrates exceptional efficiency with large, well-structured datasets, significantly reducing retraining time while maintaining high model accuracy. However, its performance shows fluctuations when applied to more complex or irregular datasets. Ultimately, this work presents the SISA method as a scalable and viable strategy for data privacy in modern artificial intelligence systems, offering a crucial balance between model accuracy, operational efficiency, and legal compliance.

Keywords

• Machine Unlearning
• Model Forgetting
• Data Privacy
• SISA Training
• Multi-Dataset Comparison

References

1. Ali, Z., Muhammad, A., Adnan, R., Alkhalifah, T., & Aslam, S. (2025). "Evaluating machine unlearning: Applications, approaches, and accuracy". Engineering Reports, Vol. 7, Issue 1, p. e13081.
2. "Aptos 2019 blindness detection". https://www.kaggle.com/competitions/aptos2019-blindness-detection, accessed: 2025-06-11.
3. Bisong, E. (2019). "Google colaboratory". Building machine learning and deep learning models on google cloud platform: a comprehensive guide for beginners, Springer, pp. 59–64.
4. Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021). "Machine unlearning". 2021 IEEE symposium on security and privacy (SP), pp. 141–159.
5. Buyuktanir, B., Altinkaya, Ş., Karatas Baydogmus, G., & Yildiz, K. (2025). "Federated learning in intrusion detection: advancements, applications, and future directions". Cluster Computing, Vol. 28, Issue 7, pp. 1–25.
6. Buyuktanir, B., Yildiz, K., & Baydogmus, G.K. (2025). "A systematic mapping study on machine unlearning in federated learning". 2025 7th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA), pp. 1–10.
7. Büyüktanir, B., Yakar, Ö., & Altinel Girgin, A. (2024). "Hate speech framework in social media: A comprehensive review from past to present". Journal of the Faculty of Engineering and Architecture of Gazi University, Vol. 40, Issue 1.
8. Büyüktanir, B., Yildiz, K., Ülkü, E., & Bütüktanir, T. (2023). "DU-CBA: Data-agnostic and incremental classification-based association rules extraction architecture". Journal of the Faculty of Engineering and Architecture of Gazi University, Vol. 38, Issue 3.

9. Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2022). "Graph unlearning". Proceedings of the 2022 ACM SIGSAC conference on computer and communications security, pp. 499–513. Dilworth, R. (2025). "Privacy preservation through practical machine unlearning". arXiv preprint arXiv:2502.10635.
10. Dua, D., & Graff, C. (2019). "UCI machine learning repository". [Online]. Available: http://archive.ics.uci.edu/ml.
11. Fisher, R.A. (1936). "The use of multiple measurements in taxonomic problems". Annals of eugenics, Vol. 7, Issue 2, pp. 179–188.
12. Halimi, A., Kadhe, S., Rawat, A., & Baracaldo, N. (2022). "Federated unlearning: How to efficiently erase a client in fl?". arXiv preprint arXiv:2207.05521.
13. He, E., Sarwar, T., Khalil, I., Yi, X., & Wang, K. (2025). "Deep contrastive unlearning for language models". arXiv preprint arXiv:2503.14900.
14. Hu, Q., Ma, L., & Zhao, J. (2018). "Deepgraph: A pycharm tool for visualizing and understanding deep learning models". 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pp. 628–632.
15. Kadhe, S.R., Halimi, A., Rawat, A., & Baracaldo, N. (2023). "Fairsisa: Ensemble post-processing to improve fairness of unlearning in llms". arXiv preprint arXiv:2312.07420.
16. Koch, K., & Soll, M. (2023). "No matter how you slice it: Machine unlearning with sisa comes at the expense of minority classes". 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 622–637.
17. Kumar, V.B., Gangadharaiah, R., & Roth, D. (2022). "Privacy adhering machine un-learning in nlp". arXiv preprint arXiv:2212.09573.
18. Kurmanji, M., Triantafillou, E., & Triantafillou, P. (2024). "Machine unlearning in learned databases: An experimental analysis". Proceedings of the ACM on Management of Data, Vol. 2, Issue 1, pp. 1–26.
19. Li, K., Huang, Z., Cheng, Y.-C., & Lee, C.-H. (2014). "A maximal figure-of-merit learning approach to maximizing mean average precision with deep neural network based classifiers". 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4503–4507.
20. Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025). "Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects". IEEE Transactions on Neural Networks and Learning Systems.
21. Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A.D., & Van De Weijer, J. (2022). "Class-incremental learning: survey and performance evaluation on image classification". IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, Issue 5, pp. 5513–5533.
22. Miao, J., Hu, L., Zhang, Q., & Cao, L. (2025). "Graph memory learning: Imitating lifelong remembering and forgetting of brain networks". IEEE Transactions on Pattern Analysis and Machine Intelligence.
23. Mondschein, C.F., & Monda, C. (2019). "The eu’s general data protection regulation (gdpr) in a research context". Fundamentals of clinical data science, Vol. 1, pp. 55–71.
24. Muresanu, A., Thudi, A., Zhang, M.R., & Papernot, N. (2024). "Unlearnable algorithms for in-context learning". arXiv preprint arXiv:2402.00751.
25. Nguyen, T.T., Huynh, T.T., Ren, Z., Nguyen, P.L., Liew, A.W.-C., Yin, H., & Nguyen, Q.V.H. (2025). "A survey of machine unlearning". ACM Transactions on Intelligent Systems and Technology, Vol. 16, Issue 5, pp. 1–46.
26. Pandey, D., Niwaria, K., & Chourasia, B. (2019). "Machine learning algorithms: a review". Mach. Learn, Vol. 6, Issue 2.
27. Piczak, K.J. (2015). "ESC: Dataset for environmental sound classification". Proceedings of the 23rd ACM international conference on Multimedia, pp. 1015–1018.
28. Pratama, I., & Gambetta, W. (2024). "Implementation of exact machine unlearning algorithm in credit scoring cases". 2024 IEEE International Conference on Data and Software Engineering (ICoDSE), pp. 155–160.
29. Quan, Y., Li, Z., & Montana, G. (2025). "Efficient verified machine unlearning for distillation". arXiv preprint arXiv:2503.22539.
30. Reddy, S., Anand, R., et al. (2024). "Comparison of model adaptation techniques with machine unlearning". 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), pp. 1–11.
31. Ruta, D., & Gabrys, B. (2005). "Classifier selection for majority voting". Information fusion, Vol. 6, Issue 1, pp. 63–81.
32. Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017). "Membership inference attacks against machine learning models". 2017 IEEE Symposium on Security and Privacy (SP), pp. 3–18.
33. Sula, N., Kumar, A., Wang, H., Hou, J., & Tourani, R. (2025, September). Silver Linings in the Shadows: Harnessing Membership Fingerprinting for Machine Unlearning. In 2025 IEEE Conference on Communications and Network Security (CNS) (pp. 1-9). IEEE.
34. Tarun, A.K., Chundawat, V.S., Mandal, M., & Kankanhalli, M. (2023). "Fast yet effective machine unlearning". IEEE Transactions on Neural Networks and Learning Systems, Vol. 35, Issue 9, pp. 13046–13055.
35. Wang, B. (2022). "A parallel implementation of computing mean average precision". arXiv preprint arXiv:2206.09504.
36. Wang, J., Bie, H., Jing, Z., & Zhi, Y. (2025). "Scrub-and-learn: Category-aware weight modification for machine unlearning". AI, Vol. 6, Issue 6, p. 108.
37. Xu, J., Wu, Z., Wang, C., & Jia, X. (2024). "Machine unlearning: Solutions and challenges". IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. 8, Issue 3, pp. 2150–2168.
38. Yang, Y., Wang, Z., Alghazzawi, D.M., Cheng, L., Liu, G., Wang, C., Zeng, C., & Li, Y. (2025). "Fedcsa: Enhancing federated unlearning efficiency through adaptive clustering under data heterogeneity". Chinese Journal of Electronics, Vol. 34, Issue 3, pp. 970–979.
39. Yonekura, H., Ozeki, R., Amano, T., Rizk, H., & Yamaguchi, H. (2025). "Mobtext-sisa: Efficient machine unlearning for mobility logs with spatio-temporal and natural-language data". arXiv preprint arXiv:2508.19554.
40. Yu, G., Jiang, Y., Wang, Q., Wang, X., Ma, B., Sun, C., Ni, W., & Liu, R.P. (2023). "Split unlearning". arXiv preprint arXiv:2308.10422.
41. Yu, S., Sun, F., Guo, J., Zhang, R., & Cheng, X. (2022). "Legonet: A fast and exact unlearning architecture". arXiv preprint arXiv:2210.16023.
42. Zhang, D., Pan, S., Hoang, T., Xing, Z., Staples, M., Xu, X., Yao, L., Lu, Q., & Zhu, L. (2024). "To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods". AI and Ethics, Vol. 4, Issue 1, pp. 83–93. Zhang, H., Nakamura, T., Isohara, T., & Sakurai, K. (2023). "A review on machine unlearning". SN Computer Science, Vol. 4, Issue 4, p. 337.