ViTEmIS: Vision Transformer Based Email Image Spam Detector

Sultan Zavrak

doi:10.17798/bitlisfen.1779957

ViTEmIS: Vision Transformer Based Email Image Spam Detector

Abstract

Email spam is still a big threat to cybersecurity, and image-based spam makes it harder for traditional text-based spam detection methods to work. As impostors change their attack techniques, the need for creative ways to solve this problem is growing. This work presents ViTEmIS, a ViT-Base fine-tuning–based approach for detecting image-based email spam, together with a systematic analysis of duplicate-image effects on performance reporting across benchmark datasets. In addition, the proposed model is compared with the methods proposed in the previous studies. According to evaluation results, the proposed model shows promising results in terms of accuracy, precision, recall, and F1 on the different datasets. Across four benchmark datasets, ViTEmIS achieved 94.00–99.42% accuracy with duplicates included and 90.85–99.42% after duplicate removal, with corresponding F1 scores of 96.47–99.46% and 93.70–99.44%, respectively. The contributions are: (i) an evaluation of a ViT-Base fine-tuning approach for image-based email spam detection on multiple benchmarks, (ii) a systematic analysis of how duplicate images affect reported performance, and (iii) a comparative discussion with prior work with explicit modality caveats.

Keywords

References

S. Zavrak and S. Yilmaz, “Email spam detection using hierarchical attention hybrid deep learning method,” Expert Syst. Appl., vol. 233, pp. 120977–120977, Dec. 2023, doi: 10.1016/J.ESWA.2023.120977.
F. Jáñez-Martino, R. Alaiz-Rodríguez, V. González-Castro, E. Fidalgo, and E. Alegre, “A review of spam email detection: analysis of spammer strategies and the dataset shift problem,” Artif. Intell. Rev. 2022 562, vol. 56, no. 2, pp. 1145–1173, May 2022, doi: 10.1007/S10462-022-10195-4.
Y. Gao, A. Choudhary, and G. Hua, “A Comprehensive Approach to Image Spam Detection: From Server to Client Solution,” IEEE Trans. Inf. Forensics Secur., vol. 5, no. 4, pp. 826–836, Dec. 2010, doi: 10.1109/TIFS.2010.2080267.
M. Dredze, R. Gevaryahu, and A. Elias-Bachrach, “Learning Fast Classifiers for Image Spam,” presented at the International Conference on Email and Anti-Spam, 2007. Accessed: Aug. 18, 2024. [Online]. Available: https://www.semanticscholar.org/paper/Learning-Fast-Classifiers-for-Image-Spam-Dredze-Gevaryahu/c6d975fcfd9a6f698e9eff0118fa3185dc8111e4
B. Mehta, S. Nangia, M. Gupta, and W. Nejdl, “Detecting image spam using visual features and near duplicate detection,” in Proceedings of the 17th international conference on World Wide Web, in WWW ’08. New York, NY, USA: Association for Computing Machinery, Apr. 2008, pp. 497–506. doi: 10.1145/1367497.1367565.
A. Annadatha and M. Stamp, “Image spam analysis and detection,” J. Comput. Virol. Hacking Tech., vol. 14, no. 1, pp. 39–52, Feb. 2018, doi: 10.1007/s11416-016-0287-x.
J. Shen, R. H. Deng, Z. Cheng, L. Nie, and S. Yan, “On robust image spam filtering via comprehensive visual modeling,” Pattern Recognit., vol. 48, no. 10, pp. 3227–3238, Oct. 2015, doi: 10.1016/j.patcog.2015.02.027.
T. Sharmin, F. Di Troia, K. Potika, and M. Stamp, “Convolutional neural networks for image spam detection,” Inf. Secur. J. Glob. Perspect., vol. 29, no. 3, pp. 103–117, May 2020, doi: 10.1080/19393555.2020.1722867.

W. M. Salama, M. H. Aly, and Y. Abouelseoud, “Deep learning-based spam image filtering,” Alex. Eng. J., vol. 68, Jan. 2023, doi: 10.1016/j.aej.2023.01.048.
T. Kumaresan, S. Saravanakumar, and R. Balamurugan, “Visual and textual features based email spam classification using S-Cuckoo search and hybrid kernel support vector machine,” Clust. Comput., vol. 22, no. 1, pp. 33–46, Jan. 2019, doi: 10.1007/s10586-017-1615-8.
S.-G. Nam, Y. Jang, D.-G. Lee, and Y.-S. Seo, “Hybrid Features by Combining Visual and Text Information to Improve Spam Filtering Performance,” Electronics, vol. 11, no. 13, Art. no. 13, Jan. 2022, doi: 10.3390/electronics11132053.
A. Makkar and N. Kumar, “PROTECTOR: An optimized deep learning-based framework for image spam detection and prevention,” Future Gener. Comput. Syst., vol. 125, pp. 41–58, Dec. 2021, doi: 10.1016/j.future.2021.06.026.
A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” Oct. 2020, [Online]. Available: https://arxiv.org/abs/2010.11929v2
E. Asadi Shamsabadi, C. Xu, A. S. Rao, T. Nguyen, T. Ngo, and D. Dias-da-Costa, “Vision transformer-based autonomous crack detection on asphalt and concrete surfaces,” Autom. Constr., vol. 140, pp. 104316–104316, Aug. 2022, doi: 10.1016/J.AUTCON.2022.104316.
G. Fumera, I. Pillai, and F. Roli, “Spam Filtering Based On The Analysis Of Text Information Embedded Into Images,” J Mach Learn Res, Dec. 2006, Accessed: Aug. 18, 2024. [Online]. Available: https://www.semanticscholar.org/paper/Spam-Filtering-Based-On-The-Analysis-Of-Text-Into-Fumera-Pillai/707cd6830fa9faffd6b8afb0a7f1b9a145e5c446
Y. Gao et al., “Image spam hunter,” in 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, Mar. 2008, pp. 1765–1768. doi: 10.1109/ICASSP.2008.4517972.
“google/vit-base-patch16-224 · Hugging Face.” Accessed: Nov. 21, 2024. [Online]. Available: https://huggingface.co/google/vit-base-patch16-224

Details

Primary Language

English

Subjects

Natural Language Processing

Journal Section

Research Article

Authors

Sultan Zavrak ^*
0000-0001-6950-8927
Türkiye

Publication Date

June 30, 2026

Submission Date

September 8, 2025

Acceptance Date

March 31, 2026

Published in Issue

Year 2026 Volume: 15 Number: 2

DOI

https://doi.org/10.17798/bitlisfen.1779957

IZ

https://izlik.org/JA75JG84WH

Cite

RIS / Bibtex

APA

Zavrak, S. (2026). ViTEmIS: Vision Transformer Based Email Image Spam Detector. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, 15(2), 557-565. https://doi.org/10.17798/bitlisfen.1779957

AMA

1.Zavrak S. ViTEmIS: Vision Transformer Based Email Image Spam Detector. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026;15(2):557-565. doi:10.17798/bitlisfen.1779957

Chicago

Zavrak, Sultan. 2026. “ViTEmIS: Vision Transformer Based Email Image Spam Detector”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15 (2): 557-65. https://doi.org/10.17798/bitlisfen.1779957.

EndNote

Zavrak S (June 1, 2026) ViTEmIS: Vision Transformer Based Email Image Spam Detector. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15 2 557–565.

IEEE

[1]S. Zavrak, “ViTEmIS: Vision Transformer Based Email Image Spam Detector”, Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, vol. 15, no. 2, pp. 557–565, June 2026, doi: 10.17798/bitlisfen.1779957.

ISNAD

Zavrak, Sultan. “ViTEmIS: Vision Transformer Based Email Image Spam Detector”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi 15/2 (June 1, 2026): 557-565. https://doi.org/10.17798/bitlisfen.1779957.

JAMA

1.Zavrak S. ViTEmIS: Vision Transformer Based Email Image Spam Detector. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026;15:557–565.

MLA

Zavrak, Sultan. “ViTEmIS: Vision Transformer Based Email Image Spam Detector”. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, vol. 15, no. 2, June 2026, pp. 557-65, doi:10.17798/bitlisfen.1779957.

Vancouver

1.Sultan Zavrak. ViTEmIS: Vision Transformer Based Email Image Spam Detector. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi. 2026 Jun. 1;15(2):557-65. doi:10.17798/bitlisfen.1779957