Evaluating Vision Transformer Models for Breast Cancer Detection in Mammographic Imaging

Abstract

Breast cancer is a leading cause of mortality among women, with early detection being crucial for effective treatment. Mammographic analysis, particularly the identification and classification of breast masses, plays a crucial role in early diagnosis. Recent advancements in deep learning, particularly Vision Transformers (ViTs), have shown significant potential in image classification tasks across various domains, including medical imaging. This study evaluates the performance of different Vision Transformer (ViT) models—specifically, base-16, small-16, and tiny-16—on a dataset of breast mammography images with masses. We perform a comparative analysis of these ViT models to determine their effectiveness in classifying mammographic images. By leveraging the self-attention mechanism of ViTs, our approach addresses the challenges posed by complex mammographic textures and low contrast in medical imaging. The experimental results provide insights into the strengths and limitations of each ViT model configuration, contributing to an informed selection of architectures for breast mass classification tasks in mammography. This research underscores the potential of ViTs in enhancing diagnostic accuracy and serves as a benchmark for future exploration of transformer-based architectures in the field of medical image classification.

Keywords

• Breast Mammography with Masses
• Image Classification
• Vision Transformers
• base-16
• small-16
• tiny-16

Ethical Statement

The study is complied with research and publication ethics.

References

1. M. Arnold et al., "Current and future burden of breast cancer: Global statistics for 2020 and 2040," The Breast, vol. 66, pp. 15-23, 2022.
2. C. I. Lee and J. G. Elmore, "Beyond survival: a closer look at lead-time bias and disease-free intervals in mammography screening," JNCI: Journal of the National Cancer Institute, vol. 116, no. 3, pp. 343-344, 2024.
3. L. N. Fuzzell et al., "Cervical cancer screening in the United States: Challenges and potential solutions for underscreened groups," Preventive Medicine, vol. 144, p. 106400, 2021.
4. G. Savarese et al., "Global burden of heart failure: a comprehensive and updated review of epidemiology," Cardiovascular Research, vol. 118, no. 17, pp. 3272-3287, 2022.
5. S. Sriussadaporn et al., "Ultrasonography increases sensitivity of mammography for diagnosis of multifocal, multicentric breast cancer using 356 whole breast histopathology as a gold standard," Surgical Practice, vol. 26, no. 3, pp. 181-186, 2022.
6. N. Pashayan et al., "Personalized early detection and prevention of breast cancer: ENVISION consensus statement," Nature Reviews Clinical Oncology, vol. 17, no. 11, pp. 687-705, 2020.
7. L. Nicosia et al., "History of mammography: analysis of breast imaging diagnostic achievements over the last century," Healthcare, vol. 11, no. 11, p. 1596, 2023.
8. C. Poggi, "The Evolution of the Radiographer’s Educational Path: EBP and Communication Skills in the Mammography Room," in Breast Imaging Techniques for Radiographers, Springer Nature Switzerland, 2024, pp. 259-276.

9. H. O. Kolade-Yunusa and U. D. Itanyi, "Outcome of mammography examination in asymptomatic women," Annals of African Medicine, vol. 20, no. 1, pp. 52-58, 2021.
10. L. Abdelrahman et al., "Convolutional neural networks for breast cancer detection in mammography: A survey," Computers in Biology and Medicine, vol. 131, p. 104248, 2021.
11. D. Barba et al., "Breast cancer, screening and diagnostic tools: All you need to know," Critical Reviews in Oncology/Hematology, vol. 157, p. 103174, 2021.
12. W. Y. Sung et al., "Experiences of women who refuse recall for further investigation of abnormal screening mammography: A qualitative study," International Journal of Environmental Research and Public Health, vol. 19, no. 3, p. 1041, 2022.
13. H. J. Han et al., "Characteristics of breast cancers detected by screening mammography in Taiwan: a single institute’s experience," BMC Women's Health, vol. 23, no. 1, p. 330, 2023.
14. A. Aleissaee et al., "Transformers in remote sensing: A survey," Remote Sensing, vol. 15, no. 7, p. 1860, 2023.
15. Y. Liu et al., "A survey of visual transformers," IEEE Transactions on Neural Networks and Learning Systems, 2023.
16. A. Khan et al., "A survey of the recent architectures of deep convolutional neural networks," Artificial Intelligence Review, vol. 53, pp. 5455-5516, 2020.
17. A. Agarwal and N. Ratha, "Deep Learning in Computer Vision Progress and Threats," in Applications of Artificial Intelligence, Big Data and Internet of Things in Sustainable Development, vol. 23, 2022.
18. A. Dosovitskiy et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," arXiv:2010.11929, 2020.
19. Y. Yuan et al., "Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet," in Proc. ECCV, 2021, pp. 558–576.
20. A. Steiner et al., "How to train your vit? data, augmentation, and regularization in vision transformers," arXiv preprint arXiv:2106.10270, 2021.
21. X. Wu et al., "CTransCNN: Combining transformer and CNN in multilabel medical image classification," Knowledge-Based Systems, vol. 281, p. 111030, 2023.
22. M. Hayat et al., "Hybrid Deep Learning EfficientNetV2 and Vision Transformer (EffNetV2-ViT) Model for Breast Cancer Histopathological Image Classification," IEEE Access, 2024.
23. G. Ayana and S. W. Choe, "Vision transformers-based transfer learning for breast mass classification from multiple diagnostic modalities," Journal of Electrical Engineering & Technology, vol. 19, no. 5, pp. 3391-3410, 2024.
24. M. L. Abimouloud et al., "Advancing breast cancer diagnosis: token vision transformers for faster and accurate classification of histopathology images," Visual Computing for Industry, Biomedicine, and Art, vol. 8, no. 1, p. 1, 2025.
25. NHS to launch world’s biggest trial of AI breast cancer diagnosis, The Guardian, Feb. 4, 2025. [Online]. Available: https://www.theguardian.com/society/2025/feb/04/nhs-to-launch-worlds-biggest-trial-of-ai-breast-cancer-diagnosis.
26. Kaggle, "Mammography Dataset from INbreast, MIAS and DDSM," accessed Nov. 4, 2024. [Online]. Available: https://www.kaggle.com/datasets/emiliovenegas1/mammography-dataset-from-inbreast-mias-and-ddsm.
27. M. A. Al-Antari et al., "Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms," Computer Methods and Programs in Biomedicine, vol. 196, p. 105584, 2020.
28. X. Li et al., "Multiparametric magnetic resonance imaging for predicting pathological response after the first cycle of neoadjuvant chemotherapy in breast cancer," Investigative Radiology, vol. 50, no. 4, pp. 195-204, 2015.
29. L. G. Falconí et al., "Transfer learning in breast mammogram abnormalities classification with mobilenet and nasnet," in 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), 2019, pp. 109-114.
30. W. Hu et al., "A state-of-the-art survey of artificial neural networks for whole-slide image analysis: from popular convolutional neural networks to potential visual transformers," Computers in Biology and Medicine, vol. 161, p. 107034, 2023.
31. S. K. Hamed et al., "Enhanced Feature Representation for Multimodal Fake News Detection Using Localized Fine-Tuning of Improved BERT and VGG-19 Models," Arabian Journal for Science and Engineering, pp. 1-17, 2024.
32. Y. Fang et al., "You only look at one sequence: Rethinking transformer in vision through object detection," Advances in Neural Information Processing Systems, vol. 34, pp. 26183-26197, 2021.
33. Z. Zeng et al., "You only sample (almost) once: Linear cost self-attention via Bernoulli sampling," in International Conference on Machine Learning, PMLR, 2021, pp. 12321-12332.
34. A. Rehman, "Transformers in Computer Vision: Recent Advances and Applications," International Journal of Advanced Engineering Technologies and Innovations, vol. 1, no. 1, 2022.
35. M. Hassanin et al., "Visual attention methods in deep learning: An in-depth survey," Information Fusion, vol. 108, p. 102417, 2024.
36. M. Li et al., "SACNN: Self-attention convolutional neural network for low-dose CT denoising with self-supervised perceptual loss network," IEEE Transactions on Medical Imaging, vol. 39, no. 7, pp. 2289-2301, 2020.
37. M. H. Guo et al., "Beyond self-attention: External attention using two linear layers for visual tasks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5436-5447, 2022.
38. R. Divya and J. Prabhakar, "Vision Transformer-based Model for Human Action Recognition in Still Images," Journal of Computational Analysis and Applications, vol. 33, no. 08, pp. 522-531, 2024.
39. E. Şahin et al., "Multi-objective optimization of ViT architecture for efficient brain tumor classification," Biomedical Signal Processing and Control, vol. 91, p. 105938, 2024.
40. Marqo, "Introduction to Vision Transformers," accessed Nov. 4, 2024. [Online]. Available: https://www.marqo.ai/course/introduction-to-vision-transformers.
41. A. Sriwastawa and J. A. Arul Jothi, "Vision transformer and its variants for image classification in digital breast cancer histopathology: A comparative study," Multimedia Tools and Applications, vol. 83, no. 13, pp. 39731-39753, 2024.
42. V. Jain et al., "Transformers are adaptable task planners," in Conference on Robot Learning, PMLR, 2023, pp. 1011-1037.
43. S. Wang et al., "When transformer meets robotic grasping: Exploits context for efficient grasp detection," IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8170-8177, 2022.
44. L. Xu et al., "Mctformer+: Multi-class token transformer for weakly supervised semantic segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
45. A. Deihim et al., "STTRE: A Spatio-Temporal Transformer with Relative Embeddings for multivariate time series forecasting," Neural Networks, vol. 168, pp. 549-559, 2023.
46. W. Wang et al., "Semi-supervised vision transformer with adaptive token sampling for breast cancer classification," Frontiers in Pharmacology, vol. 13, p. 929755, 2022.
47. H. E. Kim et al., "Transfer learning for medical image classification: A literature review," BMC Medical Imaging, vol. 22, no. 1, p. 69, 2022.
48. L. Xu et al., "Multi-class token transformer for weakly supervised semantic segmentation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4310-4319.
49. O. S. Khedr et al., "The classification of bladder cancer based on Vision Transformers (ViT)," Scientific Reports, vol. 13, no. 1, p. 20639, 2023.