Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification

Khaled Hejja; Rifat Kurban

doi:10.54559/amesia.1958087

Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification

Abstract

The classification of breast cancer from histopathological slides is a critical task in oncology, requiring a balance between fine-grained cellular texture recognition and global tissue architecture. While Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have achieved high performance, their relative behavior under matched patient-wise evaluation remains important for model selection. This paper presents a systematic evaluation of five ImageNet-initialized architectures on the BreaKHis 40X dataset, including a ConvNeXt-Hybrid variant that adds one Global Multi-Head Self-Attention block to the final ConvNeXt feature map. Across the dataset-provided patient-wise train/test partitions repeated over five splits, ConvNeXt-Hybrid achieved 89.77% ± 1.62% accuracy, 0.9530 ± 0.0185 Receiver Operating Characteristic Area Under the Curve (ROC-AUC), and 0.9681 ± 0.0194 Precision-Recall Area Under the Curve (PR-AUC). This performance was statistically indistinguishable from ConvNeXt-Base (89.63% ± 2.95% accuracy; exact Wilcoxon p = 1.00), but consistently higher than Swin-Base (85.68% ± 2.67% accuracy; paired t-test p = 0.0028; exact Wilcoxon p = 0.0625). Since 0.0625 is the smallest attainable two-sided exact Wilcoxon p-value at n = 5, the Swin comparison represents maximal non-parametric evidence at this sample size. Additional analyses include computational-cost profiling, attention-head sensitivity, Gradient- weighted Class Activation Mapping (Grad-CAM)visualization, a frozen-feature multi-instance patch aggregator, within-BreaKHis magnification-shift tests, and BreAst Cancer Histology (BACH) external validation using zero-shot and frozen-head protocols. Overall, the results support a bounded conclusion: explicit global attention is useful for a window-constrained Swin backbone, but it does not provide a statistically detectable improvement over a strong ConvNeXt backbone in this setting.

Keywords

Supporting Institution

The authors would like to thank Abdullah Gül University for providing the computational resources used in this study.

References

A. M. Alhassan, An improved breast cancer classification with hybrid chaotic sand cat and Remora Optimization, PLOS One 19 (4) (2024) e0300622.
F. A. Spanhol, L. S. Oliveira, C. Petitjean, L. Heutte, A dataset for breast cancer histopathological image classification, IEEE Transactions on Biomedical Engineering 63 (7) (2016) 1455–1462.
R. Brixtel, S. Bougleux, O. L´ ezoray, Y. Caillot, B. Lemoine, M. Fontaine, D. Nebati, A. Renouf, Whole slide image quality in digital pathology: Review and perspectives, IEEE Access 10 (2022) 131005–131035.
M. Asadi-Aghbolaghi, A. Darbandsari, A. Zhang, A. Contreras-Sanz, J. Boschman, P. Ahmadvand, M. K¨ obel, D. Farnell, D. G. Huntsman, A. Churg, P. C. Black, G. Wang, C. B. Gilks, H. Farahani, A. Bashashati, Learning generalizable AI models for multi-center histopathology image classification, npj Precision Oncology 8 (1) (2024) 151.
A. S. Syed Fiaz, B. Raza, M. Faheem, A. Raza, A deep fusion-based vision transformer for breast cancer classification, Healthcare Technology Letters 11 (6) (2024) 471–484.
F. A. Spanhol, L. S. Oliveira, C. Petitjean, L. Heutte, Breast cancer histopathological image classification using CNN, Proceedings of the International Joint Conference on Neural Networks, Vancouver, 2016, pp. 2560–2567.
M. Z. Alom, C. Yakopcic, M. S. Nasrin, T. M. Taha, V. K. Asari, Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural Network, Journal of Digital Imaging 32 (4) (2019) 605–617.
N. Ahmad, S. Asghar, S. A. Gillani, Transfer learning-assisted multi-resolution breast cancer histopathological images classification, The Visual Computer 38 (8) (2022) 2751–2770.

C. Zhu, F. Song, Y. Wang, H. Dong, Y. Guo, J. Liu, Breast cancer histopathology image classification through assembling multiple compact CNNs, BMC Medical Informatics and Decision Making 19 (1) (2019), 198.
P. J. Sudharshan, C. Petitjean, F. Spanhol, L. E. Oliveira, L. Heutte, P. Honeine, Multiple instance learning for histopathological breast cancer image classification, Expert Systems with Applications 117 (2019) 103–111.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale (2021) Accessed 23 Feb 2026.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, Proceedings of the IEEE International Conference on Computer Vision, Montreal, 2021, pp. 9992–10002.
K. Xia, J. Wang, Recent advances of transformers in medical image analysis: A comprehensive review, MedComm – Future Medicine 2 (1) (2023) e38.
M. Gour, S. Jain, T. Sunil Kumar, Residual learning based CNN for breast cancer histopathological image classification, International Journal of Imaging Systems and Technology 30 (3) (2020) 621–635.
M. Caron, H. Touvron, I. Misra, H. Jegou, J. Mairal, P. Bojanowski, A. Joulin, Emerging properties in self-supervised vision transformers (2021) Accessed 23 Feb 2026.
K. He, X. Chen, S. Xie, Y. Li, P. Dollar, R. Girshick, Masked Autoencoders Are Scalable Vision Learners, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 2022, pp. 15979–15988.
M. Y. Lu, D. F. K. Williamson, T. Chen, R. J. Chen, M. Barbieri, F. Mahmood, Data-efficient and weakly supervised computational pathology on whole-slide images, Nature Biomedical Engineering 5 (6) (2021) 555–570.
R. J. Chen, T. Ding, M. Y. Lu, D. F. K. Williamson, G. Jaume, A. H. Song, B. Chen, A. Zhang, D. Shao, M. Shaban, M. Williams, L. Oldenburg, L. L. Weishaupt, J. J. Wang, A. Vaidya, L. P. Le, G. Gerber, S. Sahai, W. Williams, F. Mahmood, A general-purpose self-supervised model for computational pathology (2023) Accessed 2 March 2026.
M. Y. Lu, B. Chen, D. F. K. Williamson, R. J. Chen, M. Zhao, A. K. Chow, K. Ikemura, A. Kim, D. Pouli, A. Patel, A. Soliman, T. Chen, F. Mahmood, A visual-language foundation model for computational pathology, Nature Medicine 30 (3) (2024) 863–874.
Q. Hou, D. Zhou, J. Feng, Coordinate Attention for Efficient Mobile Network Design, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, 2021, pp. 13713–13722.
J. Potsangbam, S. S. Devi, EMViT-BCC: Enhanced Mobile Vision Transformer for Breast Cancer Classification, International Journal of Imaging Systems and Technology 35 (2) (2025) e70053.
S. Mehta, M. Rastegari, MobileViT: Light-weight, general-purpose, and mobile-friendly vision transformer (2022) Accessed 23 Feb 2026.
S. Oh, N. Kim, J. Ryu, Analyzing to discover origins of CNNs and ViT architectures in medical images, Scientific Reports 14 (1) (2024) 8755.
E. Jenkinson, O. Arandjelovic, Whole slide image understanding in pathology: What is the salient scale of analysis?, Biomedinformatics 4 (1) (2024) 489–518.
Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, S. Xie, A ConvNet for the 2020s, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 2022, pp. 11966–11976.
G. Aresta, T. Ara´ ujo, S. Kwok, S. S. Chennamsetty, M. Safwan, V. Alex, B. Marami, M. Prastawa, M. Chan, M. Donovan, G. Fernandez, J. Zeineh, M. Kohl, C. Walz, F. Ludwig, S. Braunewell, M. Baust, Q. D. Vu, M. N. N. To, E. Kim, J. T. Kwak, S. Galal, V. Sanchez-Freire, N. Brancati, M. Frucci, D. Riccio, Y. Wang, L. Sun, K. Ma, J. Fang, I. Kone, L. Boulmane, A. Campilho, C. Eloy, A. Polonia, P. Aguiar, BACH: Grand challenge on breast cancer histology images, Medical Image Analysis 56 (2019) 122–139.
S. Border, P. Sarder, From what to why, the growing need for a focus shift toward explainability of AI in digital pathology, Frontiers in Physiology 12 (2022).

Details

Primary Language

English

Subjects

Computer Vision, Image Processing, Pattern Recognition, Deep Learning

Journal Section

Research Article

Authors

Khaled Hejja
0009-0003-9346-6618
Türkiye

Rifat Kurban ^*
0000-0002-0277-2210
Türkiye

Publication Date

June 30, 2026

Submission Date

May 24, 2026

Acceptance Date

June 26, 2026

Published in Issue

Year 2026 Volume: 7 Number: 1

DOI

https://doi.org/10.54559/amesia.1958087

IZ

https://izlik.org/JA37UH74GZ

Cite

RIS / Bibtex

APA

Hejja, K., & Kurban, R. (2026). Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification. Amesia, 7(1), 72-91. https://doi.org/10.54559/amesia.1958087

AMA

1.Hejja K, Kurban R. Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification. Amesia. 2026;7(1):72-91. doi:10.54559/amesia.1958087

Chicago

Hejja, Khaled, and Rifat Kurban. 2026. “Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification”. Amesia 7 (1): 72-91. https://doi.org/10.54559/amesia.1958087.

EndNote

Hejja K, Kurban R (June 1, 2026) Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification. Amesia 7 1 72–91.

IEEE

[1]K. Hejja and R. Kurban, “Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification”, Amesia, vol. 7, no. 1, pp. 72–91, June 2026, doi: 10.54559/amesia.1958087.

ISNAD

Hejja, Khaled - Kurban, Rifat. “Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification”. Amesia 7/1 (June 1, 2026): 72-91. https://doi.org/10.54559/amesia.1958087.

JAMA

1.Hejja K, Kurban R. Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification. Amesia. 2026;7:72–91.

MLA

Hejja, Khaled, and Rifat Kurban. “Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification”. Amesia, vol. 7, no. 1, June 2026, pp. 72-91, doi:10.54559/amesia.1958087.

Vancouver

1.Khaled Hejja, Rifat Kurban. Systematic Evaluation of Hybrid Vision Transformers and Convolutional Neural Networks for Enhanced Breast Cancer Histopathology Classification. Amesia. 2026 Jun. 1;7(1):72-91. doi:10.54559/amesia.1958087

EBSCO	DOAJ
Scilit	SOBIAD