CBMIR_SMEr: An Effective Multi-Domain Content-Based Image Retrieval Method with Feature Selection for Colorectal Histopathological Images

Öz

Histopathological images used in colorectal cancer diagnosis can enhance the efficiency of clinical processes when supported by content-based medical image retrieval (CBMIR) systems. In this study, a method called CBMIR_SMEr, based on low-parameter and computationally efficient deep learning architectures, is proposed. The method was applied to the EBHI-Seg dataset as a CBMIR approach for histopathological colorectal images. Features extracted from ShuffleNet, MobileNetV2, and EfficientNet-B0 architectures were combined, and the most discriminative features were selected using the ReliefF algorithm. Image similarities were calculated using Euclidean and Cosine distance metrics. On the EBHI-Seg dataset, using the Cosine similarity, mAP@5 = 0.9511 and Precision@5 = 0.9141 were achieved for 800 features, while using Euclidean similarity, mAP@5 = 0.9353 (for 800 features) and Precision@5 = 0.9109 (for 600 features) were obtained. To evaluate the robustness of the method, experiments were also conducted on the UC Merced dataset, where Euclidean similarity achieved mAP@5 = 0.9461 (600 features) and Precision@5 = 0.9259 (800 features), while Cosine similarity achieved mAP@5 = 0.9577 (600 features) and Precision@5 = 0.9340 (800 features). These results represent the highest retrieval performance obtained on both datasets. Experimental findings demonstrate that the proposed CBMIR_SMEr method performs with high accuracy and stability on both histopathological and general image datasets, contributing as one of the first CBMIR-based approaches in colorectal image analysis.

Anahtar Kelimeler

• Colorectal cancer
• content-based medical image retrieval (CBMIR)
• ReliefF algorithm
• EBHI-Seg dataset
• UC Merced dataset

CBMIR_SMEr: Kolorektal Histopatolojik Görüntüler için Öznitelik Seçimi Destekli ve Çok Alanlı İçerik Tabanlı Görüntü Geri Getirme Yaklaşımı

Öz

Kolorektal kanser tanısında kullanılan histopatolojik görüntüler, içerik tabanlı medikal görüntü geri getirme (CBMIR) sistemleriyle desteklenerek klinik süreçlerin etkinliğini artırabilir. Bu çalışmada, düşük parametreli ve hesaplama açısından verimli derin öğrenme mimarilerine dayanan CBMIR_SMEr adlı bir yöntem önerilmektedir. Yöntem, histopatolojik kolorektal görüntülere yönelik bir CBMIR yaklaşımı olarak EBHI-Seg veri seti üzerinde uygulanmıştır. ShuffleNet, MobileNetV2 ve EfficientNet-B0 mimarilerinden çıkarılan öznitelikler birleştirilmiş, ardından ReliefF algoritmasıyla en ayırt edici öznitelikler seçilmiştir. Görüntü benzerlikleri Öklidyen (Euclidean) ve Kosinüs (Cosine) ölçüm yöntemleriyle hesaplanmıştır. EBHI-Seg veri seti üzerinde, Kosinüs benzerliği kullanıldığında 800 öznitelik için mAP@5 = 0.9511 ve Precision@5 = 0.9141, Öklidyen benzerliği kullanıldığında ise 800 öznitelik için mAP@5 = 0.9353 ve 600 öznitelik için Precision@5 = 0.9109 değerleri elde edilmiştir. Yöntemin kararlılığı UC Merced görüntü veri kümesi üzerinde test edilmiş, Öklidyen benzerliği ile 600 öznitelikte mAP@5 = 0.9461 ve 800 öznitelikte Precision@5 = 0.9259, Kosinüs benzerliği ile 600 öznitelikte mAP@5 = 0.9577 ve 800 öznitelikte Precision@5 = 0.9340 değerleri elde edilmiştir. Bu sonuçlar, her iki veri kümesi için de elde edilen en yüksek geri getirme performanslarını göstermektedir. Deneysel bulgular, önerilen CBMIR_SMEr yönteminin hem histopatolojik hem de genel görüntü veri kümelerinde yüksek doğruluk ve kararlılıkla çalıştığını ortaya koymakta ve kolorektal görüntü analizi alanında literatüre önemli bir katkı sağlamaktadır.

Anahtar Kelimeler

• Kolorektal kanser
• içerik tabanlı medikal görüntü geri getirme (CBMIR)
• ReliefF algoritması
• EBHI-Seg veri seti
• UC Merced veri seti

Kaynakça

1. B. Ozogul, A. Kisaoglu, G. Ozturk, S. S. Atamanalp, M. İ. Yıldırgan, and B. Aydinli, “Perfore kolon kanserinin tedavisi,” European Journal of General Medicine, vol. 11, no. 3, pp. 164–168, 2014, doi: 10.15197/sabad.1.11.63.
2. M. T. Bostancı, I. Yilmaz, Y. Akturk, A. Gökçe, M. Saydam, and I. Esen Bostancı, “Accuracy of Preoperative Computed Tomography for Lymph Node Status Screening in Colon Cancer,” Acta Oncologica Turcica, vol. 54, no. 3, pp. 328–334, 2021, doi: 10.5505/aot.2021.37790.
3. V. C. Özcan et al., “Comparison of Clavien-Dindo and Common Terminology Criteria for Adverse Events Classifications in Postoperative Early-Stage Complications of Patients with Colorectal Cancer,” Acta Oncologica Turcica, vol. 55, no. 2, pp. 165–173, 2022, doi: 10.5505/aot.2022.67699.
4. A. ULGEN, Ş. ÇETİN, and İ. DEDE, “Survival Analysis in Colon Cancer Patients,” Journal of Contemporary Medicine, vol. 11, no. 3, pp. 374–379, May 2021, doi: 10.16899/jcm.902588.
5. O. M. Celayir, “Evaluation of HER2 overexpression, clinicopathological characteristics, and factors affecting survival in gastric cancer,” SiSli Etfal Hastanesi Tip Bulteni / The Medical Bulletin of Sisli Hospital, 2021, doi: 10.14744/semb.2021.23356.
6. W. Hu et al., “EBHI: A new Enteroscope Biopsy Histopathological H&E Image Dataset for image classification evaluation,” Physica Medica, vol. 107, Mar. 2023, doi: 10.1016/j.ejmp.2023.102534.
7. U. A. Khan, A. Javed, and R. Ashraf, “An effective hybrid framework for content based image retrieval (CBIR),” Multimed Tools Appl, vol. 80, no. 17, pp. 26911–26937, Jul. 2021, doi: 10.1007/s11042-021-10530-x.
8. R. Shetty, V. S. Bhat, and J. Pujari, “Content-based medical image retrieval using deep learning-based features and hybrid meta-heuristic optimization,” Biomed Signal Process Control, vol. 92, Jun. 2024, doi: 10.1016/j.bspc.2024.106069.

9. S. Kumar, M. K. Singh, and M. Mishra, “Efficient Deep Feature Based Semantic Image Retrieval,” Neural Process Lett, vol. 55, no. 3, pp. 2225–2248, Jun. 2023, doi: 10.1007/s11063-022-11079-y.
10. M. Alrahhal and K. P. Supreethi, “Integrating machine learning algorithms for robust content-based image retrieval,” International Journal of Information Technology (Singapore), vol. 16, no. 8, pp. 5005–5021, Dec. 2024, doi: 10.1007/s41870-024-02169-2.
11. S. R. Dubey, S. K. Singh, and R. K. Singh, “Multichannel decoded local binary patterns for content-based image retrieval,” IEEE Transactions on Image Processing, vol. 25, no. 9, pp. 4018–4032, Sep. 2016, doi: 10.1109/TIP.2016.2577887.
12. R. Kapoor, D. Sharma, and T. Gulati, “State of the art content based image retrieval techniques using deep learning: a survey,” Multimed Tools Appl, vol. 80, no. 19, pp. 29561–29583, Aug. 2021, doi: 10.1007/s11042-021-11045-1.
13. Y. Dogan, “A New Global Pooling Method for Deep Neural Networks: Global Average of Top-K Max- Pooling,” Traitement du Signal, vol. 40, no. 2, pp. 577–587, Apr. 2023, doi: 10.18280/ts.400216.
14. P. Liu, J. M. Guo, C. Y. Wu, and D. Cai, “Fusion of deep learning and compressed domain features for content-based image retrieval,” IEEE Transactions on Image Processing, vol. 26, no. 12, pp. 5706–5717, Dec. 2017, doi: 10.1109/TIP.2017.2736343.
15. S. Murala and Q. M. J. Wu, “Local mesh patterns versus local binary patterns: Biomedical image indexing and retrieval,” IEEE J Biomed Health Inform, vol. 18, no. 3, pp. 929–938, 2014, doi: 10.1109/JBHI.2013.2288522.
16. T. Song, X. Yu, S. Yu, Z. Ren, and Y. Qu, “Feature Extraction Processing Method of Medical Image Fusion Based on Neural Network Algorithm,” Complexity, vol. 2021, 2021, doi: 10.1155/2021/7523513.
17. M. Toğaçar, N. Muzoğlu, B. Ergen, B. S. B. Yarman, and A. M. Halefoğlu, “Detection of COVID-19 findings by the local interpretable model-agnostic explanations method of types-based activations extracted from CNNs,” Biomed Signal Process Control, vol. 71, Jan. 2022, doi: 10.1016/j.bspc.2021.103128.
18. E. Akbacak, “An efficient and robust supervised video hashing scheme based on a timedistributed CNN-BLSTM model and principal component analysis,” Multimed Tools Appl, vol. 83, no. 21, pp. 60965–60985, Jun. 2024, doi: 10.1007/s11042-023-17810-8.
19. M. Ghahremani, H. Ghadiri, and M. Hamghalam, “Local features integration for content-based image retrieval based on color, texture, and shape,” Multimed Tools Appl, vol. 80, no. 18, pp. 28245–28263, Jul. 2021, doi: 10.1007/s11042-021-10895-z.
20. I. Issaoui, M. A. Alohali, W. Mtouaa, F. A. Alotaibi, A. Mahmud, and M. Assiri, “Archimedes Optimization Algorithm With Deep Learning Assisted Content-Based Image Retrieval in Healthcare Sector,” IEEE Access, vol. 12, pp. 29768–29777, 2024, doi: 10.1109/ACCESS.2024.3367430.
21. R. S. Bressan, P. H. Bugatti, and P. T. M. Saito, “Breast cancer diagnosis through active learning in content-based image retrieval,” Neurocomputing, vol. 357, pp. 1–10, Sep. 2019, doi: 10.1016/j.neucom.2019.05.041.
22. K. Zhang et al., “Content-based image retrieval with a Convolutional Siamese Neural Network: Distinguishing lung cancer and tuberculosis in CT images,” Comput Biol Med, vol. 140, Jan. 2022, doi: 10.1016/j.compbiomed.2021.105096.
23. N. B. Mohite and A. B. Gonde, “Deep features based medical image retrieval,” Multimed Tools Appl, vol. 81, no. 8, pp. 11379–11392, Mar. 2022, doi: 10.1007/s11042-022-12085-x.
24. Q. Chen, S. Cai, C. Cai, Z. Yu, D. Qian, and S. Xiang, “Colo-SCRL: Self-Supervised Contrastive Representation Learning for Colonoscopic Video Retrieval,” in Proceedings - IEEE International Conference on Multimedia and Expo, IEEE Computer Society, 2023, pp. 1056–1061. doi: 10.1109/ICME55011.2023.00185.
25. R. Yang et al., “EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis,” Jul. 2024, [Online]. Available: http://arxiv.org/abs/2407.11401
26. H. Nosato et al., “Image retrieval method for multiscale objects from optical colonoscopy images,” Int J Biomed Imaging, vol. 2017, 2017, doi: 10.1155/2017/7089213.
27. R. Boukerma, B. Boucheham, and S. Bougueroua, “Optimized Deep Features for Colon Histology Image Retrieval,” in PAIS 2024 - Proceedings: 6th International Conference on Pattern Analysis and Intelligent Systems, Institute of Electrical and Electronics Engineers Inc., 2024. doi: 10.1109/PAIS62114.2024.10541159.
28. S. Mohammad Alizadeh, M. Sadegh Helfroush, and H. Müller, “A novel Siamese deep hashing model for histopathology image retrieval,” Expert Syst Appl, vol. 225, Sep. 2023, doi: 10.1016/j.eswa.2023.120169.
29. A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” Apr. 2017, [Online]. Available: http://arxiv.org/abs/1704.04861
30. X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices,” Dec. 2017, [Online]. Available: http://arxiv.org/abs/1707.01083
31. M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Sep. 2020, [Online]. Available: http://arxiv.org/abs/1905.11946
32. N. Muzoğlu and E. Akbacak, “Ensemble-Based Alzheimer’s Disease Classification Using Features Extracted from Hog Descriptor and Pre-trained Models,” Sakarya University Journal of Computer and Information Sciences, vol. 7, no. 3, pp. 416–426, 2024, doi: 10.35377/saucis…1493368.
33. P. Dhal and C. Azad, “A comprehensive survey on feature selection in the various fields of machine learning,” Applied Intelligence, vol. 52, no. 4, pp. 4543–4581, Mar. 2022, doi: 10.1007/s10489-021-02550-9.
34. X. Cui, Y. Li, J. Fan, and T. Wang, “A novel filter feature selection algorithm based on relief”, doi: 10.1007/s10489-021-02659-x/Published.
35. A. Prakash and V. P. Singh, “Content-based X-ray image retrieval using fusion of local neighboring patterns and deep features for lung disease detection,” Radiol Phys Technol, vol. 18, no. 3, pp. 785–804, Sep. 2025, doi: 10.1007/s12194-025-00932-z.
36. U. Chaudhuri, B. Banerjee, A. Bhattacharya, and M. Datcu, “CMIR-NET : A deep learning based model for cross-modal retrieval in remote sensing,” Pattern Recognit Lett, vol. 131, pp. 456–462, Mar. 2020, doi: 10.1016/j.patrec.2020.02.006.
37. Z. Shao, W. Zhou, L. Zhang, and J. Hou, “Improved color texture descriptors for remote sensing image retrieval”, doi: 10.1117/1.
38. K. Walter, M. J. Gibson, and A. Sowmya, “Self-Supervised Remote Sensing Image Retrieval,” in International Geoscience and Remote Sensing Symposium (IGARSS), Institute of Electrical and Electronics Engineers Inc., Sep. 2020, pp. 1683–1686. doi: 10.1109/IGARSS39084.2020.9323294.
39. X. Lu, L. Zhang, L. Niu, Q. Chen, and J. Wang, “A novel adaptive feature fusion strategy for image retrieval,” Entropy, vol. 23, no. 12, Dec. 2021, doi: 10.3390/e23121670.
40. X. Li, J. Yang, and J. Ma, “Recent developments of content-based image retrieval (CBIR),” Neurocomputing, vol. 452, pp. 675–689, Sep. 2021, doi: 10.1016/j.neucom.2020.07.139.
41. K. N. Sukhia, S. S. Ali, M. M. Riaz, A. Ghafoor, and B. Amin, “Content-Based Image Retrieval Using Angles Across Scales,” IEEE Geoscience and Remote Sensing Letters, vol. 19, 2022, doi: 10.1109/LGRS.2021.3131340.