Araştırma Makalesi
BibTex RIS Kaynak Göster

A Comparative Study of Breast Mass Detection Using YOLOv8 Deep Learning Model in Various Data Scenarios on Multi-View Digital Mammograms

Yıl 2023, , 1212 - 1225, 28.12.2023
https://doi.org/10.17798/bitlisfen.1364332

Öz

Breast cancer is one of the most common types of cancer among women worldwide. It typically begins with abnormal cell growth in the breast glands or milk ducts and can spread to other tissues. Many breast cancer cases start with the presence of a mass and should be carefully examined. Masses can be monitored using X-ray-based digital mammography images, including right craniocaudal, left craniocaudal, right mediolateral oblique, and left mediolateral oblique views. In this study, automatic mass detection and localization were performed on mammography images taken from the full-field digital mammography VinDr-Mammo dataset using the YOLOv8 deep learning model. Three different scenarios were tested: raw data, data with preprocessing to crop breast regions, and data with only mass regions cropped to a 1.2x ratio. The data were divided into 80% for training and 10% each for validation and testing. The results were evaluated using performance metrics such as precision, recall, F1-score, mAP, and training graphs. At the end of the study, it is demonstrated that the YOLOv8 deep learning model provides successful results in mass detection and localization, indicating its potential use as a computer-based decision support system.

Etik Beyan

The authors declare that there are no ethical violations involved.

Destekleyen Kurum

Pamukkale University the Scientific Research Coordination Unit

Proje Numarası

2023LÖKAP007

Teşekkür

We would like to thank Pamukkale University the Scientific Research Coordination Unit for supporting this study.

Kaynakça

  • [1] M. Arnold et al., "Current and future burden of breast cancer: Global statistics for 2020 and 2040," The Breast, vol. 66, pp. 15-23, 2022.
  • [2] L. Wilkinson and T. Gathani, "Understanding breast cancer as a global health concern," The British Journal of Radiology, vol. 95, no. 1130, p. 20211033, 2022.
  • [3] R. Ali, A. Sultan, R. Ishrat, S. Haque, N. J. Khan, and M. A. Prieto, "Identification of New Key Genes and Their Association with Breast Cancer Occurrence and Poor Survival Using In Silico and In Vitro Methods," Biomedicines, vol. 11, no. 5, p. 1271, 2023.
  • [4] M. Buda et al., "Detection of masses and architectural distortions in digital breast tomosynthesis: a publicly available dataset of 5,060 patients and a deep learning model," arXiv preprint arXiv:2011.07995, 2020.
  • [5] V. Barros et al., "Virtual biopsy by using artificial intelligence–based multimodal modeling of binational mammography data," Radiology, vol. 306, no. 3, p. e220027, 2022.
  • [6] Y. Yan, P.-H. Conze, G. Quellec, M. Lamard, B. Cochener, and G. Coatrieux, "Two-stage multi-scale breast mass segmentation for full mammogram analysis without user intervention," Biocybernetics and Biomedical Engineering, vol. 41, no. 2, pp. 746-757, 2021.
  • [7] I. Domingues and J. Cardoso, "Mass detection on mammogram images: a first assessment of deep learning techniques," in 19th Portuguese Conference on Pattern Recognition (RECPAD), 2013.
  • [8] R. Gayathri and K. Kasirajan, "Yolo Algorithm Based Breast Masses Detection And Classification Technique For Digital Mammograms," Latin American Journal of Pharmacy, vol. 42, no. 3, pp. 374-381, 2023.
  • [9] A. Mohiyuddin et al., "Breast tumor detection and classification in mammogram images using modified YOLOv5 network," Computational and mathematical methods in medicine, vol. 2022, pp. 1-16, 2022.
  • [10] Z. Cao et al., "Deep learning based mass detection in mammograms," in 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2019: IEEE, pp. 1-5.
  • [11] M. A. Al-masni et al., "Detection and classification of the breast abnormalities in digital mammograms via regional convolutional neural network," in 2017 39th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2017: IEEE, pp. 1230-1233.
  • [12] G. Ayana et al., "Vision-Transformer-Based Transfer Learning for Mammogram Classification," Diagnostics, vol. 13, no. 2, p. 178, 2023.
  • [13] G. H. Aly, M. Marey, S. A. El-Sayed, and M. F. Tolba, "YOLO based breast masses detection and classification in full-field digital mammograms," Computer methods and programs in biomedicine, vol. 200, p. 105823, 2021.
  • [14] Y. Cui, Y. Li, D. Xing, T. Bai, J. Dong, and J. Zhu, "Improving the prediction of benign or malignant breast masses using a combination of image biomarkers and clinical parameters," Frontiers in Oncology, vol. 11, p. 629321, 2021.
  • [15] H. Zhang, Z. Xu, D. Yao, S. Zhang, J. Chen, and T. Lukasiewicz, "Multi-Head Feature Pyramid Networks for Breast Mass Detection," in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023: IEEE, pp. 1-5.
  • [16] Y. Yan, P.-H. Conze, M. Lamard, G. Quellec, B. Cochener, and G. Coatrieux, "Multi-tasking siamese networks for breast mass detection using dual-view mammogram matching," in Machine Learning in Medical Imaging: 11th International Workshop, MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11, 2020: Springer, pp. 312-321.
  • [17] M. A. Al-Masni et al., "Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system," Computer methods and programs in biomedicine, vol. 157, pp. 85-94, 2018.
  • [18] Y. Su, Q. Liu, W. Xie, and P. Hu, "YOLO-LOGO: A transformer-based YOLO segmentation model for breast mass detection and segmentation in digital mammograms," Computer Methods and Programs in Biomedicine, vol. 221, p. 106903, 2022.
  • [19] B. Abhisheka, S. K. Biswas, and B. Purkayastha, "A comprehensive review on breast cancer detection, classification and segmentation using deep learning," Archives of Computational Methods in Engineering, pp. 1-30, 2023.
  • [20] I. C. Moreira, I. Amaral, I. Domingues, A. Cardoso, M. J. Cardoso, and J. S. Cardoso, "Inbreast: toward a full-field digital mammographic database," Academic radiology, vol. 19, no. 2, pp. 236-248, 2012.
  • [21] S. Singh and K. Bovis, "An evaluation of contrast enhancement techniques for mammographic breast masses," IEEE Transactions on Information Technology in Biomedicine, vol. 9, no. 1, pp. 109-119, 2005.
  • [22] M. P. Sampat, A. C. Bovik, G. J. Whitman, and M. K. Markey, "A model‐based framework for the detection of spiculated masses on mammography a," Medical physics, vol. 35, no. 5, pp. 2110-2123, 2008.
  • [23] E. Song et al., "Hybrid segmentation of mass in mammograms using template matching and dynamic programming," Academic radiology, vol. 17, no. 11, pp. 1414-1424, 2010.
  • [24] H. T. Nguyen et al., "VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," Scientific Data, vol. 10, no. 1, p. 277, 2023.
  • [25] M. Ü. Öziç, M. Barstuğan, and A. Özdamar, "An autonomous system design for mold loading on press brake machines using a camera platform, deep learning, and image processing," Journal of Mechanical Science and Technology, pp. 1-9, 2023.
  • [26] F. Yuce, M. Ü. Öziç, and M. Tassoker, "Detection of pulpal calcifications on bite-wing radiographs using deep learning," Clinical Oral Investigations, vol. 27, no. 6, pp. 2679-2689, 2023.
  • [27] J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7263-7271.
  • [28] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018. [29] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
  • [30] G. Jocher et al., "ultralytics/yolov5: v3. 0," Zenodo, 2020.
  • [31] J. Terven and D. Cordova-Esparza, "A comprehensive review of YOLO: From YOLOv1 to YOLOv8 and beyond," arXiv preprint arXiv:2304.00501, 2023.
  • [32] D. G. Petrini, C. Shimizu, R. A. Roela, G. V. Valente, M. A. A. K. Folgueira, and H. Y. Kim, "Breast cancer diagnosis in two-view mammography using end-to-end trained efficientnet-based convolutional network," Ieee access, vol. 10, pp. 77723-77731, 2022.
  • [33] D. Anyfantis, A. Koutras, G. Apostolopoulos, and I. Christoyianni, "Breast Density Transformations Using CycleGANs for Revealing Undetected Findings in Mammograms," Signals, vol. 4, no. 2, pp. 421-438, 2023.
  • [34] E. Mahoro and M. A. Akhloufi, "Breast masses detection on mammograms using recent one-shot deep object detectors," in 2023 5th International Conference on Bio-engineering for Smart Technologies (BioSMART), 2023: IEEE, pp. 1-4.
  • [35] H. T. Nguyen, S. B. Tran, D. B. Nguyen, H. H. Pham, and H. Q. Nguyen, "A novel multi-view deep learning approach for BI-RADS and density assessment of mammograms," in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022: IEEE, pp. 2144-2148.
  • [36] S. B. Tran, H. T. Nguyen, C. Phan, H. H. Pham, and H. Q. Nguyen, "A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms," arXiv preprint arXiv:2203.10609, 2022.
  • [37] S. Magny, R. Shikhman, and A. Keppke, "Breast Imaging Reporting and Data System. 2022 Aug 29," StatPearls [Internet]. StatPearls Publishing (Treasure Island, FL), 2022.
  • [38] Y. Liu et al., "High-temporal resolution DCE-MRI improves assessment of intra-and peri-breast lesions categorized as BI-RADS 4," BMC Medical Imaging, vol. 23, no. 1, p. 58, 2023.
  • [39] A. Vourtsis and W. A. Berg, "Breast density implications and supplemental screening," European radiology, vol. 29, pp. 1762-1777, 2019.
  • [40] N. Otsu, "A threshold selection method from gray-level histograms," IEEE transactions on systems, man, and cybernetics, vol. 9, no. 1, pp. 62-66, 1979.
  • [41] R. Walsh and M. Tardy, "A Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer," Diagnostics, vol. 13, no. 1, p. 67, 2022.
  • [42] S. B. Tran, H. T. Nguyen, C. Phan, H. Q. Nguyen, and H. H. Pham, "A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms," in 2023 IEEE Statistical Signal Processing Workshop (SSP), 2023: IEEE, pp. 681-685.
  • [43] V. Sampaio and F. R. Cordeiro, "A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning," in Anais do XIX Encontro Nacional de Inteligência Artificial e Computacional, 2022: SBC, pp. 437-448.
  • [44] H. N. Huynh, A. T. Tran, and T. N. Tran, "Region-of-Interest Optimization for Deep-Learning-Based Breast Cancer Detection in Mammograms," Applied Sciences, vol. 13, no. 12, p. 6894, 2023.
  • [45] B. Gašparović, G. Mauša, J. Rukavina, and J. Lerga, "Evaluating YOLOV5, YOLOV6, YOLOV7, and YOLOV8 in Underwater Environment: Is There Real Improvement?," in 2023 8th International Conference on Smart and Sustainable Technologies (SpliTech), 2023: IEEE, pp. 1-4.
  • [46] F. N. Ortataş and M. Kaya, "Performance Evaluation of YOLOv5, YOLOv7, and YOLOv8 Models in Traffic Sign Detection," in 2023 8th International Conference on Computer Science and Engineering (UBMK), 2023: IEEE, pp. 151-156.
Yıl 2023, , 1212 - 1225, 28.12.2023
https://doi.org/10.17798/bitlisfen.1364332

Öz

Proje Numarası

2023LÖKAP007

Kaynakça

  • [1] M. Arnold et al., "Current and future burden of breast cancer: Global statistics for 2020 and 2040," The Breast, vol. 66, pp. 15-23, 2022.
  • [2] L. Wilkinson and T. Gathani, "Understanding breast cancer as a global health concern," The British Journal of Radiology, vol. 95, no. 1130, p. 20211033, 2022.
  • [3] R. Ali, A. Sultan, R. Ishrat, S. Haque, N. J. Khan, and M. A. Prieto, "Identification of New Key Genes and Their Association with Breast Cancer Occurrence and Poor Survival Using In Silico and In Vitro Methods," Biomedicines, vol. 11, no. 5, p. 1271, 2023.
  • [4] M. Buda et al., "Detection of masses and architectural distortions in digital breast tomosynthesis: a publicly available dataset of 5,060 patients and a deep learning model," arXiv preprint arXiv:2011.07995, 2020.
  • [5] V. Barros et al., "Virtual biopsy by using artificial intelligence–based multimodal modeling of binational mammography data," Radiology, vol. 306, no. 3, p. e220027, 2022.
  • [6] Y. Yan, P.-H. Conze, G. Quellec, M. Lamard, B. Cochener, and G. Coatrieux, "Two-stage multi-scale breast mass segmentation for full mammogram analysis without user intervention," Biocybernetics and Biomedical Engineering, vol. 41, no. 2, pp. 746-757, 2021.
  • [7] I. Domingues and J. Cardoso, "Mass detection on mammogram images: a first assessment of deep learning techniques," in 19th Portuguese Conference on Pattern Recognition (RECPAD), 2013.
  • [8] R. Gayathri and K. Kasirajan, "Yolo Algorithm Based Breast Masses Detection And Classification Technique For Digital Mammograms," Latin American Journal of Pharmacy, vol. 42, no. 3, pp. 374-381, 2023.
  • [9] A. Mohiyuddin et al., "Breast tumor detection and classification in mammogram images using modified YOLOv5 network," Computational and mathematical methods in medicine, vol. 2022, pp. 1-16, 2022.
  • [10] Z. Cao et al., "Deep learning based mass detection in mammograms," in 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2019: IEEE, pp. 1-5.
  • [11] M. A. Al-masni et al., "Detection and classification of the breast abnormalities in digital mammograms via regional convolutional neural network," in 2017 39th annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2017: IEEE, pp. 1230-1233.
  • [12] G. Ayana et al., "Vision-Transformer-Based Transfer Learning for Mammogram Classification," Diagnostics, vol. 13, no. 2, p. 178, 2023.
  • [13] G. H. Aly, M. Marey, S. A. El-Sayed, and M. F. Tolba, "YOLO based breast masses detection and classification in full-field digital mammograms," Computer methods and programs in biomedicine, vol. 200, p. 105823, 2021.
  • [14] Y. Cui, Y. Li, D. Xing, T. Bai, J. Dong, and J. Zhu, "Improving the prediction of benign or malignant breast masses using a combination of image biomarkers and clinical parameters," Frontiers in Oncology, vol. 11, p. 629321, 2021.
  • [15] H. Zhang, Z. Xu, D. Yao, S. Zhang, J. Chen, and T. Lukasiewicz, "Multi-Head Feature Pyramid Networks for Breast Mass Detection," in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023: IEEE, pp. 1-5.
  • [16] Y. Yan, P.-H. Conze, M. Lamard, G. Quellec, B. Cochener, and G. Coatrieux, "Multi-tasking siamese networks for breast mass detection using dual-view mammogram matching," in Machine Learning in Medical Imaging: 11th International Workshop, MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11, 2020: Springer, pp. 312-321.
  • [17] M. A. Al-Masni et al., "Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system," Computer methods and programs in biomedicine, vol. 157, pp. 85-94, 2018.
  • [18] Y. Su, Q. Liu, W. Xie, and P. Hu, "YOLO-LOGO: A transformer-based YOLO segmentation model for breast mass detection and segmentation in digital mammograms," Computer Methods and Programs in Biomedicine, vol. 221, p. 106903, 2022.
  • [19] B. Abhisheka, S. K. Biswas, and B. Purkayastha, "A comprehensive review on breast cancer detection, classification and segmentation using deep learning," Archives of Computational Methods in Engineering, pp. 1-30, 2023.
  • [20] I. C. Moreira, I. Amaral, I. Domingues, A. Cardoso, M. J. Cardoso, and J. S. Cardoso, "Inbreast: toward a full-field digital mammographic database," Academic radiology, vol. 19, no. 2, pp. 236-248, 2012.
  • [21] S. Singh and K. Bovis, "An evaluation of contrast enhancement techniques for mammographic breast masses," IEEE Transactions on Information Technology in Biomedicine, vol. 9, no. 1, pp. 109-119, 2005.
  • [22] M. P. Sampat, A. C. Bovik, G. J. Whitman, and M. K. Markey, "A model‐based framework for the detection of spiculated masses on mammography a," Medical physics, vol. 35, no. 5, pp. 2110-2123, 2008.
  • [23] E. Song et al., "Hybrid segmentation of mass in mammograms using template matching and dynamic programming," Academic radiology, vol. 17, no. 11, pp. 1414-1424, 2010.
  • [24] H. T. Nguyen et al., "VinDr-Mammo: A large-scale benchmark dataset for computer-aided diagnosis in full-field digital mammography," Scientific Data, vol. 10, no. 1, p. 277, 2023.
  • [25] M. Ü. Öziç, M. Barstuğan, and A. Özdamar, "An autonomous system design for mold loading on press brake machines using a camera platform, deep learning, and image processing," Journal of Mechanical Science and Technology, pp. 1-9, 2023.
  • [26] F. Yuce, M. Ü. Öziç, and M. Tassoker, "Detection of pulpal calcifications on bite-wing radiographs using deep learning," Clinical Oral Investigations, vol. 27, no. 6, pp. 2679-2689, 2023.
  • [27] J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7263-7271.
  • [28] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018. [29] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
  • [30] G. Jocher et al., "ultralytics/yolov5: v3. 0," Zenodo, 2020.
  • [31] J. Terven and D. Cordova-Esparza, "A comprehensive review of YOLO: From YOLOv1 to YOLOv8 and beyond," arXiv preprint arXiv:2304.00501, 2023.
  • [32] D. G. Petrini, C. Shimizu, R. A. Roela, G. V. Valente, M. A. A. K. Folgueira, and H. Y. Kim, "Breast cancer diagnosis in two-view mammography using end-to-end trained efficientnet-based convolutional network," Ieee access, vol. 10, pp. 77723-77731, 2022.
  • [33] D. Anyfantis, A. Koutras, G. Apostolopoulos, and I. Christoyianni, "Breast Density Transformations Using CycleGANs for Revealing Undetected Findings in Mammograms," Signals, vol. 4, no. 2, pp. 421-438, 2023.
  • [34] E. Mahoro and M. A. Akhloufi, "Breast masses detection on mammograms using recent one-shot deep object detectors," in 2023 5th International Conference on Bio-engineering for Smart Technologies (BioSMART), 2023: IEEE, pp. 1-4.
  • [35] H. T. Nguyen, S. B. Tran, D. B. Nguyen, H. H. Pham, and H. Q. Nguyen, "A novel multi-view deep learning approach for BI-RADS and density assessment of mammograms," in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022: IEEE, pp. 2144-2148.
  • [36] S. B. Tran, H. T. Nguyen, C. Phan, H. H. Pham, and H. Q. Nguyen, "A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms," arXiv preprint arXiv:2203.10609, 2022.
  • [37] S. Magny, R. Shikhman, and A. Keppke, "Breast Imaging Reporting and Data System. 2022 Aug 29," StatPearls [Internet]. StatPearls Publishing (Treasure Island, FL), 2022.
  • [38] Y. Liu et al., "High-temporal resolution DCE-MRI improves assessment of intra-and peri-breast lesions categorized as BI-RADS 4," BMC Medical Imaging, vol. 23, no. 1, p. 58, 2023.
  • [39] A. Vourtsis and W. A. Berg, "Breast density implications and supplemental screening," European radiology, vol. 29, pp. 1762-1777, 2019.
  • [40] N. Otsu, "A threshold selection method from gray-level histograms," IEEE transactions on systems, man, and cybernetics, vol. 9, no. 1, pp. 62-66, 1979.
  • [41] R. Walsh and M. Tardy, "A Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer," Diagnostics, vol. 13, no. 1, p. 67, 2022.
  • [42] S. B. Tran, H. T. Nguyen, C. Phan, H. Q. Nguyen, and H. H. Pham, "A Novel Transparency Strategy-based Data Augmentation Approach for BI-RADS Classification of Mammograms," in 2023 IEEE Statistical Signal Processing Workshop (SSP), 2023: IEEE, pp. 681-685.
  • [43] V. Sampaio and F. R. Cordeiro, "A Study on Class Activation Map Methods to Detect Masses in Mammography Images using Weakly Supervised Learning," in Anais do XIX Encontro Nacional de Inteligência Artificial e Computacional, 2022: SBC, pp. 437-448.
  • [44] H. N. Huynh, A. T. Tran, and T. N. Tran, "Region-of-Interest Optimization for Deep-Learning-Based Breast Cancer Detection in Mammograms," Applied Sciences, vol. 13, no. 12, p. 6894, 2023.
  • [45] B. Gašparović, G. Mauša, J. Rukavina, and J. Lerga, "Evaluating YOLOV5, YOLOV6, YOLOV7, and YOLOV8 in Underwater Environment: Is There Real Improvement?," in 2023 8th International Conference on Smart and Sustainable Technologies (SpliTech), 2023: IEEE, pp. 1-4.
  • [46] F. N. Ortataş and M. Kaya, "Performance Evaluation of YOLOv5, YOLOv7, and YOLOv8 Models in Traffic Sign Detection," in 2023 8th International Conference on Computer Science and Engineering (UBMK), 2023: IEEE, pp. 151-156.
Toplam 45 adet kaynakça vardır.

Ayrıntılar

Birincil Dil İngilizce
Konular Yapay Zeka (Diğer), Biyomedikal Tanı, Biyomedikal Mühendisliği (Diğer)
Bölüm Araştırma Makalesi
Yazarlar

Muhammet Üsame Öziç 0000-0002-3037-2687

Ayşe Sidenur Yılmaz 0009-0005-9014-1038

Halil İbrahim Sandıraz 0009-0002-0904-5806

Baıhaqı Hılmı Estanto 0009-0005-8468-2970

Proje Numarası 2023LÖKAP007
Erken Görünüm Tarihi 25 Aralık 2023
Yayımlanma Tarihi 28 Aralık 2023
Gönderilme Tarihi 21 Eylül 2023
Kabul Tarihi 14 Kasım 2023
Yayımlandığı Sayı Yıl 2023

Kaynak Göster

IEEE M. Ü. Öziç, A. S. Yılmaz, H. İ. Sandıraz, ve B. H. Estanto, “A Comparative Study of Breast Mass Detection Using YOLOv8 Deep Learning Model in Various Data Scenarios on Multi-View Digital Mammograms”, Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, c. 12, sy. 4, ss. 1212–1225, 2023, doi: 10.17798/bitlisfen.1364332.



Bitlis Eren Üniversitesi
Fen Bilimleri Dergisi Editörlüğü

Bitlis Eren Üniversitesi Lisansüstü Eğitim Enstitüsü        
Beş Minare Mah. Ahmet Eren Bulvarı, Merkez Kampüs, 13000 BİTLİS        
E-posta: fbe@beu.edu.tr