Research Article
BibTex RIS Cite

Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini

Year 2023, Volume: 13 Issue: 3, 1468 - 1481, 01.09.2023
https://doi.org/10.21597/jist.1249908

Abstract

Orman yangını, ormanda yaşama birliği içinde bulunan canlı ve cansız bütün varlıkları yakarak yok eden, ekonomik ve ekolojik zararları olan bir afettir. Son yıllarda küresel ısınma sebebi ile mevsim normalleri üzerinde seyreden sıcaklıklar ve kuraklıklar orman yangını riskini daha da artırmaktadır. Orman yangınları nedeniyle meydana gelen zararı en aza indirmek için yangınla mücadelede erken uyarı, hızlı ve etkin müdahale çok önemlidir. Makine öğrenmesi yöntemleri ise günümüzde erken uyarı sistemlerinde kullanılmaktadır. Bu çalışmada orman yangınlarıyla mücadele için olası orman yangınını önceden tahmin ederek yangınların kontrol edilmesi ve etkisinin azaltılması hedeflenmiştir. Orman yangını tahmin modeli için veri seti, NASA’nın Oak Ridge Ulusal Laboratuvarı (ORNL) Dağıtılmış Aktif Arşiv Merkezi’nin (DAAC) resmi web sitesinden alınarak geliştirilmiştir. Bu veriler makine öğrenmesi yöntemleriyle işlenerek orman yangını tahmin modeli oluşturulmuştur. Veri setine çeşitli ön işleme adımı uygulayarak sınıflandırma modeline uygun hale getirilmiştir. Öznitelik seçme teknikleri ile veri setinin tümü kullanılmadan en yüksek oranda veri bütünlüğü sağlanarak en az sayıda öznitelik alt kümesi seçilmiştir. Hedef değişkeni bulmada en önemli ve en faydalı öznitelikler seçilerek makine öğrenmesi algoritmalarından Destek Vektör Makinesi, Karar Ağacı, Rasgele Orman, Gradyan Artırma, K-En Yakın Komşu ve Naive Bayes olmak üzere 6 farklı sınıflandırma algoritmaları ile model oluşturulmuştur. Model performansını değerlendirmek için validasyon işlemi ve en iyi parametre seçimi için ise hiperparametre optimizasyonu yapılmıştır. Bu çalışmada kullanılan sınıflandırma algoritmaları arasında validasyon işlemi ile birlikte en başarılı iki algoritmadan Rasgele Orman ile %97 ve Naive Bayes ile %96 doğruluk oranı elde edilmiştir.

References

  • Arif, M., Alghamdi, K. K., Sahel, S. A., Alosaimi, S. O., Alsahaft, M. E., Alharthi, M. A., & Arif, M. (2021). Role of machine learning algorithms in forest fire management: A literature review. J. Robot. Autom, 5, 212-226.
  • Arpaci, A., Malowerschnig, B., Sass, O., & Vacik, H. (2014). Using multi variate data mining techniques for estimating fire susceptibility of Tyrolean forests. Applied Geography, 53, 258-270.
  • Bayat, G., & Yıldız, K. (2022). Comparison of the Machine Learning Methods to Predict Wildfire Areas. Turkish Journal of Science and Technology, 17(2), 241-250.
  • Castelli, M., Vanneschi, L., & Popovič, A. (2015). Predicting burned areas of forest fires: an artificial intelligence approach. Fire ecology, 11(1), 106-118.
  • Caruana, R., & Niculescu-Mizil, A. (2006, June). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning (pp. 161-168).
  • Chen, J., Wang, X., Yu, Y., Yuan, X., Quan, X., & Huang, H. (2022). Improved Prediction of Forest Fire Risk in Central and Northern China by a Time-Decaying Precipitation Model. Forests, 13(3), 480.
  • Coughlan, R., Di Giuseppe, F., Vitolo, C., Barnard, C., Lopez, P., & Drusch, M. (2021). Using machine learning to predict fire‐ignition occurrences from lightning forecasts. Meteorological applications, 28(1), e1973.
  • Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189-1232.
  • Fidanboy, M., Nihat, A., & Okyay, S. (2022). Derin öğrenmeye dayalı orman yangını tahmin modeli geliştirilmesi ve Türkiye yangın risk haritasının oluşturulması. Ormancılık Araştırma Dergisi, 9(2), 206-218.
  • FAO. (2020). Global Forest Resources Assessment 2020 – Key findings. Rome., Erişim adresi: https://www.fao.org/3/ca8753en/ca8753en.pdf (Erişim Tarihi: 16.10.2022).
  • Gislason, P. O., Benediktsson, J. A., & Sveinsson, J. R. (2006). Random forests for land cover classification. Pattern recognition letters, 27(4), 294-300.
  • García, S., Ramírez-Gallego, S., Luengo, J., Benítez, J. M., & Herrera, F. (2016). Big data preprocessing: methods and prospects. Big Data Analytics, 1(1), 1-22.
  • Islam, M.J., Wu, Q.M., Ahmadi, M., & Sid-Ahmed, M.A. (2007). Investigating the Performance of Naive- Bayes Classifiers and K- Nearest Neighbor Classifiers. 2007 International Conference on Convergence Information Technology (ICCIT 2007), 1541-1546.
  • Lorena, A. C., Jacintho, L. F., Siqueira, M. F., De Giovanni, R., Lohmann, L. G., De Carvalho, A. C., & Yamamoto, M. (2011). Comparing machine learning classifiers in potential distribution modelling. Expert Systems with Applications, 38(5), 5268-5275.
  • Liang, H., Zhang, M., & Wang, H. (2019). A neural network model for wildfire scale prediction using meteorological factors. IEEE Access, 7, 176746-176755.
  • Lai, C., Zeng, S., Guo, W., Liu, X., Li, Y., & Liao, B. (2022). Forest Fire Prediction with Imbalanced Data Using a Deep Neural Network Method. Forests, 13(7), 1129.
  • Lin, H., & Ding, H. (2011). Predicting ion channels and their types by the dipeptide mode of pseudo amino acid composition. Journal of theoretical biology, 269(1), 64-69.
  • Moreira, L., Dantas, C., Oliveira, L., Soares, J., & Ogasawara, E. (2018). On evaluating data preprocessing methods for machine learning models for flight delays. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
  • Mimboro, P., Yanuargi, B., Surimbac, R., Kusrini, K., & Khusnawi, K. (2022). Forest Fire Prediction Using K-Mean Clustering and Random Forest Classifier. CSRID Journal, 14(2): 157-165. DOI: http://dx.doi.org/10.22303/csrid.14.2.2022.157-165.
  • Mitchell, T. M., & Mitchell, T. M. (1997). Machine learning (Vol. 1, No. 9). New York: McGraw-hill.
  • Niranjan, T., Swetha, D., Charitha, V., & Stephen, A. J. (2019). Predicting Burned Area Of Forest Fires. IRJCS:: International Research Journal of Computer Science, 6, 132-136.
  • OGM. (2021). Orman Genel Müdürlüğü, Ormancılık İstatistikleri. Erişim adresi: https://www.ogm.gov.tr/tr/e-kutuphane/resmi-istatistikler (Erişim Tarihi: 18.10.2022).
  • Pang, Y., Li, Y., Feng, Z., Feng, Z., Zhao, Z., Chen, S., & Zhang, H. (2022). Forest Fire Occurrence Prediction in China Based on Machine Learning Methods. Remote Sensing, 14(21), 5546.
  • Preeti, T., Kanakaraddi, S., Beelagi, A., Malagi, S.,& Sudi, A. (2021). Forest Fire Prediction Using Machine Learning Techniques, 2021 International Conference on Intelligent Technologies (CONIT), pp. 1-6, doi: 10.1109/CONIT51480.2021.9498448.
  • Qiu, J., Wang, H., Lu, J., Zhang, B., & Du, K. L. (2012). Neural network implementations for PCA and its extensions. International Scholarly Research Notices, 2012.
  • Rakshit, P., Sarkar, S., Khan, S., Saha, P., Bhattacharyya, S., Dey, N., Islam, S., & Pal, S., (2021). Prediction of Forest Fire Using Machine Learning Algorithms: The Search for the Better Algorithm, 2021 6th International Conference on Innovative Technology in Intelligent System and Industrial Applications (CITISIA), pp. 1-6, doi: 10.1109/CITISIA53721.2021.9719887.
  • Ray, S. (2019). A quick review of machine learning algorithms. In 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon) (pp. 35-39). IEEE. doi: 10.1109/COMITCon.2019.8862451.
  • Rish, I. (2001). An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence (Vol. 3, No. 22, pp. 41-46).
  • Silva, I. D. B., Valle, M. E., Barros, L. C., & Meyer, J. F. C. (2020). A wildfire warning system applied to the state of Acre in the Brazilian Amazon. Applied Soft Computing, 89, 106075.
  • Sevinc, V., Kucuk, O., & Goltas, M. (2020). A Bayesian network model for prediction and analysis of possible forest fire causes. Forest Ecology and Management, 457, 117723.
  • Shao, Y., Feng, Z., Sun, L., Yang, X., Li, Y., Xu, B., & Chen, Y. (2022). Mapping China’s Forest Fire Risks with Machine Learning. Forests, 13(6), 856. https://doi.org/ 10.3390/f13060856.
  • Spoorthy, M. R., & Kumar, H. 2022. Detection of Forest Fire Areas using Machine Learning. Communication and Technology (IJARSCT), 2(2): DOI: 10.48175/IJARSCT-5623.
  • Singh, A., Thakur, N., & Sharma, A. (2016). A review of supervised machine learning algorithms. In 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom) (pp. 1310-1315). Ieee.
  • Uddin, S., Khan, A., Hossain, M. E., & Moni, M. A. (2019). Comparing different supervised machine learning algorithms for disease prediction. BMC medical informatics and decision making, 19(1), 1-16.
  • Walker, X.J., Baltzer, J.L, Bourgeau-Chavez, L.L., Day, N.J., De groot W.J., Dieleman, C., Hoy, E.E, Johnstone, J.F., Kane, E.S., Parisien, M.A., Potter, S., Rogers, B.M., Turetsky, M.R., Veraverbeke, S., Whitman, E., & Mack, M.C. (2020). ABoVE: Synthesis of Burned and Unburned Forest Site Data, AK and Canada, 1983-2016. ORNL DAAC, Oak Ridge, Tennessee, USA. https://doi.org/10.3334/ORNLDAAC/1744.
  • Xie, Y., & Peng, M. (2019). Forest fire forecasting using ensemble learning approaches. Neural Computing and Applications, 31, 4541-4550. https://doi.org/10.1007/s00521-018-3515-0.
  • Xie, Y., Jiang, B., Gong, E., Li, Y., Zhu, G., Michel, P., & Zaharchuk, G. (2019). Use of gradient boosting machine learning to predict patient outcome in acute ischemic stroke on the basis of imaging, demographic, and clinical information. American Journal of Roentgenology, 212(1), 44-51.
  • Yoon, J. (2021). Forecasting of real GDP growth using machine learning models: Gradient boosting and random forest approach. Computational Economics, 57(1), 247-265.

Forest Fire Prediction with Machine Learning Methods

Year 2023, Volume: 13 Issue: 3, 1468 - 1481, 01.09.2023
https://doi.org/10.21597/jist.1249908

Abstract

Forest fire is a disaster that destroys all living and non-living beings in the unity of life in the forest by burning and has economic and ecological damages. In recent years, temperatures and droughts that have been above the seasonal norms due to global warming have increased the risk of forest fires. In order to minimize the damage caused by forest fires, early warning, fast and effective intervention is very important in firefighting. Machine learning methods are used in early warning systems today. In this study, it is aimed to control and reduce the effects of fires by predicting possible forest fires in order to fight forest fires. The dataset for the wildfire prediction model was developed from the official website of NASA's Oak Ridge National Laboratory (ORNL) Center for Distributed Active Archives (DAAC). A forest fire prediction model was created by processing these data with machine learning methods. The data set was adapted to the classification model by applying various preprocessing steps. With the feature selection techniques, the least number of feature subsets were selected by providing the highest level of data integrity without using the entire data set. By choosing the most important and useful features in finding the target variable, a model was created with 6 different classification algorithms, namely Support Vector Machine, Decision Tree, Random Forest, Gradient Boosting, K-Nearest Neighbor and Naive Bayes. Validation process was performed to evaluate model performance and hyperparameter optimization was performed for best parameter selection. Among the classification algorithms used in this study, an accuracy rate of 97% was obtained with Random Forest and 96% with Naive Bayes, which is one of the two most successful algorithms with the validation process.

References

  • Arif, M., Alghamdi, K. K., Sahel, S. A., Alosaimi, S. O., Alsahaft, M. E., Alharthi, M. A., & Arif, M. (2021). Role of machine learning algorithms in forest fire management: A literature review. J. Robot. Autom, 5, 212-226.
  • Arpaci, A., Malowerschnig, B., Sass, O., & Vacik, H. (2014). Using multi variate data mining techniques for estimating fire susceptibility of Tyrolean forests. Applied Geography, 53, 258-270.
  • Bayat, G., & Yıldız, K. (2022). Comparison of the Machine Learning Methods to Predict Wildfire Areas. Turkish Journal of Science and Technology, 17(2), 241-250.
  • Castelli, M., Vanneschi, L., & Popovič, A. (2015). Predicting burned areas of forest fires: an artificial intelligence approach. Fire ecology, 11(1), 106-118.
  • Caruana, R., & Niculescu-Mizil, A. (2006, June). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning (pp. 161-168).
  • Chen, J., Wang, X., Yu, Y., Yuan, X., Quan, X., & Huang, H. (2022). Improved Prediction of Forest Fire Risk in Central and Northern China by a Time-Decaying Precipitation Model. Forests, 13(3), 480.
  • Coughlan, R., Di Giuseppe, F., Vitolo, C., Barnard, C., Lopez, P., & Drusch, M. (2021). Using machine learning to predict fire‐ignition occurrences from lightning forecasts. Meteorological applications, 28(1), e1973.
  • Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics, 1189-1232.
  • Fidanboy, M., Nihat, A., & Okyay, S. (2022). Derin öğrenmeye dayalı orman yangını tahmin modeli geliştirilmesi ve Türkiye yangın risk haritasının oluşturulması. Ormancılık Araştırma Dergisi, 9(2), 206-218.
  • FAO. (2020). Global Forest Resources Assessment 2020 – Key findings. Rome., Erişim adresi: https://www.fao.org/3/ca8753en/ca8753en.pdf (Erişim Tarihi: 16.10.2022).
  • Gislason, P. O., Benediktsson, J. A., & Sveinsson, J. R. (2006). Random forests for land cover classification. Pattern recognition letters, 27(4), 294-300.
  • García, S., Ramírez-Gallego, S., Luengo, J., Benítez, J. M., & Herrera, F. (2016). Big data preprocessing: methods and prospects. Big Data Analytics, 1(1), 1-22.
  • Islam, M.J., Wu, Q.M., Ahmadi, M., & Sid-Ahmed, M.A. (2007). Investigating the Performance of Naive- Bayes Classifiers and K- Nearest Neighbor Classifiers. 2007 International Conference on Convergence Information Technology (ICCIT 2007), 1541-1546.
  • Lorena, A. C., Jacintho, L. F., Siqueira, M. F., De Giovanni, R., Lohmann, L. G., De Carvalho, A. C., & Yamamoto, M. (2011). Comparing machine learning classifiers in potential distribution modelling. Expert Systems with Applications, 38(5), 5268-5275.
  • Liang, H., Zhang, M., & Wang, H. (2019). A neural network model for wildfire scale prediction using meteorological factors. IEEE Access, 7, 176746-176755.
  • Lai, C., Zeng, S., Guo, W., Liu, X., Li, Y., & Liao, B. (2022). Forest Fire Prediction with Imbalanced Data Using a Deep Neural Network Method. Forests, 13(7), 1129.
  • Lin, H., & Ding, H. (2011). Predicting ion channels and their types by the dipeptide mode of pseudo amino acid composition. Journal of theoretical biology, 269(1), 64-69.
  • Moreira, L., Dantas, C., Oliveira, L., Soares, J., & Ogasawara, E. (2018). On evaluating data preprocessing methods for machine learning models for flight delays. In 2018 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
  • Mimboro, P., Yanuargi, B., Surimbac, R., Kusrini, K., & Khusnawi, K. (2022). Forest Fire Prediction Using K-Mean Clustering and Random Forest Classifier. CSRID Journal, 14(2): 157-165. DOI: http://dx.doi.org/10.22303/csrid.14.2.2022.157-165.
  • Mitchell, T. M., & Mitchell, T. M. (1997). Machine learning (Vol. 1, No. 9). New York: McGraw-hill.
  • Niranjan, T., Swetha, D., Charitha, V., & Stephen, A. J. (2019). Predicting Burned Area Of Forest Fires. IRJCS:: International Research Journal of Computer Science, 6, 132-136.
  • OGM. (2021). Orman Genel Müdürlüğü, Ormancılık İstatistikleri. Erişim adresi: https://www.ogm.gov.tr/tr/e-kutuphane/resmi-istatistikler (Erişim Tarihi: 18.10.2022).
  • Pang, Y., Li, Y., Feng, Z., Feng, Z., Zhao, Z., Chen, S., & Zhang, H. (2022). Forest Fire Occurrence Prediction in China Based on Machine Learning Methods. Remote Sensing, 14(21), 5546.
  • Preeti, T., Kanakaraddi, S., Beelagi, A., Malagi, S.,& Sudi, A. (2021). Forest Fire Prediction Using Machine Learning Techniques, 2021 International Conference on Intelligent Technologies (CONIT), pp. 1-6, doi: 10.1109/CONIT51480.2021.9498448.
  • Qiu, J., Wang, H., Lu, J., Zhang, B., & Du, K. L. (2012). Neural network implementations for PCA and its extensions. International Scholarly Research Notices, 2012.
  • Rakshit, P., Sarkar, S., Khan, S., Saha, P., Bhattacharyya, S., Dey, N., Islam, S., & Pal, S., (2021). Prediction of Forest Fire Using Machine Learning Algorithms: The Search for the Better Algorithm, 2021 6th International Conference on Innovative Technology in Intelligent System and Industrial Applications (CITISIA), pp. 1-6, doi: 10.1109/CITISIA53721.2021.9719887.
  • Ray, S. (2019). A quick review of machine learning algorithms. In 2019 International conference on machine learning, big data, cloud and parallel computing (COMITCon) (pp. 35-39). IEEE. doi: 10.1109/COMITCon.2019.8862451.
  • Rish, I. (2001). An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence (Vol. 3, No. 22, pp. 41-46).
  • Silva, I. D. B., Valle, M. E., Barros, L. C., & Meyer, J. F. C. (2020). A wildfire warning system applied to the state of Acre in the Brazilian Amazon. Applied Soft Computing, 89, 106075.
  • Sevinc, V., Kucuk, O., & Goltas, M. (2020). A Bayesian network model for prediction and analysis of possible forest fire causes. Forest Ecology and Management, 457, 117723.
  • Shao, Y., Feng, Z., Sun, L., Yang, X., Li, Y., Xu, B., & Chen, Y. (2022). Mapping China’s Forest Fire Risks with Machine Learning. Forests, 13(6), 856. https://doi.org/ 10.3390/f13060856.
  • Spoorthy, M. R., & Kumar, H. 2022. Detection of Forest Fire Areas using Machine Learning. Communication and Technology (IJARSCT), 2(2): DOI: 10.48175/IJARSCT-5623.
  • Singh, A., Thakur, N., & Sharma, A. (2016). A review of supervised machine learning algorithms. In 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom) (pp. 1310-1315). Ieee.
  • Uddin, S., Khan, A., Hossain, M. E., & Moni, M. A. (2019). Comparing different supervised machine learning algorithms for disease prediction. BMC medical informatics and decision making, 19(1), 1-16.
  • Walker, X.J., Baltzer, J.L, Bourgeau-Chavez, L.L., Day, N.J., De groot W.J., Dieleman, C., Hoy, E.E, Johnstone, J.F., Kane, E.S., Parisien, M.A., Potter, S., Rogers, B.M., Turetsky, M.R., Veraverbeke, S., Whitman, E., & Mack, M.C. (2020). ABoVE: Synthesis of Burned and Unburned Forest Site Data, AK and Canada, 1983-2016. ORNL DAAC, Oak Ridge, Tennessee, USA. https://doi.org/10.3334/ORNLDAAC/1744.
  • Xie, Y., & Peng, M. (2019). Forest fire forecasting using ensemble learning approaches. Neural Computing and Applications, 31, 4541-4550. https://doi.org/10.1007/s00521-018-3515-0.
  • Xie, Y., Jiang, B., Gong, E., Li, Y., Zhu, G., Michel, P., & Zaharchuk, G. (2019). Use of gradient boosting machine learning to predict patient outcome in acute ischemic stroke on the basis of imaging, demographic, and clinical information. American Journal of Roentgenology, 212(1), 44-51.
  • Yoon, J. (2021). Forecasting of real GDP growth using machine learning models: Gradient boosting and random forest approach. Computational Economics, 57(1), 247-265.
There are 38 citations in total.

Details

Primary Language Turkish
Subjects Computer Software
Journal Section Bilgisayar Mühendisliği / Computer Engineering
Authors

Orhan Yıldırım 0000-0003-3117-1147

Faruk Baturalp Gunay 0000-0001-5472-3608

Mete Yağanoğlu 0000-0003-3045-169X

Early Pub Date August 29, 2023
Publication Date September 1, 2023
Submission Date February 10, 2023
Acceptance Date April 12, 2023
Published in Issue Year 2023 Volume: 13 Issue: 3

Cite

APA Yıldırım, O., Gunay, F. B., & Yağanoğlu, M. (2023). Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini. Journal of the Institute of Science and Technology, 13(3), 1468-1481. https://doi.org/10.21597/jist.1249908
AMA Yıldırım O, Gunay FB, Yağanoğlu M. Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini. J. Inst. Sci. and Tech. September 2023;13(3):1468-1481. doi:10.21597/jist.1249908
Chicago Yıldırım, Orhan, Faruk Baturalp Gunay, and Mete Yağanoğlu. “Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini”. Journal of the Institute of Science and Technology 13, no. 3 (September 2023): 1468-81. https://doi.org/10.21597/jist.1249908.
EndNote Yıldırım O, Gunay FB, Yağanoğlu M (September 1, 2023) Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini. Journal of the Institute of Science and Technology 13 3 1468–1481.
IEEE O. Yıldırım, F. B. Gunay, and M. Yağanoğlu, “Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini”, J. Inst. Sci. and Tech., vol. 13, no. 3, pp. 1468–1481, 2023, doi: 10.21597/jist.1249908.
ISNAD Yıldırım, Orhan et al. “Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini”. Journal of the Institute of Science and Technology 13/3 (September 2023), 1468-1481. https://doi.org/10.21597/jist.1249908.
JAMA Yıldırım O, Gunay FB, Yağanoğlu M. Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini. J. Inst. Sci. and Tech. 2023;13:1468–1481.
MLA Yıldırım, Orhan et al. “Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini”. Journal of the Institute of Science and Technology, vol. 13, no. 3, 2023, pp. 1468-81, doi:10.21597/jist.1249908.
Vancouver Yıldırım O, Gunay FB, Yağanoğlu M. Makine Öğrenmesi Yöntemleriyle Orman Yangını Tahmini. J. Inst. Sci. and Tech. 2023;13(3):1468-81.