Araştırma Makalesi
BibTex RIS Kaynak Göster

Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması

Yıl 2022, Cilt: 11 Sayı: 4, 1449 - 1463, 26.10.2022
https://doi.org/10.33206/mjss.1026666

Öz

Bu çalışmada objektif kriter ağırlıklandırma yöntemlerinden Entropy, Criteria Importance Through Intercriteria Correlation (CRITIC), Eşit ağırlık, MEthod based on the Removal Effects of Criteria (MEREC), Standart sapma ve Integrated Data Oriented Weighting System (IDDWS) yaklaşımları kullanılarak, bahsi geçen altı farklı tekniğin Simple Additive Weighting (SAW) Çok Kriterli Karar Verme (ÇKKV) yöntem sonuçları üzerindeki etkisinin incelenmesi amaçlanmıştır. Gerçek bir hayat uygulamasının ele alındığı bu çalışmada, Zorlu Enerji firmasının 2016-2020 dönem aralığı finansal performansının çalışmada önerilen model yardımıyla ölçülmesi amaçlanmıştır. Çalışma sonunda, Entropy, CRITIC, MEREC, Eşit ağırlık, Standart Sapma, IDDWS temelinde SAW yöntemiyle elde edilen sıralama sonuçlarının birbiriyle aynı olduğu tespit edilmiştir. Veri setine yeni alternatifler eklenerek elde edilen sonuçlara göre ise kriter ağırlıklarının sıralama sonuçları üzerinde farklı etkiye sahip olduğu tespit edilmiştir. Bilindiği kadarıyla çalışmada kullanılan teknikler daha önce bir arada kullanılmamıştır. Bu açıdan çalışmanın literatürdeki boşluğu dolduracağı ve literatüre katkı sunacağı düşünülmektedir.

Kaynakça

  • Adriyendi, S. (2015). Multi attribute decision making using simple additive weighting and weighted product in food choice. Information Engineering and Electronic Business, 6: 8-14.
  • Akbulut, O. Y. ve Şenol, Z. (2021). Bütünleşik SD ve PROMETHEE ÇKKV Yöntemleri ile Portföy Optimizasyonu: BİST Gıda, İçecek ve Tütün Sektöründe Ampirik Bir Uygulama. Muhasebe ve Finansman Dergisi, (92), 161-182.
  • Akhtar, S., Javed, B., Maryam, A. ve Sadia, H. (2012). Relationship Between Financial Leverage and Financial Performance: Evidence From Fuel & Energy Sector of Pakistan. European Journal of Business and Management, 4(11),7-17.
  • Alemi-Ardakani, M., Milani, A. S., Yannacopoulos, S., ve Shokouhi, G. (2016). On the effect of subjective, objective and combinative weighting in multiple criteria decision making: A case study on impact optimization of composites. Expert Systems with Applications, 46, 426-438.
  • Anupama, G., ve Kesava Rao, V. V. S. (2020). Some Objective Methods for Determining Relative Importance of Financial Ratios. International Journal of Management, 10(4), 2019.
  • Cheng, L. H., Cao, D. Q., ve Guo, H. M. (2020). Analysis of coal mine occupational disease hazard evaluation index based on AHP-DEMATEL. Archives of Environmental & Occupational Health, 1-13.
  • Churchman C.W. ve Ackoff R.L. (1954). An approximatemeasure of value, Journal of Operations Research Societyof America. 2(1), 172-187.
  • Clausius, R. (1865). Ueber Verschiedene für die Anwendung Bequeme Formen der Hauptgleichungen der Mechanischen Wärmetheorie: Vorgetragen in der Naturforsch. Gesellschaft den 24.
  • Diakoulaki D., Mavrotas G. ve Papayannakis L. (1995). Determining objective Weights in Multiple Criteria Problems: The Critic Method. Computers & Operations Research, 22,763–770.
  • Du, Y., Zheng, Y., Wu, G., ve Tang, Y. (2020). Decision-making method of heavy-duty machine tool remanufacturing based on AHP-entropy weight and extension theory. Journal of Cleaner Production, 252, 119607.
  • Halkos, G. E. ve Tzeremes, N. G. (2012). Analyzing the Greek Renewable Energy Sector: A Data Envelopment Analysis Approach. Renewable and Sustainable Energy Reviews, 16(5), 2884-2893.
  • Inam, A. ve Mir, G. M. (2014). The İmpact of Financial Leverage on Firm Performance in Fuel and Energy Sector, Pakistan. European Journal of Business and Management, 6(37), 339-347.
  • Jahan, A., Mustapha, F., Sapuan, S.M., Ismail, M. Y., Bahraminasab, M. (2012). A framework for weighting of criteria in ranking stage of material selection process. The International Journal of Advanced Manufacturing Technology, 58, 411–420.
  • Jain, V. and Raj, T. (2013). Evaluation of flexibility in FMS using SAW and WPM. Decision Science Letters, 2(4): 223- 230.
  • Keshavarz-Ghorabaee, M., Amiri, M., Zavadskas, E. K., Turskis, Z., ve Antucheviciene, J. (2021). Determination of Objective Weights Using a New Method Based on the Removal Effects of Criteria (MEREC). Symmetry, 13(4), 525.
  • Li, H., Wang, W., Fan, L., Li, Q., ve Chen, X. (2020). A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL, entropy weighting and later defuzzification VIKOR. Applied Soft Computing, 91, 106207.
  • Maheshwari, N.; Choudhary, J.; Rath, A.; Shinde, D.; Kalita, K. (2021). Finite Element Analysis and Multi-criteria Decision-Making (MCDM)-Based Optimal Design Parameter Selection of Solid Ventilated Brake Disc. J. Inst. Eng. (India) Ser. C.
  • Memariani, A., Amini, B. and Alinezhad, A. (2009). Sensitivity analysis of Simple Additive Weighting method (SAW): The results of change in the weight of one attribute on the 77 final ranking of alternatives. Journal of Industrial Engineering, 2(4): 13-18.
  • Mukhametzyanov, I. (2021). Specific character of objective methods for determining weights of criteria in MCDM problems: Entropy, CRITIC and SD. Decision Making: Applications in Management and Engineering, 4(2), 76-105.
  • Pala, O. (2021). IDOCRIW ve MARCOS Temelli BIST Ulaştırma İşletmelerinin Finansal Performans Analizi. Kafkas Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 12(23), 263-294.
  • Paradowski, B., Shekhovtsov, A., Bączkiewicz, A., Kizielewicz, B., ve Sałabun, W. (2021). Similarity Analysis of Methods for Objective Determination of Weights in Multi-Criteria Decision Support Systems. Symmetry, 13(10), 1874.
  • Peng, X., Krishankumar, R., ve Ravichandran, K. S. (2021). A novel interval-valued fuzzy soft decision-making method based on CoCoSo and CRITIC for intelligent healthcare management evaluation. Soft Computing, 25(6), 4213-4241.
  • Piasecki, M., ve Kostyrko, K. (2020). Development of weighting scheme for indoor air quality model using a multi- attribute decision making method. Energies, 13(12), 3120.
  • Sahin, M. (2020). Hybrid Multi-attribute Decision Method for Material Selection. Int. J. Pure Appl. Sci., 6, 107–117.
  • Sahin, M. (2021). Location selection by multi-criteria decision-making methods based on objective and subjective weightings. Knowledge and Information Systems, 63(8), 1991-2021.
  • Sałabun, W.; Atróbski, J.; Shekhovtsov, A. (2020). Are MCDA methods benchmarkable? A comparative study of TOPSIS, VIKOR, COPRAS, and PROMETHEE II Methods. Symmetry, 12, 1549.
  • Shaverdi, M., Heshmati, M. R. ve Ramezani, I. (2014). Application of Fuzzy AHP Approach for Financial Performance Evaluation of Iranian Petrochemical Sector. Procedia Computer Science, 31, 995-1004.
  • Sisay, G., Gebre, S. L., ve Getahun, K. (2021). GIS-based potential landfill site selection using MCDM-AHP modeling of Gondar Town, Ethiopia. African Geographical Review, 40(2), 105-124.
  • Sivakumar, V. L., Krishnappa, R. R., ve Nallanathel, M. (2021). Drought vulnerability assessment and mapping using Multi-Criteria decision making (MCDM) and application of Analytic Hierarchy process (AHP) for Namakkal District, Tamilnadu, India. Materials Today: Proceedings, 43, 1592-1599.
  • Sotoudeh-Anvari, A., Sadjadi, S., Molana, S., ve Sadi-Nezhad, S. (2018). A new MCDM-based approach using BWM and SAW for optimal search model. Decision Science Letters, 7(4), 395-404.
  • Souissi, D., Zouhri, L., Hammami, S., Msaddek, M. H., Zghibi, A., ve Dlala, M. (2020). GIS-based MCDM–AHP modeling for flood susceptibility mapping of arid areas, southeastern Tunisia. Geocarto International, 35(9), 991-1017.
  • Sueyoshi, T. (2005). Financial Ratio Analysis of the Electric Power Industry. Asia-Pacific Journal of Operational Research, 22(03), 349-376.
  • Torkayesh, A. E., Ecer, F., Pamucar, D., ve Karamaşa, Ç. (2021). Comparative assessment of social sustainability performance: Integrated data-driven weighting system and CoCoSo model. Sustainable Cities and Society, 71, 102975.
  • Vavrek, R.; Becica, J. (2020). Efficiency evaluation of cultural services in the Czech Republic via multi-criteria decision analysis. Sustainability 2020, 12, 3409.
  • Wang, T. C. ve Lee, H. D. (2009). Developing a Fuzzy TOPSIS Approach Based on Subjective Weights and Objective Weights. Expert systems with applications, 36(5), 8980-8985.
  • Wang, Y. M. ve Luo, Y. (2010). Integration of Correlations with Standard Deviations for Determining Attribute Weights in Multiple Attribute Decision Making. Mathematical and Computer ModellingVolume, 51(1–2), 1–12.
  • Wu, Z., Sun, J., Liang, L. ve Zha, Y. (2011), Determination of Weights for Ultimate Cross Efficiency Using Shannon Entropy.Expert Systems With Applications, 38(5),5162–5165.
  • Yeh, C. H. and Willis, R. J. (2001). A validation procedure for multicriteria analysis: application to the selection of scholarship students. Asia Pacific Managemenl Review, 6(1): 39-52.
  • Zavadskas, E. K., ve Podvezko, V. (2016). Integrated determination of objective criteria weights in MCDM. International Journal of Information Technology Decision Making, 15(02), 267-283.
  • Zhang, X., Wang, C., Li, E. ve Xu, C. (2014). Assessment Model of Ecoenvironmental Vulnerability Based on Improved Entropy Weight Method. The Scientific World Journal, 2014, 1-7.
  • Zhang,H., Gu, C., Gu, L. ve Zhang, Y. (2011). The Evaluation of Tourism Destination Competitiveness by TOPSIS & Information Entropy - A Case in the Yangtze River Delta of China. Tourism Management, 32(2), 443-451.

A Real-Life Application of the Effect of Criterion Weighting Methods on MCDM Results

Yıl 2022, Cilt: 11 Sayı: 4, 1449 - 1463, 26.10.2022
https://doi.org/10.33206/mjss.1026666

Öz

This study aimed to examine the effects of six different techniques on simple additive weighting (SAW) Multiple Criteria Decision Making (MCDM) method results by using Entropy, Criteria Importance Through Intercriteria Correlation (CRITIC), Equal weight, MEthod based on the Removal Effects of Criteria (MEREC), Standard deviation and Integrated Data Oriented Weighting System (IDDWS) approaches, which are objective criteria weighting methods. In this study, which includes a real life application, it is aimed to measure the financial performance of Zorlu Energy Company for the period of 2016-2020 using the proposed model in the study. It was concluded that the ranking results obtained by the SAW method on the basis of Entropy, CRITIC, MEREC, Equal weight, standard deviation technique, IDDWS are the same. According to the results obtained by adding new alternatives to the data set, it was observed that the criteria weights had a different effect on the ranking results. As far as is known, the techniques used in the study have not been used together before, in this respect, it is thought that the study will fill the gap in the literature and contribute to the literature.

Kaynakça

  • Adriyendi, S. (2015). Multi attribute decision making using simple additive weighting and weighted product in food choice. Information Engineering and Electronic Business, 6: 8-14.
  • Akbulut, O. Y. ve Şenol, Z. (2021). Bütünleşik SD ve PROMETHEE ÇKKV Yöntemleri ile Portföy Optimizasyonu: BİST Gıda, İçecek ve Tütün Sektöründe Ampirik Bir Uygulama. Muhasebe ve Finansman Dergisi, (92), 161-182.
  • Akhtar, S., Javed, B., Maryam, A. ve Sadia, H. (2012). Relationship Between Financial Leverage and Financial Performance: Evidence From Fuel & Energy Sector of Pakistan. European Journal of Business and Management, 4(11),7-17.
  • Alemi-Ardakani, M., Milani, A. S., Yannacopoulos, S., ve Shokouhi, G. (2016). On the effect of subjective, objective and combinative weighting in multiple criteria decision making: A case study on impact optimization of composites. Expert Systems with Applications, 46, 426-438.
  • Anupama, G., ve Kesava Rao, V. V. S. (2020). Some Objective Methods for Determining Relative Importance of Financial Ratios. International Journal of Management, 10(4), 2019.
  • Cheng, L. H., Cao, D. Q., ve Guo, H. M. (2020). Analysis of coal mine occupational disease hazard evaluation index based on AHP-DEMATEL. Archives of Environmental & Occupational Health, 1-13.
  • Churchman C.W. ve Ackoff R.L. (1954). An approximatemeasure of value, Journal of Operations Research Societyof America. 2(1), 172-187.
  • Clausius, R. (1865). Ueber Verschiedene für die Anwendung Bequeme Formen der Hauptgleichungen der Mechanischen Wärmetheorie: Vorgetragen in der Naturforsch. Gesellschaft den 24.
  • Diakoulaki D., Mavrotas G. ve Papayannakis L. (1995). Determining objective Weights in Multiple Criteria Problems: The Critic Method. Computers & Operations Research, 22,763–770.
  • Du, Y., Zheng, Y., Wu, G., ve Tang, Y. (2020). Decision-making method of heavy-duty machine tool remanufacturing based on AHP-entropy weight and extension theory. Journal of Cleaner Production, 252, 119607.
  • Halkos, G. E. ve Tzeremes, N. G. (2012). Analyzing the Greek Renewable Energy Sector: A Data Envelopment Analysis Approach. Renewable and Sustainable Energy Reviews, 16(5), 2884-2893.
  • Inam, A. ve Mir, G. M. (2014). The İmpact of Financial Leverage on Firm Performance in Fuel and Energy Sector, Pakistan. European Journal of Business and Management, 6(37), 339-347.
  • Jahan, A., Mustapha, F., Sapuan, S.M., Ismail, M. Y., Bahraminasab, M. (2012). A framework for weighting of criteria in ranking stage of material selection process. The International Journal of Advanced Manufacturing Technology, 58, 411–420.
  • Jain, V. and Raj, T. (2013). Evaluation of flexibility in FMS using SAW and WPM. Decision Science Letters, 2(4): 223- 230.
  • Keshavarz-Ghorabaee, M., Amiri, M., Zavadskas, E. K., Turskis, Z., ve Antucheviciene, J. (2021). Determination of Objective Weights Using a New Method Based on the Removal Effects of Criteria (MEREC). Symmetry, 13(4), 525.
  • Li, H., Wang, W., Fan, L., Li, Q., ve Chen, X. (2020). A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL, entropy weighting and later defuzzification VIKOR. Applied Soft Computing, 91, 106207.
  • Maheshwari, N.; Choudhary, J.; Rath, A.; Shinde, D.; Kalita, K. (2021). Finite Element Analysis and Multi-criteria Decision-Making (MCDM)-Based Optimal Design Parameter Selection of Solid Ventilated Brake Disc. J. Inst. Eng. (India) Ser. C.
  • Memariani, A., Amini, B. and Alinezhad, A. (2009). Sensitivity analysis of Simple Additive Weighting method (SAW): The results of change in the weight of one attribute on the 77 final ranking of alternatives. Journal of Industrial Engineering, 2(4): 13-18.
  • Mukhametzyanov, I. (2021). Specific character of objective methods for determining weights of criteria in MCDM problems: Entropy, CRITIC and SD. Decision Making: Applications in Management and Engineering, 4(2), 76-105.
  • Pala, O. (2021). IDOCRIW ve MARCOS Temelli BIST Ulaştırma İşletmelerinin Finansal Performans Analizi. Kafkas Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 12(23), 263-294.
  • Paradowski, B., Shekhovtsov, A., Bączkiewicz, A., Kizielewicz, B., ve Sałabun, W. (2021). Similarity Analysis of Methods for Objective Determination of Weights in Multi-Criteria Decision Support Systems. Symmetry, 13(10), 1874.
  • Peng, X., Krishankumar, R., ve Ravichandran, K. S. (2021). A novel interval-valued fuzzy soft decision-making method based on CoCoSo and CRITIC for intelligent healthcare management evaluation. Soft Computing, 25(6), 4213-4241.
  • Piasecki, M., ve Kostyrko, K. (2020). Development of weighting scheme for indoor air quality model using a multi- attribute decision making method. Energies, 13(12), 3120.
  • Sahin, M. (2020). Hybrid Multi-attribute Decision Method for Material Selection. Int. J. Pure Appl. Sci., 6, 107–117.
  • Sahin, M. (2021). Location selection by multi-criteria decision-making methods based on objective and subjective weightings. Knowledge and Information Systems, 63(8), 1991-2021.
  • Sałabun, W.; Atróbski, J.; Shekhovtsov, A. (2020). Are MCDA methods benchmarkable? A comparative study of TOPSIS, VIKOR, COPRAS, and PROMETHEE II Methods. Symmetry, 12, 1549.
  • Shaverdi, M., Heshmati, M. R. ve Ramezani, I. (2014). Application of Fuzzy AHP Approach for Financial Performance Evaluation of Iranian Petrochemical Sector. Procedia Computer Science, 31, 995-1004.
  • Sisay, G., Gebre, S. L., ve Getahun, K. (2021). GIS-based potential landfill site selection using MCDM-AHP modeling of Gondar Town, Ethiopia. African Geographical Review, 40(2), 105-124.
  • Sivakumar, V. L., Krishnappa, R. R., ve Nallanathel, M. (2021). Drought vulnerability assessment and mapping using Multi-Criteria decision making (MCDM) and application of Analytic Hierarchy process (AHP) for Namakkal District, Tamilnadu, India. Materials Today: Proceedings, 43, 1592-1599.
  • Sotoudeh-Anvari, A., Sadjadi, S., Molana, S., ve Sadi-Nezhad, S. (2018). A new MCDM-based approach using BWM and SAW for optimal search model. Decision Science Letters, 7(4), 395-404.
  • Souissi, D., Zouhri, L., Hammami, S., Msaddek, M. H., Zghibi, A., ve Dlala, M. (2020). GIS-based MCDM–AHP modeling for flood susceptibility mapping of arid areas, southeastern Tunisia. Geocarto International, 35(9), 991-1017.
  • Sueyoshi, T. (2005). Financial Ratio Analysis of the Electric Power Industry. Asia-Pacific Journal of Operational Research, 22(03), 349-376.
  • Torkayesh, A. E., Ecer, F., Pamucar, D., ve Karamaşa, Ç. (2021). Comparative assessment of social sustainability performance: Integrated data-driven weighting system and CoCoSo model. Sustainable Cities and Society, 71, 102975.
  • Vavrek, R.; Becica, J. (2020). Efficiency evaluation of cultural services in the Czech Republic via multi-criteria decision analysis. Sustainability 2020, 12, 3409.
  • Wang, T. C. ve Lee, H. D. (2009). Developing a Fuzzy TOPSIS Approach Based on Subjective Weights and Objective Weights. Expert systems with applications, 36(5), 8980-8985.
  • Wang, Y. M. ve Luo, Y. (2010). Integration of Correlations with Standard Deviations for Determining Attribute Weights in Multiple Attribute Decision Making. Mathematical and Computer ModellingVolume, 51(1–2), 1–12.
  • Wu, Z., Sun, J., Liang, L. ve Zha, Y. (2011), Determination of Weights for Ultimate Cross Efficiency Using Shannon Entropy.Expert Systems With Applications, 38(5),5162–5165.
  • Yeh, C. H. and Willis, R. J. (2001). A validation procedure for multicriteria analysis: application to the selection of scholarship students. Asia Pacific Managemenl Review, 6(1): 39-52.
  • Zavadskas, E. K., ve Podvezko, V. (2016). Integrated determination of objective criteria weights in MCDM. International Journal of Information Technology Decision Making, 15(02), 267-283.
  • Zhang, X., Wang, C., Li, E. ve Xu, C. (2014). Assessment Model of Ecoenvironmental Vulnerability Based on Improved Entropy Weight Method. The Scientific World Journal, 2014, 1-7.
  • Zhang,H., Gu, C., Gu, L. ve Zhang, Y. (2011). The Evaluation of Tourism Destination Competitiveness by TOPSIS & Information Entropy - A Case in the Yangtze River Delta of China. Tourism Management, 32(2), 443-451.
Toplam 41 adet kaynakça vardır.

Ayrıntılar

Birincil Dil Türkçe
Bölüm Araştırma Makalesi
Yazarlar

Nazlı Ersoy 0000-0003-0011-2216

Erken Görünüm Tarihi 26 Ekim 2022
Yayımlanma Tarihi 26 Ekim 2022
Gönderilme Tarihi 21 Kasım 2021
Yayımlandığı Sayı Yıl 2022 Cilt: 11 Sayı: 4

Kaynak Göster

APA Ersoy, N. (2022). Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması. MANAS Sosyal Araştırmalar Dergisi, 11(4), 1449-1463. https://doi.org/10.33206/mjss.1026666
AMA Ersoy N. Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması. MJSS. Ekim 2022;11(4):1449-1463. doi:10.33206/mjss.1026666
Chicago Ersoy, Nazlı. “Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması”. MANAS Sosyal Araştırmalar Dergisi 11, sy. 4 (Ekim 2022): 1449-63. https://doi.org/10.33206/mjss.1026666.
EndNote Ersoy N (01 Ekim 2022) Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması. MANAS Sosyal Araştırmalar Dergisi 11 4 1449–1463.
IEEE N. Ersoy, “Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması”, MJSS, c. 11, sy. 4, ss. 1449–1463, 2022, doi: 10.33206/mjss.1026666.
ISNAD Ersoy, Nazlı. “Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması”. MANAS Sosyal Araştırmalar Dergisi 11/4 (Ekim 2022), 1449-1463. https://doi.org/10.33206/mjss.1026666.
JAMA Ersoy N. Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması. MJSS. 2022;11:1449–1463.
MLA Ersoy, Nazlı. “Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması”. MANAS Sosyal Araştırmalar Dergisi, c. 11, sy. 4, 2022, ss. 1449-63, doi:10.33206/mjss.1026666.
Vancouver Ersoy N. Kriter Ağırlıklandırma Yöntemlerinin ÇKKV Sonuçları Üzerindeki Etkisine Yönelik Gerçek Bir Hayat Uygulaması. MJSS. 2022;11(4):1449-63.

MANAS Journal of Social Studies (MANAS Sosyal Araştırmalar Dergisi)     


16155