The Effect of Online Learning Format, Online Learning Readiness and Self-directed Learning with Technology on Achievement and Perceived Learning

Öz

This study aims to examine the effects of different online learning formats on academic achievement and perceived learning. Another aim of the study is to examine the effects of online learning formats, online learning readiness and self-directed learning with technology on achievement and perceived learning. In the study, a quasi-experimental design was used. Two groups were involved in the study. While hybrid (synchronous and asynchronous) online applications were applied in the first experimental group, asynchronous online applications were applied in the second experimental group. The participants of this study are 73 students who are enrolled in the “Open and Distance Learning” course, which is one of the common elective courses of the faculty of education. To analyze the data, independent sample t-tests, Pearson correlation analysis, and hierarchical regression analysis were used. According to the results of the study, it was concluded that there was a significant difference in achievement according to the online learning format used and that the achievement scores of the students were higher in the asynchronous online learning format. When the perceived learning of the students was analyzed according to the online learning format used in the study, it was concluded that there was no significant difference. As a result of the hierarchical regression analysis, it was seen that the online learning format used in the study (hybrid or asynchronous) was a predictor of academic achievement. Finally, it was concluded that the online learning method and self-directed learning with technology variables did not significantly predict perceived learning, but online learning readiness was a significant predictor of perceived learning.

Anahtar Kelimeler

• Online Learning Readiness
• Self-directed Learning with Technology
• Perceived Learning
• Synchronous Distance Education
• Asynchronous Distance Education

Çevrim içi Öğrenme Formatı, Çevrim içi Öğrenmeye Yönelik Hazır Bulunuşluk ve Teknoloji ile Öz-yönelimli Öğrenmenin Başarı ve Algılanan Öğrenmeye Etkisi

Öz

Bu araştırmada farklı çevrim içi öğrenme formatlarının akademik başarı ve algılanan öğrenme üzerindeki etkisini incelemek amaçlanmıştır. Araştırmanın bir diğer amacı ise kullanılan çevrim içi öğrenme formatları, çevrim içi öğrenmeye yönelik hazır bulunuşluk ve teknoloji ile öz-yönelimli öğrenmenin başarı ve algılanan öğrenme üzerindeki etkisini incelemektir. Araştırmada iki deney grubu ile çalışılmış ve deney grupları hâlihazırda mevcut olan iki gruba atandığından yarı deneysel desen kullanılmıştır. Birinci deney grubunda hibrit (eşzamanlı ve eşzamansız) çevrim içi uygulamalar yapılırken, ikinci deney grubunda eşzamansız çevrim içi uygulamalar yapılmıştır. Araştırmanın çalışma grubunu eğitim fakültesi ortak seçmeli derslerinden biri olan “Açık ve Uzaktan Öğrenme” dersine kayıtlı 73 öğrenci oluşturmaktadır. Araştırmanın verilerinin analizinde bağımsız örneklem t-testi, Pearson korelasyon analizi ve hiyerarşik regresyon analizi kullanılmıştır. Araştırmanın sonucunda kullanılan çevrim içi öğrenme formatına göre başarı üzerinde anlamlı bir farklılık olduğu ve eşzamansız çevrim içi öğrenme formatında öğrencilerin başarı puanlarının daha yüksek olduğu sonucuna ulaşılmıştır. Araştırmada kullanılan çevrim içi öğrenme formatına göre öğrencilerin algılanan öğrenmeleri incelendiğinde ise anlamlı bir farklılık olmadığı sonucuna ulaşılmıştır. Yapılan hiyerarşik regresyon analizi sonucunda araştırmada kullanılan çevrim içi öğrenme formatının (hibrit veya eşzamansız) akademik başarının yordayıcısı olduğu görülmüştür. Son olarak kullanılan çevrim içi öğrenme yöntemi ve teknoloji ile öz-yönelimli öğrenme değişkenlerinin algılanan öğrenmeyi anlamlı olarak yordamadığı, ancak çevrim içi öğrenmeye yönelik hazır bulunuşluğun algılanan öğrenmenin anlamlı bir yordayıcısı olduğu sonucuna ulaşılmıştır.

Anahtar Kelimeler

• Çevrim içi Öğrenmeye Yönelik Hazır Bulunuşluk
• Teknoloji ile Öz-yönelimli Öğrenme
• Algılanan Öğrenme
• Eşzamanlı Uzaktan Eğitim
• Eşzamansız Uzaktan Eğitim

Kaynakça

1. Acar, C. (2014). Fen bilgisi öğretmen adaylarının kendi kendine öğrenme becerilerinin çeşitli değişkenler açısından incelenmesi [Yayınlanmamış yüksek lisans tezi]. Pamukkale Üniversitesi.
2. Albayrak, E., Canan-Güngören, Ö., & Horzum, M. B. (2014). Algılanan öğrenme ölçeğinin Türkçeye uyarlaması. Ondokuz Mayıs Üniversitesi Eğitim Fakültesi, 33(1), 1-27. http://doi.org/10.7822/egt252
3. Allen, I. E., & Seaman, J. (2017). Distance education enrollment report 2017. Digital Learning Compass. https://www.bayviewanalytics.com/reports/digtiallearningcompassenrollment2017.pdf
4. Amiti, F. (2020). Synchronous and asynchronous e-learning. European Journal of Open Education and E-learning Studies, 5(2), 60-70. https://doi.org/10.46827/ejoe.v5i2.3313
5. Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries. Organisation for Economic Cooperation and Development. EDU Working paper no. 41. https://doi.org/10.1787/218525261154
6. Aşkın-Tekkol, İ., & Demirel, M. (2018). An investigation of self-directed learning skills of undergraduate students. Frontiers in Psychology, 9, 1-14. http://doi.org/10.3389/fpsyg.2018.02324
7. Balaman, F. (2014). Web tabanlı uzaktan eğitimin meslek yüksekokulu öğrencilerinin İnternet Programcılığı 2 dersindeki akademik başarılarına etkisi [Yayınlanmamış doktora tezi]. Dicle Üniversitesi.
8. Barzilai, S., & Blau, I. (2014). Scaffolding game-based learning: Impact on learning achievements, perceived learning, and game experiences. Computers & Education, 70, 65-79. http://dx.doi.org/10.1016/j.compedu.2013.08.003

9. Batista, I. V. C., & Cornachione, E. B. Jr. (2005). Learning styles influences on satisfaction and perceived learning: Analysis of an online business game. Developments in Business Simulation and Experiential Learning, 32, 22-30.
10. Blau, I., Weiser, O., & Eshet-Alkalai, Y. (2016, June). Face-to-face versus one-way and two-way videoconferencing: How medium naturalness and personality traits influence achievement and perceived learning [Conference presentation]? 11th Iberian Conference on Information Systems and Technologies (CISTI), Gran Canaria, Spain.
11. Bernard, R. M., Brauer, A., Abrami, P. C., & Surkes, M. (2004). The development of a questionnaire for predicting online learning achievement. Distance Education, 25(1), 31-47. http://doi.org/10.1080/0158791042000212440
12. Boeglin, J. A., & Campbell, K. (2002). Effects of learners' readiness on their perceived learning outcomes. Canadian Journal of Learning and Technology, 28(2). http://doi.org/10.21432/T2CW24
13. Büyüköztürk, Ş., Kılıç-Çakmak, E., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2018). Eğitimde bilimsel araştırma yöntemleri (25. Baskı). Pegem Akademi Yayıncılık.
14. Cazan, A. M., & Schiopca, B. A. (2014). Self-directed learning, personality traits and academic achievement. Procedia-Social and Behavioral Sciences, 127, 640-644. http://doi.org/10.1016/j.sbspro.2014.03.327
15. Chou, P. N. (2012). The relationship between engineering students’ self-directed learning abilities and online learning performances: A pilot study. Contemporary Issues In Education Research, 5(1), 33-38. http://doi.org/10.19030/cier.v5i1.6784
16. Çakır, Ö., & Horzum, M. B. (2015). Öğretmen adaylarının çevrimiçi öğrenmeye hazır bulunuşluk düzeylerinin çeşitli değişkenler açısından incelenmesi. Eğitimde Kuram ve Uygulama, 11(1), 1-15.
17. Demir, Ö., & Yurdugül, H. (2013). Self-directed learning with technology scale for young students: A validation study. International Journal of Educational Research, 4(3), 58-73.
18. Demir, Ö., Yaşar, S., Sert, G., & Yurdugül, H. (2014). Çocukların bilgisayara yönelik tutumları ile teknolojiyle kendi kendine öğrenmeleri arasındaki ilişkinin incelenmesi. Eğitim ve Bilim, 39(176), 257-266.
19. Demir-Kaymak, Z., & Horzum, M. B. (2013). Çevrimiçi öğrenme öğrencilerinin çevrimiçi öğrenmeye hazır bulunuşluk düzeyleri, algıladıkları yapı ve etkileşim arasındaki ilişki. Kuram ve Uygulamada Eğitim Bilimleri, 13(3), 1783-1797.
20. Demir-Kaymak, Z., & Horzum, M. B. (2022). Student barriers to online learning as predictors of perceived learning and academic achievement. Turkish Online Journal of Distance Education-TOJDE, 23(2), 97-106. https://doi.org/10.17718/tojde.1096250
21. Dikbaş-Torun, E. (2020). Online distance learning in higher education: E-learning readiness as a predictor of academic achievement. Open Praxis, 12(2), 191-208. http://doi.org/10.5944/openpraxis.12.2.1092
22. Doğru, E. (2020). Aşamalı öz-yönetimli öğrenme modelinin uzaktan İngilizce eğitiminde kullanılmasının hazırbulunuşluğa, başarıya, tutuma ve kalıcılığa etkileri [Yayınlanmamış doktora tezi]. Bolu Abant İzzet Baysal Üniversitesi.
23. Eroğlu, M., & Özbek, R. (2018). The investigation of the relationship between attitudes towards e-learning and self-directed learning with technology of secondary school students. International Online Journal of Educational Sciences, 10(5), 297-314. http://dx.doi.org/10.15345/iojes.2018.05.019
24. Field, A. (2009). Discovering statistics using spss (Third Edition). SAGE Publications Limited.
25. Glass, J., & Sue, V. (2008). Student preferences, satisfaction, and perceived learning in an online mathematics class. MERLOT Journal of Online Learning and Teaching, 4(3), 325-338.
26. Gülbahar, Y. (2012). Study of developing scales for assessment of the levels of readiness and satisfaction of participants in e-learning environments. Ankara University Journal of Faculty of Educational Sciences, 45(2), 119-137. https://doi.org/10.1501/Egifak_0000001256
27. Gürer, M. D., & Yıldırım, Z. (2014). Effectiveness of learning objects in primary school social studies education: achievement, perceived learning, engagement and usability. Education and Science Journal, 39(176), 131-143. https://doi.org/10.15390/EB.2014.3714
28. Gür-Erdoğan, D., Tutar, P., & Horzum, M. B. (2015, June). The relationship between lifelong learning tendency and self-directed with technology in education faculty students [Conference presentation]. 2015 ERPA International Congress on Education, Athens, Greece.
29. Haggerty, D. L. (2000). Engaging adult learners in self-directed learning and its impact on learning styles [Unpublished doctoral dissertation]. University of New Orleans.
30. Haverila, M. (2010). Factors related to perceived learning outcomes in e-learning. International Journal of Knowledge and Learning, 6(4), 308-328. http://doi.org/10.1504/IJKL.2010.038652
31. Haverila, M. (2011). Prior e-learning experience and perceived learning outcomes in an undergraduate e-learning course. MERLOT Journal of Online Learning and Teaching, 7(2), 206-218.
32. Horzum, M. B., Demir-Kaymak, Z., & Canan-Güngören, Ö. (2015). Structural equation modeling towards online learning readiness, academic motivations, and perceived learning. Educational Sciences: Theory and Practice, 15(3), 759-770. http://doi.org/10.12738/estp.2015.3.2410
33. Horzum, M. B., Önder, İ., & Beşoluk, Ş. (2014). Chronotype and academic achievement among online learning students. Learning and Individual Differences, 30, 106-111. http://doi.org/10.1016/j.lindif.2013.10.017
34. Hung, M. L., Chou, C., Chen, C. H., & Own, Z. Y. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080-1090. https://doi.org/10.1016/j.compedu.2010.05.004
35. İlhan, M., & Çetin, B. (2013). Çevrimiçi öğrenmeye yönelik hazır bulunuşluk ölçeğinin (ÇÖHBÖ) Türkçe formunun geçerlik ve güvenirlik çalışması. Eğitim Teknolojisi: Kuram ve Uygulama, 3(2), 72-99.
36. International Society for Technology in Education. (2016). ISTE Standards For Student. https://www.iste.org/
37. Karataş, K., & Arpacı, İ. (2021). The role of self-directed learning, metacognition, and 21st century skills predicting the readiness for online learning. Contemporary Educational Technology, 13(3), ep300. https://doi.org/10.30935/cedtech/10786
38. Khalid, M., Bashir, S., & Amin, H. (2020). Relationship between self-directed learning (SDL) and academic achievement of university students: A case of online distance learning and traditional universities. Bulletin of Education and Research, 42(2), 131-148.
39. Khalil, H., & Ebner, M. (2017). Using electronic communication tools in online group activities to develop collaborative learning skills. Universal Journal of Educational Research, 5(4), 529-536. https://10.13189/ujer.2017.050401
40. Kılınç, Ş., & Demir-Kaymak, Z. (2019, June). Lisansüstü uzaktan eğitim öğrencilerinin çevrimiçi öğrenmeye hazır bulunuşluk düzeylerinin kişilik yapılarına göre incelenmesi [Sözlü Sunum]. 2019 ERPA International Congress on Education, Sakarya, Turkey.
41. Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Prentice Hall Regents.
42. Korkmaz, Ö., Çakır, R., & Tan, S. S. (2015). Öğrencilerin e-öğrenmeye hazır bulunuşluk ve memnuniyet düzeylerinin akademik başarıya etkisi. Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi (KEFAD), 16(3), 219-241.
43. Kunin, M., Julliard, K. N., & Rodriguez, T. E. (2014). Comparing face-to-face, synchronous, and asynchronous learning: Postgraduate dental resident preferences. Journal of Dental Education, 78(6), 856-866. https://doi.org/10.1002/j.0022-0337.2014.78.6.tb05739.x
44. Larson, L. C., & Miller, T. N. (2011). 21st century skills: Prepare students for the future. Kappa Delta Pi Record, 47(3), 121-123. http://doi.org/10.1080/00228958.2011.10516575
45. Latheef, Z.I., Robinson, R., & Smith, S. (2021). Realistic job preview as an alternative tool to improve student readiness for online learning. Online Learning, 25(2), 208-232. https://doi.org/10.24059/olj.v25i2.2216
46. Linares, A. Z. (1999). Learning styles of students and faculty in selected health care professions. Journal of Nursing Education, 38(9), 407-414. http://doi.org/10.3928/0148-4834-19991201-07
47. Lopes, C. T. (2007). Evaluating e-learning readiness in a health sciences higher education institution [Conference presentation]. IADIS International Conference of E-learning 2007 (pp. 59-67). Instituto Politécnico do Porto.
48. Martin, F., Stamper, B., & Flowers, C. (2020). Examining student perception of their readiness for online learning: Importance and confidence. Online Learning, 24(2), 38-58. https://doi.org/10.24059/olj.v24i2.2053
49. McVay, M. (2000). Developing a web-based distance student orientation to enhance student success in an online bachelors degree completion program [Unpublished doctoral dissertation]. Nova Southeastern University.
50. Myers, R. (1990). Classical and modern regression with applications (2nd ed.). Duxbury.
51. Partnership for 21st Century Learning. (2019). Framework for 21st century learning. http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf
52. Patton, B. A. (2008). Synchronous meetings: A way to put personality in an online class. Turkish Online Journal of Distance Education, 9(4).
53. Perveen, A. (2016). Synchronous and asynchronous e-language learning: A case study of Virtual University of Pakistan. Open Praxis, 8(1), 21-39. http://dx.doi.org/10.5944/openpraxis.8.1.212
54. Raymond, E., Atsumbe, B. N., Okwori, R. O., & Jebba, M. A. (2016). Comparative effects of the synchronous and the asynchronous instructional approaches concerning the students’ achievements and interests in electrical engineering at the Niger State College of Education. International Journal of Engineering Pedagogy (iJEP), 6(3), 4-9.
55. Roblyer, M. D., Freeman, J., Donaldson, M. B., & Maddox, M. (2007). A comparison of outcomes of virtual school courses offered in synchronous and asynchronous formats. Internet and Higher Education, 10(4), 261-268. https://10.1016/j.iheduc.2007.08.003
56. Rockinson-Szapkiw, A. J., Baker, J. D., Neukrug, E., & Hanes, J. (2010). The efficacy of computer mediated communication technologies to augment and support effective online helping profession education. Journal of Technology in Human Services, 28(3), 161-177. https://doi.org/10.1080/15228835.2010.508363
57. Rovai, A. P., Wighting, M. J., Baker, J. D., & Grooms, L. (2009). Development of an instrument to measure perceived cognitive, affective, and psychomotor learning in traditional and virtual classroom higher education settings. The Internet and Higher Education, 12(1), 7-13. http://doi.org/10.1016/j.iheduc.2008.10.002
58. Sakal, M. (2017). Çevrimiçi öğrenmede öğrencilerin hazırbulunuşluk düzeylerinin demografik özelliklerine göre incelenmesi. Sosyal ve Beşeri Bilimler Araştırmaları Dergisi, 18(39), 81-102.
59. Sarıtaş, E., & Barutçu, S. (2020). Öğretimde dijital dönüşüm ve öğrencilerin çevrimiçi öğrenmeye hazır bulunuşluluğu: Pandemi döneminde Pamukkale Üniversitesi öğrencileri üzerinde bir araştırma. İnternet Uygulamaları ve Yönetimi Dergisi, 11(1), 5-22. http://doi.org/10.34231/iuyd.706397
60. Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. Babson Survey Research Group. http://onlinelearningsurvey.com/reports/gradeincrease.pdf
61. Selvi, K. (2019). Kendi kendine öğrenmenin kuramsal çerçevesi. In K. Selvi (Ed.), Kendi kendine öğrenme (pp.7-34). Pegem Akademi Yayıncılık.
62. Sharifrazi, F., & Stone, S. (2019, April). Students perception of learning online: Professor's presence in synchronous versus asynchronous modality [Conference presentation]. 5th International Conference on Computer and Technology Applications, İstanbul, Turkey.
63. Skylar, A. A. (2009). A comparison of asynchronous online text-based lectures and synchronous interactive web conferencing lectures. Issues in Teacher Education, 18(2), 69-84.
64. So, T., & Swatman, P. M. C. (2005, July). E-learning readiness of Hong Kong teacher [Conference presentation]. 5. IEEE International Conference on Advanced Learning Technologies ICALT'05, University of South Australia. http://doi.org/10.1109/ICALT.2005.266
65. Song, L., & Hill, J. R. (2007). A conceptual model for understanding self-directed learning in online environments. Journal of Interactive Online Learning, 6(1), 27-42.
66. Stojan, J., Haas, M., Thammasitboon, S., Lander, L., Evans, S., Pawlik, C., Pawilkowska, T., Lew, M., Khamees, D., Peterson, W., Hider, A., Grafton-Clarke, C., Uraiby, H., Gordon, M., & Daniel, M. (2022). Online learning developments in undergraduate medical education in response to the COVID-19 pandemic: A BEME systematic review: BEME Guide No. 69. Medical Teacher, 44(2), 109-129. https://doi.org/10.1080/0142159X.2021.1992373
67. Stuart, J., O’Donnell, A. W., Scott, R., O’Donnell, K., Lund, R., & Barber, B. (2022). Asynchronous and synchronous remote teaching and academic outcomes during COVID-19. Distance Education, 43(3), 408-425. https://doi.org/10.1080/01587919.2022.2088477
68. Şumuer, E. (2018). Factors related to college students’ self-directed learning with technology. Australasian Journal of Educational Technology, 34(4), 29-43. https://doi.org/10.14742/ajet.3142
69. Tabatabaei, O., & Parsafar, S. M. (2012). The effect of self-directed learning on critical thinking of Iranian EFL learners. Journal of Educational and Social Research, 2(2), 55-64. https://www.richtmann.org/journal/index.php/jesr/article/view/11808
70. Teo, T., Tan, S. C., Lee, C. B., Chai, C. S., Koh, J. H. L., Chen, W. L., & Cheah, H. M. (2010). The self-directed learning with technology scale (SDLTS) for young students: An initial development and validation. Computers & Education, 55 (4), 1764-1771. http://doi.org/10.1016/j.compedu.2010.08.001
71. Tercan, S. S., Horzum, M. B., & Uysal, M. (2014). Teknoloji ile öz-yönelimli öğrenme ölçeğinin Türkçeye uyarlanması. Eğitim Teknolojileri Araştırmaları Dergisi, 5(3). https://www.academia.edu/9785181/TEKNOLOJ%C4%B0_%C4%B0LE_%C3%96Z-Y%C3%96NEL%C4%B0ML%C4%B0_%C3%96%C4%9ERENME_%C3%96L%C3%87E%C4%9E%C4%B0N%C4%B0N_T%C3%9CRK%C3%87E_YE_UYARLANMASI?auto=download
72. Trespalacios, J., & Perkins, R. (2016). Sense of community, perceived learning, and achievement relationships in an online graduate course. Turkish Online Journal of Distance Education-TOJDE, 17(3), 31-49. https://doi.org/10.17718/tojde.12984
73. Uysal, M. (2015). Çevrimiçi öğrenme ve karma öğrenme öğrencilerinin teknoloji ile öz-yönelimli öğrenmeleri ve sorgulama topluluğu arasındaki ilişkinin incelenmesi: Yapısal eşitlik modeli [Yayınlanmamış yüksek lisans tezi]. Sakarya Üniversitesi.
74. Ünsal-Avdal, E. (2013). The effect of self-directed learning abilities of student nurses on success in Turkey. Nurse Education Today, 33(8), 838-841. http://doi.org/10.1016/j.nedt.2012.02.006
75. Viriya, C. (2022). Exploring the impact of synchronous, asynchronous, and bichronous online learning modes on EFL students’ self-regulated and perceived English language learning. rEFLections, 29(1), 88-111.
76. Watson, A. (2020, June 18). Device usage increase due to the coronavirus worldwide 2020, by country. Statista. https://www.statista.com/statistics/1106607/device-usage-coronavirusworldwide-by-country/
77. Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Network, 8(2), 139-152. http://doi.org/10.1142/S1609945104000115
78. Yılmaz, R., Karaoğlan-Yılmaz, F. G., & Çavuş-Ezin, Ç. (2017). Self-directed learning with technology and academic motivation as predictors of tablet PC acceptance. In A. A. Khan & S. Umair (Eds.), Handbook of research on mobile devices and smart gadgets in K-12 education (pp. 87-102). IGI Global.
79. Yu, T., & Richardson, J. C. (2015). An exploratory factor analysis and reliability analysis of the student online learning readiness (SOLR) instrument. Online Learning, 19(5), 120-141. http://dx.doi.org/10.24059/olj.v19i5.593
80. Yurdugül, H., & Alsancak-Sarıkaya, D. (2013). Çevrimiçi öğrenme hazır bulunuşluluk ölçeği: Geçerlik ve güvenirlik çalışması. Eğitim ve Bilim, 38(169), 391-406.
81. Yurdugül, H., & Demir, Ö. (2017). Öğretmen yetiştiren lisans programlarındaki öğretmen adaylarının e-öğrenmeye hazır bulunuşluklarının incelenmesi: Hacettepe Üniversitesi örneği. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 32(4), 896-915. http://doi.org/10.16986/HUJE.2016022763
82. Zhu, M. (2021). Enhancing MOOC learners’ skills for self-directed learning. Distance Education, 42(3), 441-460. https://doi.org/10.1080/01587919.2021.1956302
83. Zhu, M., & Bonk, C. J. (2019). Designing MOOCs to facilitate participant self-directed learning: An analysis of instructor perspectives and practices. International Journal of Self-Directed Learning, 16(2), 39-60. https://doi.org/10.24059/olj.v23i4.2037