Dilemmas of Teachers Related to the Supervision in Schools: A Case Study

Şenol Sezer; Gamzegül Engin

doi:10.17275/per.24.16.11.2

EN

Dilemmas of Teachers Related to the Supervision in Schools: A Case Study

Abstract

School supervision is generally defined as the process of determining the realization level of performance indicators in schools, and evaluating the quality of education. Supervisors are not directly involved in these activities, and are appointed by the Ministry to assess fulfilment level of these educational activities. In current study, teachers’ perspective related to the supervision in Türkiye was examined in a qualitative research design. A single case study model was adopted to explore the case in depth. The aim of this research was to explore views of 28 teachers on supervision. The study group was chosen through maximum variation sampling technique. Data were collected via semi-structured interviews, observations, and focus group interviews. Thematic content analysis technique was adopted to analysis of raw data. Four themes emerged based on the analysis. These themes were low stakes inspection, high stakes inspection, decentralization, and decreasing in quality. The results revealed that the participants were in a dilemma about the supervision in schools. Moreover, this dilemma was related to the audit of teacher performance by school principal and the supervision of school by ministry inspectors. In addition, through the focus group interviews, the suggestions of the participants with different perspectives on inspection were explored and thus more generalizable themes were produced.

Keywords

supervision, inspector, teacher assessment, school inspection, accountability

References

Akbasli, S., & Tunc, Z. (2019). The assessment of primary school teachers’ opinions about course supervision of school principals. Erzincan University Journal of Education Faculty, 21(3), 75-92. https://doi.org/10.17556/erziefd.474511
Akyuz, Y. (2015). Türk eğitim tarihi [The history of Turkish education] (27th ed.). Pegem Akademi.
Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13(4), 544-559. https://doi.org/10.46743/2160-3715/2008.1573.
Beaver, M. D. (2002). Instructional supervision: Perspectives of middle school fine arts teachers. (Unpublished PhD Dissertation). Graduate Faculty of the University of Georgia, Athens, Georgia.
Bilir, M. (1991). The structure and functioning of the inspection subsystem in the Turkish education system (Unpublished PhD Dissertation). Ankara University, Educational Sciences Institution, Ankara, Turkey. https://tez.yok.gov.tr/UlusalTezMerkezi/tezSorguSonucYeni.jsp
Binkley, N. (1995). Reforming teacher evaluation policy: A qualitative study of the principal’s role. Journal of Personnel Evaluation in Education, 9, 223-242. doi:10.1007/BF00972639
Bland, A. R., & Rossen, E. K. (2005). Clinical supervision of nurses working with patients with borderline personality disorder. Issues in Mental Health Nursing, 26(5), 507-517. https://doi.org/10.1080/01612840590931957.
Blasé, J., Blasé, J., & Phillips, D. Y. (2010). Handbook of school improvement: How high performing principals create high-performing schools. Corwin Press.
Boydak-Ozan, M., & Sener, G. (2015). Examination of supervision-related perceptions and expectations of classroom teachers through metaphors. Journal of Educational Sciences, 41, 19-33. https://doi.org/10.15285/ebd.55126.
Brady, A. M. (2016). The regime of self-evaluation: Self-conception for teachers and schools. British Journal of Educational Studies, 64(4), 523-541. https://doi.org/10.1080/00071005.2016.1164829.

Cagan, V. (1998). Primary school teachers’ perceptions and expectations of principals’ supervision and leadership skills, (Unpublished master’s thesis). Dokuz Eylul University Institute of Social Sciences, Izmir, Turkiye.
Can, E., & Gunduz, Y. (2016). Investigating the level of benefit primary school teachers get from the guidance of educational supervisors and principals. Educational Administration: Theory and Practice, 22(1), 1-28. https://doi.org/10.14527/kuey.2016.001
Cokluk, O., Yilmaz, K., & Oguz, E. (2011). A qualitative interview method: Focus group interview. Journal of Theoretical Educational Science, 4(1), 95-107. http://www.keg.aku.edu.tr/arsiv/c4s1/c4s1m6.pdf.
Creswell, J. W. & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage.
Dagli, A. (2000). Effective principal behaviors of primary school principals according to primary school teachers’ perceptions. Educational Administration: Theory and Practice, 23(23), 431-442. https://dergipark.org.tr/tr/pub/kuey/issue/10373/126953
Daresh, J. C. (2006). Leading and supervising instruction. Sage.
Darling-Hammond, L. (1994). Performance-based assessment and educational equity. Harvard Educational Review, 64(1), 5-31. https://doi.org/10.17763/haer.64.1.j57n353226536276
Dedering, K. & Sowada, M. G. (2017). Reaching a conclusion-procedures and processes of judgement formation in school inspection teams. Educational Assessment, Evaluation and Accountability, 29, 5-22. https://doi.org/10.1007/s11092-016-9246-9
De Fraine, B., Van Damme, J., & Onghena, P. (2002). Accountability of schools and teachers: What should be taken into account? European Educational Research Journal, 1(3), 403-428. https://doi.org/10.2304/eerj.2002.1.3.2
Denner, P. R., Salzman, S. A., & Bangert, A. W. (2001). Linking teacher assessment to student performance: A benchmarking, generalizability, and validity study of the use of teacher work samples. Journal of Personnel Evaluation in Education, 15, 287-307. https://doi.org/10.1023/A:1015405715614.
Donmez, B. & Demirtas, Ç. (2018). Opinions of school principals and teachers on the school principals’ duties of lesson supervision (An example of Adıyaman province). Adıyaman University Journal of Social Sciences Institute, 10(29), 454-478. https://dergipark.org.tr/tr/pub/adyusbd/issue /39062/379209
Durnali, M., & Limon, İ. (2018). Contemporary educational supervision system of Turkey (The latest changes and their legal basis). Kastamonu Education Journal, 26(2), 413-425. doi:10.24106/kefdergi.389801
Ehren, M. C. M., & Honingh, M. E. (2011). Risk-based school inspections in the Netherlands. Studies in Educational Evaluation, 37(4), 239-248. https://doi.org/10.1016/j.stueduc.2012.02.00
Ehren, M. C. M. & S Altrichter, H., McNamara, G., & O’Hara, J. (2013). Impact of school inspections on improvement of schools-describing assumptions on causal mechanisms in six European countries. Educ Asse Eval Acc, 25, 3-43. https://doi.org/10.1007/s11092-012-9156-4
Ehren, M. C. M. & Hatch, T. (2013). Responses of schools to accountability systems using multiple measures: the case of New York City elementary schools. Educ Asse Eval Acc, 25, 341-373. https://doi.org/10.1007/s11092-013-9175-9
Ehren, M. C. M. & Shackleton, N. (2016). Risk-based school inspections: impact of targeted inspection approaches on Dutch secondary schools. Educ Asse Eval Acc, 28, 299-321. https://doi.org/10.1007/s11092-016-9242-0
Ehren, M. C. M., & Godfrey, D. (2017). External accountability of collaborative arrangements; a case study of a Multi Academy Trust in England. Educ Asse Eval Acc, 29, 339-362. https://doi.org/10.1007/s11092-017-9267-z
Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges. The Academy of Management Journal, 50(1), 25-32. https://www.jstor.org/stable/20159839.
Ekleme, Y. (2001). Primary school administrators’ fulfilment level of duties related to supervision. (Unpublished master’s thesis). Yıldız Technical University Institute of Social Sciences, Istanbul, Turkey.
Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107-115. https://doi.org/10.1111/j.1365-2648.2007.04569.x.
Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative content analysis: A focus on trustworthiness. SAGE Open, 4(1), 1-10. https://doi.org/10.1177/2158244014522633.
Erlingsson, C., & Brysiewicz, P. (2017). A hands-on guide to doing content analysis. African Journal of Emergency Medicine, 7(3), 93-99. https://doi.org/10.1016/j.afjem.2017.08.001.
Glatthorn, A. A. (1997). Differentiated supervision. (2nd ed.). ASCD.
Glesne, C. (2016). Becoming qualitative researchers: An introduction. (5th ed.). Allyn & Bacon.
Glickman, C. D., Gordon, S. P., & Ross-Gordon, J. M. (2012). The basic guide to supervision and teaching leadership (3rd ed.) Pearson.
Grashel, J. N. (1997). The impact of the use of developmental supervision by principals on teachers’ efficacy, expectations, collegiality, trust and commitment. (Unpublished PhD Dissertation), University of Kansas, USA. http://proquest.umi.com/
Grover, B. W. (1991). The teacher assessment dilemma: What is versus what ought to be! Journal of Personnel Evaluation in Education 5, 103-119. https://link.springer.com/content/pdf/10.1007/BF00117332.pdf.
Grudens-Schuck, N., Allen, B. L., & Larson, K. (2004). Methodology brief: Focus group fundamentals. Extension Community and Economic Development Publications, 12. Retrieved from https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1011&context=extension_communities_pubs.
Gustafsson, J-E., Lander, R., & Myrberg, E. (2014). Inspections of Swedish schools: A critical reflection on intended effects, causal mechanisms and methods. Education Inquiry, 5(4), 461-479. https://doi.org/10.3402/edui.v5.23862
Gunduz, Y., & Balyer, A. (2011). Alternative approaches in educational supervision. M.U. Atatürk Faculty of Education Journal of Educational Sciences, 33, 61-78. https://dergipark.org.tr/tr/download/article-file/1711
Gunduz, Y. (2017). A study on fulfilment levels of supervisory duties of school principals in primary schools. Kastamonu Education Journal, 25(5), 1681-1694. https://dergipark.org.tr/tr/pub/kefdergi/issue/31226/342716
Holland, P. (2005). The case for expanding standards for teacher evaluation to include an instructional supervision perspective. J Pers Eval Educ, 18, 67-77. https://doi.org/10.1007/s11092-006-9009-0
Hyrkäs, K. (2005). Clinical supervision, burnout and job satisfaction among mental health and psychiatric nurses in Finland. Issues in Mental Health Nursing, 26(5), 531-556. https://doi.org/10.1080/01612840590931975
Jaffer, K. (2010). School inspection and supervision in Pakistan: Approaches and issues. Prospects, 40, 375-392. https://doi.org/10.1007/s11125-010-9163-5
Jones, K., & Tymms, P. (2014). Ofsted’s role in promoting school improvement. Oxford Review of Education, 40(3), 315-330. https://dro.dur.ac.uk/12184/1/12184.pdf
Kaplan, I. (2016). The views of supervisors related to exemption from class/lesson supervisions. Journal of Education for Life, 30(1), 43-60. http://journals.iku.edu.tr/yed/index.php/yed/article/view/48
Kayikci, K., Ozdemir, I., & Ozyildirim, G. (2018). School principals’ views about changes in supervision perspective and implementations. Elementary Education Online, 17(4), 2170-2187. https://doi.org/10.17051/ilkonline.2019.506995
Kel, M. A., & Akin, U. (2021). Educational supervision in a change process: The inspectors’, school administrators’ and teachers’ views. TEBD, 19(1), 219-243. https://doi.org/10.37217/tebd.851827
Kemethofer, D., Gustafsson, J. H., & Altrichter, H. (2017). Comparing effects of school inspections in Sweden and Austria. Educ Asse Eval Acc, 29, 319-337. https://doi.org/10.1007/s11092-017-9265-1
Koc, M. H. (2018). Primary school principals’ views on school principals’ activities for the supervision of teachers. MU Atatürk Education Faculty Journal of Educational Sciences, 48, 91-110. https://doi.org/10.15285/maruaebd.349727
Kocak, S., & Arslan, S. Y. (2018). Opinions and suggestions of teachers working at secondary education institutions on performance evaluation practices. Ahi Evran University Journal of Kırşehir Education Faculty, 19(1), 602-620. http://dx.doi.org/10.29299/kefad.2018.19.017
Konan, N., Bozanoglu, B., & Cetin, R. B. (2019). School managers’ and teachers’ opinions on supervision policies of the Ministry of National Education. Journal of Qualitative Research in Education, 7(4), 1449-1475. https://doi.org/10.14689/issn.2148-2624.1.7c.4s.7m
Kosar, S., & Buran, K. (2019). An analysis of school principals’ course supervision activities in regard of instructional leadership. Journal of Qualitative Research in Education, 7(3), 1232-1265. https://doi.org/10.14689/issn.2148-2624.1.7c.3s.14.m
Kraft, M. A., & Gilmour, A. (2016). Can principals promote teacher development as evaluators? A case study of principals’ views and experiences. Educational Administration Quarterly, 52(5), 711-753. https://doi.org/10.1177/0013161X16653445
Kurum, G., & Cinkir, Ş. (2017). Marriage made in hell: Views of education supervisors on the unification of education supervision in Turkey. Education and Science, 42(192), 35-57. https://doi.org/10.15390/EB.2017.7288.
Li, H. & Li, X. (2020). Rural and urban general education teachers’ instructional strategies in inclusive classrooms in China: A dual system perspective. International Journal of Inclusive Education, https://doi.org/10.1080/13603116.2020.1821796.
Lincoln, Y. S. & Guba, E. G. (1985). Naturalistic inquiry. Sage.
Lindblad, T. L. (2021). Ethical considerations in clinical supervision: components of effective clinical supervision across an inter-professional team. Behav Analysis Practice (Early release). https://doi.org/10.1007/s40617-020-00514-y.
Low, E. L., Goodwin, A. L., & Snyder, J. (2017). Focused on learning: Student and teacher time in a Singapore school. Stanford Center for Opportunity Policy in Education. https://edpolicy.stanford.edu/sites/default/files/scope-singapore-student-and-teacher-time-report-final_0.pdf
Mayring, P. (2014). Qualitative content analysis: theoretical foundation, basic procedures and software solution. http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173
McMahon, M., & Patton, W. (2000). Conversations on clinical supervision: benefits perceived by school counsellors. British Journal of Guidance & Counselling, 28(3), 339-351. https://doi.org/10.1080/713652301
MacNab, D. (2004). Hearts, minds and external supervision of schools: Direction and development. Educational Review, 56(1), 53-64. https://doi.org/10.1080/0013191032000152273
Memduhoglu, H. B., Aydin, İ., Yilmaz, K., Gungor, S., & Oguz, E. (2007). The process of supervision in the Turkish Educational System: Purpose, structure, operation. Asia Pacific Education Review, 8(1), 56-70. doi: https://doi.org/10.1007/BF03025833
Memduhoglu, H. B. & Zengin, M. (2012). Implementability of instructional supervision as a contemporary educational supervision model in Turkish Education System. Journal of Theoretical Educational Science, 5(1), 131-142. https://dergipark.org.tr/tr/download/article-file/304184
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. Jossey-Bass.
Milanowski, A. T. & Heneman, H. G. (2001). Assessment of teacher reactions to a standards-based teacher evaluation system: A pilot study. Journal of Personnel Evaluation in Education, 15, 193-212. https://doi.org/10.1023/A:1012752725765
MoNE, National Education Basic Law and the Law on the Amendments of Some Laws. (2014, March 14). T. R. Official Gazette, 28941. https://www.resmigazete.gov.tr/eskiler/2014/03/20140314-1.html
MoNE, Regulation on Teacher Appointment and Relocation. (2015, April 17). T. R. Official Gazette, 29329. https://www.resmigazete.gov.tr/eskiler/2015/04/20150417-4.html
MoNE, Inspection Board Regulation. (2017, August 20). T. R. Official Gazette, 30160. https://www.resmigazete.gov.tr/eskiler/2017/08/20170820-1.html
Montean, J. J. (1970). The role of the science supervisor in the teacher education process. Science Education, 54(3), 295-298. https://doi.org/10.1002/sce.3730540321
Nusche, D., Braun, H., Halász, G., & Santiago, P. (2014). OECD reviews of evaluation and assessment in education: Netherlands. OECD Publishing. http://dx.doi.org/10.1787/9789264211940-en
Oncel, Y. (2006). Primary school principals’ roles and competencies in supervision, (Unpublished PhD thesis). Harran University Social Sciences Institution, Şanlıurfa, Turkiye.
Ozbas, M. (2002). The opinions of the principals and the teachers about the works that primary school principals should and are doing in the supervision of classroom activities, (Unpublished master’s thesis). Hacettepe University Social Sciences Institution, Ankara, Turkiye.
Pasaribu, A., Purba, S., & Matondang, Z. (2017). Implementation of academic supervision of artistic model in improving the role of English teacher as learning agent in SMA Negeri 5 Binjai. Journal of Research & Method in Education, 7(6), 34-41. https://doi.org/10.9790/7388-0706013441
Patton, M. Q. (2014). Qualitative research & evaluation methods (4th ed.). Sage.
Perry, C. (2013). Approaches to school supervision. (Research Report), NIAR 521-13. http://www.niassembly.gov.uk/globalassets/documents/raise/publications/2013/education/1261.pdf
Reezigt, G. J., Creemers, B. P., & de Jong, R. (2003). Teacher evaluation in the Netherlands and its relationship to educational effectiveness research. Journal of Personnel Evaluation in Education 17, 67-81 https://doi.org/10.1023/A:1025079013714
Regan, H. B. (1993). Integrated portfolios as tools for differentiated teacher evaluation: A proposal. J Pers Eval Educ 7, 275-290. https://doi.org/10.1007/BF00972505
Richards, C. (2001). School inspection: A re-appraisal. Journal of Philosophy of Education, 35(4), 655-666. https://doi.org/10.1111/1467-9752.00251
Rossman, G. B., & S. Rallis, F. (2017). An introduction to qualitative research: Learning in the field. (4th ed). Sage Publications. https://doi.org/10.4135/9781071802694.
Sahin, S., Elcicek, Z., & Tosten, R. (2013). Historical development of supervision in Turkish Education System and the problems in this course of development. International Journal of Social Science, 6(5), 1105-1126. http://dx.doi.org/10.9761/JASSS1336
Saylan, N. (2013). Realization degree in functions of human resource management in special elementary and secondary schools, (Unpublished master’s thesis). Hacettepe University, Social Sciences Intitution, Ankara, Turkiye.
Scheerens, J., Ehren, M., Sleegers, P., & de Leeuw, R. (2012). OECD review on evaluation and assessment frameworks for improving school outcomes: Country background report for the Netherlands. https://www.oecd.org/education/school/nld_cbr_evaluation_ and_assessment.pdf
Schwandt, T. A., & Gates, E. F. (2018). Case study methodology. In N. K. Denzin & Y. S. Lincoln (Eds.). The Sage handbook of qualitative research, (pp. 341-354). (5th ed.). Sage Publications.
Sezgin, F., Tinmaz, A., & Tetik, S. (2017). Opinions of school principals and teachers about teacher evaluation according to performance criteria. International Journal of Human Sciences,14(2), 1647-1668. https://www.j-humansciences.com/ojs/index.php/IJHS/article/view/4557/2201
Skedsmo, G. & Huber, S. G. (2017). Governing education quality-challenges of perspective and methodology. Educ Asse Eval Acc, 29, 315-318. https://doi.org/10.1007/s11092-017-9271-3
Smith, E. S. (1997). Investigating the path mechanism linking developmental supervision and instructional efficacy using EQS and structural equation modelling, (Unpublished PhD Dissertation), University of Kansas, USA. http://proquest.umi.com/
Smylie, M. A. (2010). Continuous school improvement. Corwin Press.
Stevenson, C. (2005). Postmodernising clinical supervision in nursing. Issues in Mental Health Nursing, 26(5), 519-529. https://doi.org/10.1080/01612840590931966
Sullivan, S., & Glanz, J. (2000). Alternative approaches to supervision: Case from the field. Journal of Curriculum and Supervision, 15(3), 212-235. https://eric.ed.gov/?id=EJ603257
Taymaz, A. H. (2002). Eğitim sisteminde teftiş: kavramlar, ilkeler, yöntemler (Inspection in the education system: concepts, principles, methods). PegemA Yayıncılık.
Timmermans, A. C., de Wolf, I. F., Bosker, R. J., & Doolaard, S. (2015). Risk-based educational accountability in Dutch primary education. Educ Asse Eval Acc, 27, 323-346. http://dx.doi.org/10.1007/s11092-015-9212-y
Tonbul, Y. & Baysülen, E. (2017). An evaluation of the course inspection regulation according to the views of supervisors, teachers and principals. Elementary Education Online, 16(1), 299-311. http://dx.doi.org/10.17051/io.2017.24494
Ucar, R. (2012). The views of primary school teachers on inspection practices in their classroom. Adnan Menderes University Faculty of Education Journal of Education Sciences, 3(2), 82-96. https://dergipark.org.tr/tr/pub/aduefebder/issue/33891/375246
UNESCO. (2017). Accountability in education: Meeting our commitments. Global education monitoring report 2017/18. (2nd ed.). https://unesdoc.unesco.org/ark:/48223/ pf0000259338
Van Bruggen, J. C. (2010). Inspectorates of education in Europe. https://www.sici-inspectorates.eu/Members/Comparative-Analyses/Inspectorates-of-Education-in-Europe
VanTassel-Baska, J. (2013). Performance-based assessment. Gifted Child Today, 37(1), 41-47. https://doi.org/10.1177/1076217513509618
Yesil, D., & Kış, A. (2015). Examining the views of teachers on school principals’ classroom supervision. Inonu University Journal of the Graduate School of Education, 2(3), 27-45. https://avesis.inonu.edu.tr/yayin/585dff11-d206-435c-8dd0-488dd8e4f1d7
Yin, R. K. (2014). Case study research design and methods (5th ed.). Sage.
Zepeda, J. S. (2002). Linking portfolio development to clinical supervision: A case study. Journal of Curriculum and Supervision, 18(1), 83-102. https://eric.ed.gov/?id=EJ653639

Details

Primary Language

English

Subjects

Other Fields of Education

Journal Section

Research Article

Authors

Şenol Sezer ^*
0000-0001-8800-6017
Türkiye

Gamzegül Engin
0000-0003-3863-9785
Türkiye

Early Pub Date

February 29, 2024

Publication Date

March 1, 2024

Submission Date

March 9, 2023

Acceptance Date

January 14, 2024

Published in Issue

Year 2024 Volume: 11 Number: 2

DOI

https://doi.org/10.17275/per.24.16.11.2

IZ

https://izlik.org/JA82EF79EN

APA

Sezer, Ş., & Engin, G. (2024). Dilemmas of Teachers Related to the Supervision in Schools: A Case Study. Participatory Educational Research, 11(2), 1-23. https://doi.org/10.17275/per.24.16.11.2

AMA

1.Sezer Ş, Engin G. Dilemmas of Teachers Related to the Supervision in Schools: A Case Study. PER. 2024;11(2):1-23. doi:10.17275/per.24.16.11.2

Chicago

Sezer, Şenol, and Gamzegül Engin. 2024. “Dilemmas of Teachers Related to the Supervision in Schools: A Case Study”. Participatory Educational Research 11 (2): 1-23. https://doi.org/10.17275/per.24.16.11.2.

EndNote

Sezer Ş, Engin G (March 1, 2024) Dilemmas of Teachers Related to the Supervision in Schools: A Case Study. Participatory Educational Research 11 2 1–23.

IEEE

[1]Ş. Sezer and G. Engin, “Dilemmas of Teachers Related to the Supervision in Schools: A Case Study”, PER, vol. 11, no. 2, pp. 1–23, Mar. 2024, doi: 10.17275/per.24.16.11.2.

ISNAD

Sezer, Şenol - Engin, Gamzegül. “Dilemmas of Teachers Related to the Supervision in Schools: A Case Study”. Participatory Educational Research 11/2 (March 1, 2024): 1-23. https://doi.org/10.17275/per.24.16.11.2.

JAMA

1.Sezer Ş, Engin G. Dilemmas of Teachers Related to the Supervision in Schools: A Case Study. PER. 2024;11:1–23.

MLA

Sezer, Şenol, and Gamzegül Engin. “Dilemmas of Teachers Related to the Supervision in Schools: A Case Study”. Participatory Educational Research, vol. 11, no. 2, Mar. 2024, pp. 1-23, doi:10.17275/per.24.16.11.2.

Vancouver

1.Şenol Sezer, Gamzegül Engin. Dilemmas of Teachers Related to the Supervision in Schools: A Case Study. PER. 2024 Mar. 1;11(2):1-23. doi:10.17275/per.24.16.11.2