Sıralı Yanıt Kategorili Gerçek İş Yükü Anketinin Doğruluğunun Değerlendirilmesi: Güvenilirlik için Bir Bootstrap Örnekleme
Abstract
Bologna Deklarasyonu (19 Haziran 1999), üniversite derecelerinin müfredatlarının dönüştürülmesini sağlayan Avrupa Yüksek Öğretim Alanı’nın (AYÖA) uygulamalarını motive etti. Türkiye’de yükseköğretimde kalite güvencesi ağırlıklı olarak Türkiye’nin 2001 yılında Bologna Süreci’ne katılmasından sonra gelişmeye başlamıştır. Bologna reformları ile ilgili çaba ve faaliyetler özellikle 2005 yılından sonra hız kazanmıştır. AYÖA çerçevesinde, öğrenci iş yükü, belirli bir dersi geçmek için gereken toplam sınıf içi ve bağımsız yapılandırılmamış çalışmanın sayısal tanımlayıcı bir değeri olan Avrupa Kredi Transfer Sistemi (AKTS) aracılığıyla belirlenir. AKTS’nin hesaplanmasında birincil girdi olan öğrencinin fiili iş yükü, genellikle öğretim elemanının ders dışı zamanlarda öğrenciden beklentileri dikkate alınarak hesaplanır. Bazı üniversitelerde öğretim elemanları aldıkları dersle ilgili fiili iş yükü anketini yapmak ve sonuçları bölümleri ile paylaşmakla yükümlüdürler. Bazı öğretim elemanları için böyle bir yaptırım olsa da çoğu üniversitede öğrencilerin gerçek iş yükü doğrudan öğrencilere sorulmadan tahmin edilmektedir. Bu çalışmada, öğrencilerin fiili iş yüklerini hesaplamak için yapılandırılmış bir anket tasarlanması ve bu anketten elde edilen sonucun güvenilirliğinin istatistiksel olarak kanıtlanması amaçlanmıştır. Bu amaçla 3 ders için sıralı yanıt kategorisine sahip gerçek iş yükü anketi öğrencilere uygulanmış, aynı anketin açık uçlu versiyonu yine aynı öğrencilere yüz yüze sorulmuş ve sonuçlar karşılaştırılmıştır. Ayrıca, bu karşılaştırmanın doğruluğunu değerlendirmek ve sonuçların popülasyona genellenebilirliğini sağlamak için, anketlerden alınan yanıtların ampirik dağılımları dikkate alınarak, bootstrap örnekleme yöntemi kullanılmıştır.
References
- Canty, A., & Ripley, B. (2022). Boot: Bootstrap R (S-Plus) functions (R package version 1.3-28.1). https://brieger.esalq.usp.br/CRAN/web/packages/boot/boot.pdf
- Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397), 171–185. https://doi.org/10.1080/01621459.1987.10478410
- Egea, G., Rodríguez-Lizana, A., Pérez-Urrestarazu, L., Pérez-Ruiz, M., Rallo, P., & Suárez, M. P. (2022). Assessment of actual workload and student performance in the agricultural engineering final degree project in a Spanish higher education context. Education Sciences, 12(6), 418. https://doi.org/10.3390/educsci12060418
- European Commission. (2015). ECTS users’ guide. https://doi.org/10.2766/87592 https://op.europa.eu/en/publication-detail/-/publication/da7467e6-8450-11e5-b8b7-01aa75ed71a1
- Gamer, M., Lemon, J., Fellows, I., & Singh, P. (2022). irr: Various coefficients of interrater reliability and agreement (R package version 0.84.1). https://cran.r-project.org/web/packages/irr/irr.pdf
- Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
- Souto-Iglesias, A., & Baeza-Romero, M. T. (2018). A probabilistic approach to student workload: Empirical distributions and ECTS. Higher Education, 76(2), 1007–1025. https://doi.org/10.1007/s10734-018-0244-3
- Turkish Higher Education Quality Council. (2022a, January). What is institutional accreditation program? https://yokak.gov.tr/degerlendirme-sureci/kurumsal-akreditasyon-programi-nedir
- Turkish Higher Education Quality Council. (2022b, January). Accredited institutions and reports. https://yokak.gov.tr/raporlar/akredite-olan-kurumlar
- Turkish Higher Education Quality Council. (2022c, January). The 2021 higher education evaluation and quality assurance status report. https://yokak.gov.tr/Common/Docs/Site_Activity_Reports/DurumRaporu2021.pdf
Evaluating the Accuracy of Actual Workload Questionnaire with Ordinal Response Category: Bootstrap Sampling for Reliability
Abstract
The Bologna Declaration (19 June 1999) motivated the implementation of the European Higher Education Area (EHEA), which led to the transformation of the curricula of the university degrees. In Türkiye, quality assurance in higher education was focused mainly after the participation of Türkiye in the Bologna Process in 2001. However, efforts and activities related to Bologna reforms gained momentum particularly after 2005. Within the EHEA framework, the student workload is set through the European Credit Transfer and Accumulation System (ECTS), which is a numerical descriptive value of the total in-class and independent unstructured work required to pass a particular course. The actual workload of the student, which is the primary input in the calculation of ECTS, is generally calculated by taking into account the lecturer’s expectations from the student during extracurricular times. In some universities, lecturers are obliged to conduct the actual workload questionnaire related to the course they are taking and share the results with their departments. Although there is such a sanction for some lecturers, the actual workload of students is estimated in most universities without asking students directly. In this study, it is aimed to design a structured questionnaire to calculate the actual workload of students and to statistically prove the reliability of the result obtained from this questionnaire. For this purpose, actual workload survey results with ordinal response categories for 3 courses were obtained and the open-ended version of the same questionnaire was asked to the same students again face to face and the results were compared. In addition, to evaluate the accuracy of this comparison and the generalizability of the results to the population was questioned using the bootstrap sampling method, taking into account the empirical distributions of responses got from questionnaires.
References
- Canty, A., & Ripley, B. (2022). Boot: Bootstrap R (S-Plus) functions (R package version 1.3-28.1). https://brieger.esalq.usp.br/CRAN/web/packages/boot/boot.pdf
- Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397), 171–185. https://doi.org/10.1080/01621459.1987.10478410
- Egea, G., Rodríguez-Lizana, A., Pérez-Urrestarazu, L., Pérez-Ruiz, M., Rallo, P., & Suárez, M. P. (2022). Assessment of actual workload and student performance in the agricultural engineering final degree project in a Spanish higher education context. Education Sciences, 12(6), 418. https://doi.org/10.3390/educsci12060418
- European Commission. (2015). ECTS users’ guide. https://doi.org/10.2766/87592 https://op.europa.eu/en/publication-detail/-/publication/da7467e6-8450-11e5-b8b7-01aa75ed71a1
- Gamer, M., Lemon, J., Fellows, I., & Singh, P. (2022). irr: Various coefficients of interrater reliability and agreement (R package version 0.84.1). https://cran.r-project.org/web/packages/irr/irr.pdf
- Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
- Souto-Iglesias, A., & Baeza-Romero, M. T. (2018). A probabilistic approach to student workload: Empirical distributions and ECTS. Higher Education, 76(2), 1007–1025. https://doi.org/10.1007/s10734-018-0244-3
- Turkish Higher Education Quality Council. (2022a, January). What is institutional accreditation program? https://yokak.gov.tr/degerlendirme-sureci/kurumsal-akreditasyon-programi-nedir
- Turkish Higher Education Quality Council. (2022b, January). Accredited institutions and reports. https://yokak.gov.tr/raporlar/akredite-olan-kurumlar
- Turkish Higher Education Quality Council. (2022c, January). The 2021 higher education evaluation and quality assurance status report. https://yokak.gov.tr/Common/Docs/Site_Activity_Reports/DurumRaporu2021.pdf
Details
| Primary Language | English |
|---|---|
| Subjects | Quality Assurance in Higher Education |
| Journal Section | Araştırma Makalesi |
| Authors | |
| Early Pub Date | July 2, 2025 |
| Publication Date | June 30, 2025 |
| Submission Date | August 27, 2024 |
| Acceptance Date | January 1, 2025 |
| Published in Issue | Year 2025 Volume: 15 Issue: Special Issue |
