Innovative approaches in assessment: Blockchain and MST example

Öz

New technologies are causing change and transformation in the field of education, as in most areas of life. This situation leads to the emergence of a number of innovative approaches in both theory and practice. The reflections of these innovative steps in the field of educational sciences on the dimension of assessment will be discussed in the framework of the research within the limits of 'Blockchain' and 'Multistage Testing (MST)'. Although blockchain technologies have only been known to the world since the beginning of the 2000s, it can be seen that they have rapidly become an important part of the language of technology. Assessing this situation in terms of educational science, the fact that blockchain technologies are discussed in reports by the Open University, EDUCAUSE and the EU Commission is the most obvious indication that the use of the technology in question is spreading rapidly today and in the near future. On the other hand, the security problems encountered in the implementation of online exams, which are used intensively in the field of education, lead to the search for new methods that can be used in these exams. In this context, the MST test administration method is considered as an alternative method by researchers, test administrators and decision makers due to its popularity and trust in major exams such as ETS, GRE, PIRLS, TIMMS and PISA. Based on this, this study aims to evaluate the contribution of the use of blockchain technologies and the MST method in assessment. As a result, MST enables personalised, highly secure precision measurement, while the blockchain structure provides the ability to securely store all data before and after the exam.

Anahtar Kelimeler

• Assessment
• Multistage Testing
• Blockchain
• Individualization
• Security

Ölçme ve değerlendirmede yenilikçi yaklaşımlar: Blokzincir ve MST örneği

Öz

Yeni teknolojiler hayatın akışı içerisinde çoğu alanda olduğu gibi eğitim bilimleri alanında da değişim ve dönüşüme sebep olmaktadır. Bu durum alanda gerek teoride gerekse uygulamada bir takım yenilikçi yaklaşımların ortaya çıkışını gündeme getirmektedir. Eğitim bilimleri alanındaki bu yenilikçi adımların ölçme ve değerlendirme boyutuna yansımaları ise araştırma kapsamında “Blokzincir” ve “Multistage Testing (MST)” sınırlılıklarında ele alınmıştır. Blokzincir teknolojileri ile dünya her ne kadar 2000’li yılların başlangıcı gibi yakın bir geçmişte tanışmış olsa da hızlı bir biçimde teknoloji dilinin önemli bir bileşeni konumuna gelmiş olduğu görülmektedir. Bu durum eğitim bilimleri açısından değerlendirildiğinde; Open University, EDUCAUSE ve Avrupa Birliği Komisyonu raporlarında blokzincir teknolojilerinin ele alınmış olması söz konusu teknolojinin günümüz ve yakın gelecekte kullanımının hızla yaygınlaştığının en belirgin göstergesidir. Diğer taraftan da alanda sıklıkla uygulanan çevrimiçi sınavlarda yaşanan güvenlik problemleri ise bu sınavlarda kullanılabilecek yeni yöntem arayışlarına yol açmaktadır. Bu kapsamda da MST test sunum yönteminin ETS, GRE, PIRLS, TIMMS ve PISA gibi önemli sınavlarda uygulanmasının getirdiği popülerlik ve sağladığı güven ile araştırmacı, sınav uygulayıcı ve karar alıcılar tarafından alternatif yöntem olarak değerlendirilmektedir. Bu temelden yola çıkarak araştırmada, blokzincir teknolojileri ile MST yöntemin ölçme ve değerlendirmede kullanımının alana katkısının değerlendirilmesi amaçlanmaktadır. Sonuç olarak MST kişiye özel yüksek güvenlikli hassas ölçüme olanak sağlarken Blokzincir yapısı ise sınav öncesi ve sonrasındaki tüm verileri güvenli biçimde depolayabilme olanağı sunmaktadır.

Anahtar Kelimeler

• Ölçme ve Değerlendirme
• Çok Aşamalı Testler
• Blokzincir
• Bireyselleştirme
• Güvenlik.

Kaynakça

1. Adedoyin, O. & Mokobi, T. (2013). Using IRT psychometric analysis in examining the quality of junior certificate mathematics multiple choice examination test items. International Journal of Asian Social Science, 3(4), 992–1011. Erişim adresi: https://archive.aessweb.com/index.php/5007/article/view/2471
2. Beck, R. (2018). Beyond bitcoin: The rise of blockchain world. Computer, 51(2), 54-58. Erişim adresi: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8301120
3. Berger, S., Verschoor, A. J., Eggen, T. J. H. M. & Moser, U. (2019). Improvement of measurement efficiency in multistage tests by targeted assignment. Frontiers in Education, 4(January). https://doi.org/10.3389/feduc.2019.00001
4. Chang, H.-H., & Ying, Z. (2008). To weight or not to weight? Balancing influence of initial items in adaptive testing. Psychometrika, 73(3), 441–450.
5. Chen, G., Xu, B., Lu, M. & Chen, N.-S. (2018). Exploring blockchain technology and its potential applications for education. Smart Learning Environments, 5(1), 1. https://doi.org/10.1186/s40561-017-0050-x
6. Choi, S. W., Lim, S., & van der Linden, W. J. (2022). TestDesign: an optimal test design approach to constructing fixed and adaptive tests in R. Behaviormetrika, 49(2), 191-229. https://doi.org/10.1007/s41237-021-00145-9
7. Cullen, J. & Mallet, J. (2019). The Opportunities and Challenges for Developing ICT-Based Science Learning and Teaching in Ghana. Pan-Commonwealth Forum, 7. http://oasis.col.org/bitstream/handle/11599/3276/PCF9_Papers_paper_232.pdf?sequence=1&isAllowed=y Ducas, E. & Wilner, A. (2017). The security and financial implications of blockchain technologies: Regulating emerging technologies in Canada. International Journal, 72(4), 538–562. https://doi.org/10.1177/0020702017741909
8. Dünya Ekonomik Forumu (2017). Blokzincirin potansiyelini fark etmek. Erişim adresi: http://www3.weforum.org/docs/WEF_Realizing_Potential_Blockchain.pdf

9. EDUCAUSE (2019). EDUCAUSE horizon report: 2019 higher education edition. Erişim adresi: https://library.educause.edu/-/media/files/library/2019/4/2019horizonreport.pdf? la=en&hash=C8E8D444AF372E705FA1BF9D4FF0DD4CC6F0FDD1
10. Funk, E., Riddell, J., Ankel, F. & Cabrera, D. (2018). Blockchain technology: a data framework to improve validity, trust, and accountability of information exchange in health professions education. Academic Medicine, 93(12), 1791-1794. Doi: 10.1097/ACM.0000000000002326
11. Gartner, (2021). Hype cycle for blockchain 2021; more action than hype. Erişim adresi: https://blogs.gartner.com/avivah-litan/2021/07/14/hype-cycle-for-blockchain-2021-more-action-than-hype/
12. Gartner, (2020). Hype cycle for blockchain technologies, 2020. Erişim adresi: https://www.gartner.com/en/documents/3987450/hype-cycle-for-blockchain-technologies-2020
13. Gartner, (2019). Gartner top 10 strategic technology trends for 2020. Erişim adresi: https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2020/
14. Gartner, (2019). The 4 phases of the Gartner Blockchain Spectrum. Erişim adresi: https://www.gartner.com/smarterwithgartner/the-4-phases-of-the-gartner-blockchain-spectrum/
15. Gartner, (2018). Gartner top 10 strategic technology trends for 2019. Erişim adresi: https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2019/
16. Gartner, (2017). Gartner top 10 strategic technology trends for 2018. Erişim adresi: https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2018/
17. Gartner, (2016). Gartner top 10 strategic technology trends for 2017. Erişim adresi: https://www.gartner.com/smarterwithgartner/gartners-top-10-technology-trends-2017/
18. Gräther, W., Kolvenbach, S., Ruland, R., Schütte, J., Torres, C. & Wendland, F. (2018). Blockchain for Education: Lifelong Learning Passport. In: W. Prinz & P. Hoschka (Eds.), Proceedings of the 1st ERCIM Blockchain Workshop 2018, Reports of the European Society for Socially Embedded Technologies (ISSN 2510-2591), Doi: 10.18420/blockchain2018_07
19. Grech, A. & Camilleri, A. F. (2017). Blockchain in education. doi: https://doi.org/10.2760/60649
20. Guo, J., Li, C., Zhang, G., Sun, Y. & Bie, R. (2020). Blockchain-enabled digital rights management for multimedia resources of online education. Multimedia Tools and Applications, 79(15), 9735-9755. Doi: https://doi.org/10.1007/s11042-019-08059-1
21. Hambleton, R. K. & Xing, D. (2006). Optimal and nonoptimal computer-based test designs for making pass–fail decisions. Applied Measurement in Education, 19(3), 221-239.
22. Hambleton, R. K. Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory library (1st ed.; D. Foster, ed.). London: SAGE.
23. Hambleton, R. K. & Linden, W. J. (1997). Handbook of modern item response theory (1st ed.). USA: Springer. https://doi.org/10.1007/978-1-4757-2691-6
24. Hendrickson, A. (2007). An NCME instructional module on multistage testing. Educational Measurement: Issues and Practice, 26(2), 44-52. https://doi.org/10.1111/j.1745-3992.2007.00093.x
25. Jodoin, M. G., Zenisky, A. & Hambleton, R. K. (2006). Comparison of the psychometric properties of several computer-based test designs for credentialing exams with multiple purposes. Applied Measurement in Education, 19(3), 203-220.
26. Keng, L. (2008). A comparison of the performance of testlet-based computer adaptive tests and multistage tests (Doctoral dissertation, The University of Texas at Austin).
27. Khorramdel, L., Pokropek, A., Joo, S. H., Kirsch, I., & Halderman, L. (2020). Examining gender DIF and gender differences in the PISA 2018 reading literacy scale: A partial invariance approach. Psychological Test and Assessment Modeling, 62(2), 179-231. Erişim adresi: https://www.researchgate.net/profile/Artur-Pokropek/publication/342344680_Examining_gend er_DIF_and_gender_differences_in_the_PISA_2018_reading_literacy_scale_A_partial_invariance_approach/links/5eef16eb458515814a71a36e/Examining-gender-DIF-and-gender-differenc es-in-the-PISA-2018-reading-literacy-scale-A-partial-invariance-approach.pdf?origin=journal Detail&_tp=eyJwYWdlIjoiam91cm5hbERldGFpbCJ9
28. Li, H. & Han, D. (2019). EduRSS: A blockchain-based educational records secure storage and sharing scheme. IEEE Access, 7, 179273-179289. Doi: 10.1109/ACCESS.2019.2956157 Lizcano, D., Lara, J. A., White, B. & Aljawarneh, S. (2020). Blockchain-based approach to create a model of trust in open and ubiquitous higher education. Journal of Computing in Higher Education, 32(1), 109-134. Doi: https://doi.org/10.1007/s12528-019-09209-y
29. Ma, Y. & Fang, Y. (2020). Current status, issues, and challenges of blockchain applications in education. International Journal of Emerging Technologies in Learning (iJET), 15(12), 20-31. Doi: https://doi.org/10.3991/ijet.v15i12.13797
30. Magis, D., Yan, D. & Davier A. A. (2017). Computerized adaptive and multistage testing with R: Using Packages catR and mstR (1st ed.). USA: Springer. doi: 10.1007/978-3-319-69218-0
31. MacGregor, D., Yen, S. J., & Yu, X. (2022). Using multistage testing to enhance measurement of an english language proficiency test. Language Assessment Quarterly, 19(1), 54-75. https://doi.org/10.1080/15434303.2021.1988953
32. Mullis, I. V., & Martin, M. O. (2019). PIRLS 2021 Assessment Frameworks. International Association for the Evaluation of Educational Achievement. Herengracht 487, Amsterdam, 1017 BT, The Netherlands. Erişim adresi: https://files.eric.ed.gov/fulltext/ED606056.pdf
33. Nakamoto, S. (2008). Bitcoin: A peer-to-peer electronic cash system. Erişim adresi: www.bitcoin.org.https://doi.org/10.1007/s10838-008-9062-0
34. Nath, I. (2016, December). Data exchange platform to fight insurance fraud on blockchain. In 2016 IEEE 16th international conference on data mining workshops (ICDMW) (pp. 821-825). IEEE Computer Society. Doi: 10.1109/ICDMW.2016.144
35. Nofer, M., Gomber, P., Hinz, O., & Schiereck, D. (2017). Blockchain. Business & Information Systems Engineering, 59, 183-187. Doi: 10.1007/s12599-017-0467-3
36. Ockey, G. J. (2012). Item Response theory. G. Fulcher ve F. Davidson (Editörler), The routledge handbook of language testing içinde (s. 336-349). New York/London: Routledge.
37. Ogrutan, P. L. (2020). An analysis on the opportunity of introducing blockchain technology in Education-A case study. TEM Journal, 9(3), 971-976. Erişim adresi: https://www.ceeol.com/search/article-detail?id=894547
38. Open University (2016). Innovating Pedagogy 2016: Exploring new forms of teaching, learning and assessment, to guide educators and policy makers (Report 5). Erişim adresi: https://ou-iet.cdn.prismic.io/ou-iet/bfb6639f-3132-402f-8a14-db38c206f316_innovating_pedagogy_2016.pdf
39. ÖSYM (2024). Ölçme ve değerlendirmede yenilikçi yaklaşımlar. Erişim tarihi: https://symp2024.osym.gov.tr/Symposium/
40. Palma, L. M., Vigil, M. A., Pereira, F. L. & Martina, J. E. (2019). Blockchain and smart contracts for higher education registry in Brazil. International Journal of Network Management, 29(3), 1-21. doi: 10.1002/nem.2061
41. Reyna, A., Martín, C., Chen, J., Soler, E., & Díaz, M. (2018). On blockchain and its integration with IoT. Challenges and opportunities. Future Generation Computer Systems, 88, 173-190. https://doi.org/10.1016/j.future.2018.05.046
42. Rotou, O., Patsula, L., Steffen, M. ve Rizavi, S. (2007). Comparison of multistage tests with computerized adaptive and paper-and-pencil tests (Rapor No: 07-04). New Jersey: Educational Testing Service (ETS). https://doi.org/10.1002/j.2333-8504.2007.tb02046.x
43. Sarı, H. İ. (2020). Testing multistage testing configurations: Post-Hoc vs. hybrid simulations. International Journal of Psychology and Educational Studies, 7(1), 27-37. Erişim adresi: https://dergipark.org.tr/en/download/article-file/1335540
44. Sarı, H. İ., & Huggins-Manley, A. C. (2017). Examining content control in adaptive tests: computerized adaptive testing vs. computerized adaptive multistage testing. Educational Sciences: Theory & Practice, 17(5), 1759-1781. doi: 10.12738/estp.2017.5.0484
45. Tang, X., Zheng, Y., Wu, T., Hau, K. T. & Chang, H. H. (2024). Utilizing response time for ıtem selection in on‐the‐fly multistage adaptive testing for PISA assessment. Journal of Educational Measurement, 0(0), 1-28. doi: 10.1111/jedm.12403
46. Tanrıverdi, M., Uysal, M. & Üstündağ, M. T. (2019). Blokzinciri teknolojisi nedir? Ne değildir?: Alanyazın incelemesi. Bilişim Teknolojileri Dergisi, 12(3), 203-217. doi: 10.17671/gazibtd.547122
47. The Open University (2024). Innovating Peagogy 2024. Erişim adresi: https://iet.open.ac.uk/files/innovating-pedagogy-2024.pdf
48. The Open University (2023). Innovating Peagogy 2023. Erişim adresi: https://prismic-io.s3.amazonaws.com/ou-iet/4acfab6d-4e5c-4bbd-9bda-4f15242652f2_Innovating+Ped agogy+2023.pdf
49. The Open University (2022). Innovating Peagogy 2022. Erişim adresi: https://prismic-io.s3.amazonaws.com/ou-iet/5c334004-5f87-41f9-8570-e5db7be8b9dc_innovating-pedagogy-2022.pdf
50. The Open University (2017). Innovating Peagogy 2017. Erişim adresi: https://ou-iet.cdn.prismic.io/ou-iet/c80c19d1-3a86-4bae-96db-e198af6b8784_innovating-pedagogy-2017.pdf
51. The Open University (2016). Innovating Peagogy 2016. Erişim adresi: https://ou-iet.cdn.prismic.io/ou-iet/bfb6639f-3132-402f-8a14-db38c206f316_innovating_pedago gy_2016.pdf
52. The Open University (2014). Innovating Peagogy 2014. Erişim adresi: https://oro.open.ac.uk/94047/1/innovating-pedagogy-2014.pdf
53. The Open University (2013). Innovating Peagogy 2013. Erişim adresi: https://ou-iet.cdn.prismic.io/ou-iet/ccc86d85-4101-4c12-aef3-26174411e489_innovating-pedagog y-2013.pdf
54. The Open University (2012). Innovating Peagogy 2012. Erişim adresi: https://ou-iet.cdn.prismic.io/ou-iet/59abb4d3-2e6e-4791-91cf-96e2c0059f62_innovating-pedagogy-2012.pdf
55. TÜBİTAK (2024). Eğitimde ölçme ve değerlendirme alanında yenilikçi yaklaşımların uygulaması: Teoriden pratiğe, akademiden sahaya. Erişim adresi: https://tubitak.gov.tr/tr/etkinlikler/egitimde-olcme-ve-degerlendirme-alaninda-yenilikci -yaklasimlarin-uygulamasi-teoriden-pratige-akademiden-sahaya
56. TÜBİTAK BİLGEM (2021). Blokzincir araştırma laboratuvarı. Erişim adresi: https://blockchain.bilgem.tubitak.gov.tr
57. Udeozor, C., Chan, P., Russo Abegão, F. & Glassey, J. (2023). Game-based assessment framework for virtual reality, augmented reality and digital game-based learning. International Journal of Educational Technology in Higher Education, 20(1), 36. https://doi.org/10.1186/s41239-023-00405-6
58. Weiss, D. J. ve Kingsbury, G. G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21(4), 361-375. Erişim adresi: https://www.jstor.org/stable/pdf/1434587.pdf
59. Williams, P. (2019). Does competency-based education with blockchain signal a new mission for universities?. Journal of Higher Education Policy and Management, 41(1), 104-117. Doi: https://doi.org/10.1080/1360080X.2018.1520491 Wu, X. (2020). Research on english online education platform based on genetic algorithm and blockchain technology. Wireless Communications and Mobile Computing, 2020, 1-7. doi: https://doi.org/10.1155/2020/8827084
60. Yan, D., Davier, A. A. & Lewis, C. (2014). Computerized multistage testing: Theory and application (1st ed.). USA: CRC Press. doi: 10.1201/b16858
61. Yin, L. & Foy, P. (2021). TIMSS 2023 Assessment Design. TIMSS 2023 Assessment Frameworks, 71. Erişim adres: https://pirls.bc.edu/timss2023/frameworks/pdf/T23_Frameworks.pdf#page=75
62. Zenisky, A., Hambleton, R. K. & Luecht, R. M. (2009). Multistage testing: Issues, designs, and research. In Elements of adaptive testing (pp. 355-372). New York: Springer.
63. Zhao, G., Di, B., He, H. & Zhu, W. (2020). Digital education transaction object authentication service based on blockchain technology. Internet Technology Letters, 3(2), 1-6. Doi: https://doi.org/10.1002/itl2.149
64. Zhao, J. L., Fan, S., & Yan, J. (2016). Overview of business innovations and research opportunities in blockchain and introduction to the special issue. Financial Innovation, 2, 1-7. doi 10.1186/s40854-016-0049-2
65. Zheng, Y., & Chang, H. H. (2015). On-the-fly assembled multistage adaptive testing. Applied Psychological Measurement, 39(2), 104–118. https://doi.org/10.1177/0146621614544519
66. Zheng, Z., Xie, S., Dai, H. N., Chen, X., & Wang, H. (2018). Blockchain challenges and opportunities: A survey. International Journal of Web and Grid Services, 14(4), 352-375. Erişim adresi: https://allquantor.at/blockchainbib/pdf/zheng2018blockchain.pdf