INNOVATIONS IN ASSESSING STUDENTS’ DIGITAL LITERACY SKILLS IN LEARNING SCIENCE: EFFECTIVE MULTIPLE CHOICE CLOSED-ENDED TESTS USING RASCH MODEL

Year 2024, Volume: 25 Issue: 3, 44 - 56, 01.07.2024

Fitria Lafifa , Dadan Rosana

https://doi.org/10.17718/tojde.1364528

Abstract

This research goal to develop a multiple-choice closed-ended test to assessing and evaluate students’ digital literacy skills. The sample in this study were students at MTsN 1 Blitar City who were selected using a purposive sampling technique. The test was also validated by experts, namely 2 Doctors of Physics and Science from Yogyakarta State University. The test instrument was developed based on five aspects of digital literacy skills: information, communication, content creation, security and problem-solving. Data have been analyzed descriptively and inferentially using the Rasch version and the assist of Quest software. The results showed that eight multiple-choice closed-ended test instruments were declared valid based on expert validation with an Aiken V value of 1.00. The reliability result is 0.97 with a very high category, and the INFIT MNSQ standard deviation value is 0.86 - 1.16, so seven items are by the Rasch model. Thus, the seven items in the multiple-choice closed-ended test instrument can be used to assessing and evaluate students’ digital literacy skills in learning science.

Keywords

Closed-ended test, digital literacy, science learning, rasch model.

Supporting Institution

Research and Community Service Program by BIMA, Ministry of Education, Culture, Research and Technology, Indonesia.

Thanks

I would like to thank those who have helped with the research and the Directorate of Research and Community Service (DRPM)

References

• AA, A.-H., EA, A.-F., IA, A., & MO, A.-R. (2005). The criteria and analysis of good multiple choice questions in a health professional setting. Saudi Medical Journal, 26(10), 1505–1510.
• Adam, R. J., & S.T., K. (1996). Acer Quest: The Interactive Test Analysis System. Victoria: The Australia Council for Educational Research.
• Adams, R., & Kho, S.-T. (1996). Acer Quest version 21. Victoria: Victoria: The Australian Council for Educational Research.
• Aiken, L. R. (1985). Three Coefficients for Analyzing the Reliability, and Validity of Ratings. Educational and Psychological Measurement, 45, 131–142.
• 54 Alexander, B., Adams Becker, S., & Cummins, M. (2016). Digital Literacy. An NMC Horizon Project Strategic Brief, 3(3), 1–16. https://doi.org/10.1038/scientificamerican0995-190