Determining the psychometric properties of middle school statistical thinking testlet-based assessment tool

Year 2023, Volume: 10 Issue: 4, 672 - 689, 23.12.2023

Lım Hooı Lian , Wun Yew

https://doi.org/10.21449/ijate.1255859

Abstract

The majority of students from elementary to tertiary levels have misunderstandings and challenges acquiring various statistical concepts and skills. However, the existing statistics assessment frameworks challenge practice in a classroom setting. The purpose of this research is to develop and validate a statistical thinking assessment tool involving form one (Grade 7) students’ statistical thinking. The SOLO model was applied to develop five testlet tasks. Each testlet task involved four components. This study employed the survey methodology to assess the statistical thinking of 356 form one students. Content validity was determined using the Content Validity Index (CVI). The construct validity was determined using Rasch analysis. The results demonstrated that the instrument for assessing the statistical thinking of the form one students was valid and trustworthy. This finding of the study also revealed new evidence that the instrument allowed the teachers to identify the students’ progress effectively based on the hierarchical manner of item levels in the testlet format. The instrument was useful in identifying students’ statistical thinking levels. The students’ ability to respond appropriately to a task at a particular level reveals their degree of cognitive development. Testlet task was also easy to diagnose the strengths and weaknesses in learning statistics topics.

Keywords

Content validity, Rasch model, SOLO model, Statistical thinking, Construct validity

Ethical Statement

Universiti Sains Malaysia, 12345-678

Supporting Institution

UNIVERSITI SAINS MALAYSIA, PENANG, MALAYSIA

Project Number

304.PGURU 6316212

Thanks

UNIVERSITI SAINS MALAYSIA, PENANG, MALAYSIA

References

• Alston-Knox, C.L., Strickland, C.M., Gazos, T., & Mengersen, K.L. (2019). Teaching and Learning in Statistics: Harnessing the power of modern statistical software to improve students statistical reasoning and thinking, Proceedings of the 5th International Conference on Higher Education Advances (HEAd’19). http://dx.doi.org/10.4995/HEAd19.2019.9239
• American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME) (2014). Standards for Educational and Psychological Testing. American Educational Research Association.
• Aisah, M.N., MazJamilah, M., Khatijahhusna, A.R., & Safwati, I. (2018). Developing statistical reasoning and thinking assessment for engineering students: Challenges and new direction. https://pdfs.semanticscholar.org/c899/c375341f3045d3bd0a5bac0536ce18694fa2.pdf
• Anastasi, A., & Urbina, S. （1997）. Psychological testing (7th ed.). Prentice Hall.
• Aoyama, K., & Stephens, M. (2003). Graph interpretation aspects of statistical literacy: Japanese perspective. Mathematics Education Research Journal, 15(3), 3-22.
• Arteage, P., Batanero, C., Contreras, J.M., & Canadas, G.R. (2015). Statistical graphs complexity and reading levels: A study with prospective teachers. Statistique et Enseignement, 6(1), 3-23.
• Barham, A.I., Ihmeideh, F., Al-Falasi, M., &Alabdallah, A. (2019). Assessment of first grade students’ literacy and numeracy levels, and the influence of key factors. International Journal of Learning, Teaching and Educational Research, 18(12), 174-195. https://www.ijlter.org/index.php/ijlter/article/view/1840/pdf
• Biggs, J.B., & Collis, K.F. (1982). Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic.
• Bilgin, A.A.B., Bulger, D., & Fung, T. (2020). Statistics: Your ticket to anywhere. Statistics Education Research Journal, 19(1), 11-20.
• Bond, T.G., & Fox, C.M. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Routledge.
• Callingham, R., & Watson, J.M. (2005). Measuring statistical literacy. Journal of Applied Measurement, 6(1), 29, 19-47.
• Chan, S.W., & Zaleha, I. (2012). The role of data technology in developing students’ statistical reasoning. Procedia-Social and Behavioral Sciences, 46, 3660-3664.
• Chan, S.W., & Zaleha, I. (2014). A technology-based statistical reasoning assessment tool in descriptive statistics for secondary school students. The Turkish Online Journal of Educational Technology, 13(1), 29-46.
• Chan, S.W., Zaleha, I., & Bambang, S. (2013). A Rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia-Social and Behavioral Sciences 59[Online], 133 – 139. http://www.sciencedirect.com/science/article/pii/S1877042814028407
• Curcio, F. (1987). Comprehension of mathematical relationships expressed in graphs. Journal for Research in Mathematics Education,18, 382–393.
• delMas, R., & Liu, Y. (2005). Exploring students’ conceptions of the standard deviation. Statistics Education Research Journal, 4(1), 55-82.
• English, L., & Watson, J. (2015). Exploring variation in measurement as a foundation for statistical thinking in the elementary school. International Journal of STEM Education, 2(3). https://doi.org/10.1186/s40594-015-0016-x
• Fergusson, A-M., G. (2022). Towards an integration of statistical and computational thinking: Development of a task design framework for introducing code-driven tools through statistical modelling [Doctor of Philosophy thesis, The University of Auckland]. https://researchspace.auckland.ac.nz/handle/2292/64664
• Garfield, J. (2002). The challenge of developing statistical reasoning. Journal of Statistics Education, 10(3). https://doi.org/10.1080/10691898.2002.11910676
• Groth, R.E. (2003). Development of a high school statistical thinking framework (Unpublished doctoral dissertation), Illinois State University.
• Ibnatul, J.F., Adibah, A.L., & Hawa, S.S. (2021). Assessing statistical literacy level of postgraduate education research students in Malaysian research universities. Turkish Journal of Computer and Mathematics Education, 12(5), 1318-1324.
• Kerka, S. (1995). Not just a number: Critical numeracy for adults (ERIC Digest No. 163, Rep. No.EDO–CE–95–163). Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education. (ERIC Document Reproduction Service No. ED 385 780).
• Krishnan, S., & Idris, N. (2014). Investigating reliability and validity for the construct of inferential statistics. International Journal of Learning, Teaching and Educational Research, 4(1), 51-60.
• Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Meas. Trans., 8 (3), p. 370.
• Linacre, J.M. (2012). A user’s guide to Winsteps Ministeps Rasch-model computer programs [version3.74.0]. http://www.winsteps.com/index.htm
• Mairing, J.P. (2020). The effect of advance statistics learning integrated Minitab and Excel with teaching teams. International Journal of Instruction, 13 (2), 141-150.
• Malaysia Ministry of Education (2017). KSSM Mathematics Form One. Putrajaya: MOE.
• Malaysia Ministry of Education (2018). KSSM Mathematics Form Two. Putrajaya: MOE.
• Malaysia Ministry of Education (2019). KSSM Mathematics Form Three. Putrajaya: MOE.
• Matthews, D., & Clark, J. (2007). Successful students’ conceptions of mean, standard deviation and the central limit theorem. http://www1.hollins.edu/faculty/clarkjm/stats1.pdf
• Mooney, E.S. (2002). A framework for characterizing middle school students’ statistical. Mathematical Thinking and Learning, 4(1). 23-63.
• Olani, A., Hoekstra, R., Harskamp, E., & van der Werf, G. (2011). Statistical reasoning ability, self-efficacy, and value beliefs in a reform-based university statistics course. Electronic Journal of Research in Educational Psychology, 9(1), 49-72.
• Pierce, R., & Chick, H. (2012). Workplace statistical literacy for teachers: Interpreting boxplots. Mathematics Education Research Journal, 25, 189 205. http://dx.doi.org/10.1007/s13394-012-0046-3
• Polit, D.F., & Beck, C.T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Res Nursing Health, 29(5), 489-497.
• Polit, D.F., Beck, C.T., & Owen, S.V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199
• Saidi, S.S., & Siew, N.M. (2019). Assessing students’ understanding of the measures of central tendency and attitude towards statistics in rural secondary schools. International Electronic Journal of Mathematics Education, 14(1), 73 86. https://doi.org/10.12973/iejme/3968
• Setambah, M.A.B., Tajudin, N.M., Yaakob, M.F.M., & Saad, M.I.M. (2019). Adventure learning in basics statistics: Impact on students critical thinking. International Journal of Instruction, 12(3), 151-166. https://doi.org/10.29333/iji.2019.12310a
• Subanji., Nusantara, T., Rahmatina, D., & Purnomo, H. (2021). The Statistical Creative Framework in Descriptive Statistics Activities. International Journal of Instruction, 14(2), 591-608. https://doi.org/10.29333/iji.2021.14233a
• Tishkovskaya, S., & Lancaster, G.A. (2010). Teaching strategies to promote statistical literacy: Review and implementation. In data and context in statistics education: towards an evidence-based society. Proceedings of the Eighth International Conference on Teaching Statistics. International Statistical Institute.
• Tishkovskaya, S., & Lancaster, G. (2012). Statistical education in the 21st century: A review of the challenges, teaching innovations and strategies for reform. Journal of Statistics Education, 20(2), 1–55.
• Van de Walle, J.A., Karp, K.S., & Bay-Williams, J.M. (2014). Elementary and middle school mathematics: Teaching developmentally (8th ed.). Pearson.
• Wild, C.J., & Pfannkuch, M. (1999), Statistical thinking in empirical enquiry, International Statistical Review, 67, 223-265.
• Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A.-R. (2015). Design and implementation content validity study: development of an instrument for measuring patient-centred communication. Journal of Caring Sciences, 4(2), 165–178. https://doi.org/10.15171/jcs.2015.017
• Zamanzadeh, V., Rassouli, M., Abbaszadeh, A., Majd, H.A., Nikanfar, A., & Ghahramanian, A. (2015). Details of content validity and objectifying it in instrument development. Nursing Practice Today, 1(3), 163–171.