The development of diagnostic test instrument for mathematical representation ability (PhysDTRA) in high school physics learning

Abstract

Assessment in education is part of collecting and processing various information related to student achievements during learning. Good learning quality can be seen from the quality of the assessment. Assessment activities that help educators to find out the difficulty of students on a learning material so that it can be guided to achieve completeness criteria, namely by diagnostic tests. The assessment carried out must follow 21st-century learning that is integrated with the industrial era 4.0, namely how to diagnose difficulties related to the representation ability of students in physics learning by utilizing assessments that have advantages in detecting student difficulties and giving suggestions for appropriate improvement. The purpose of this research was to develop a test instrument (PhysDTRA) that could be used to diagnose students' mathematical representation abilities in high school physics learning. The results of the research were analyzed quantitatively and qualitatively using Item Response Theory (IRT). Based on the content validity, the PhysDTRA instrument was declared to be valid according to the expert judgments who were analyzed using the Aiken's V equation. All items in the test instrument were valid based on the Rasch, INFIT MNSQ, and INFIT t models. The PhysDTRA instrument has also been relied upon based on the reliability of the estimated items and TIC curves so that it can be used to diagnose and determine the profile of students' mathematical representation abilities. Thus, the PhysDTRA instrument developed has fulfilled the test characteristics that are feasible of its content, empirical evidence, validity, and reliability.

Keywords

• Diagnostic Instrument
• Mathematical Representation
• Work and Energy
• Validity
• Reliability
• Development of diagnostic test

References

1. Adams, R. J., & Khoo, S.-T. (1996). QUEST: The interactive test analysis system version 2.1. Australian Council for Educational Research.
2. Adams, W. K., Wieman, C. E., & Adams, W. K. (2015). Analyzing The Many Skills Involved in Solving Complex Physics Problems Analyzing The Many Skills Involved in Solving complex Physics Problems. American Journal of Physics, 83(459). https://doi.org/10.1119/1.4913923
3. Adlina, A., & Supahar. (2019). Developing android assisted worked example application on kinematics (weak) to improve mathematical representation ability in high school physics learning. International Journal of Scientific and Technology Research, 8(10), 3790–3793.
4. Aiken, L. R. (1985). Three Coefficients for Analysing Reliability and Validity of Ratings. Educational and Psychological Measurement, 45, 131–142. https://doi.org/10.1177/07399863870092005
5. Albe, V., Venturini, P., & Lascours, J. (2014). Electromagnetic Concepts in Mathematical Representation of Physics. Journal of Science Education and Technology, 10, 197–203.
6. Angell, C., Kind, P. M., Henriksen, E. K., & Guttersrud, Ø. (2008). An Empirical-Mathematical Modelling Approach to Upper Secondary Physics. Physics Education, 43(3), 256–264. https://doi.org/10.1088/0031-9120/43/3/001
7. Atkin, J. M., & Coffey, J. E. (2003). Everyday Assessment in The Science Classroom (Issue c). National Science Teachers Association.
8. Azwar, S. (2012). Penyusunan Skala Psikologi (Psychological Scale Compilation). Pustaka Pelajar.

9. Azwar, S. (2015). Reliability and Validity (fourth). Pustaka Pelajar.
10. Bing, T. J., & Redish, E. F. (2014). The Cognitive Blending of Mathematics and Physics Knowledge. AIP Conference Proceedings, 883(February 2007), 26–29. https://doi.org/10.1063/1.2508683
11. Bollen, L., van Kampen, P., Baily, C., Kelly, M., & De Cock, M. (2017). Student difficulties regarding symbolic and graphical representations of vector fields. Physical Review Physics Education Research, 13(2), 020109. https://doi.org/10.1103/PhysRevPhysEducRes.13.020109
12. Ceuppens, S., Deprez, J., Dehaene, W., & De Cock, M. (2018). Design and validation of a test for representational fluency of 9th grade students in physics and mathematics: The case of linear functions. Physical Review Physics Education Research, 14(2), 20105. https://doi.org/10.1103/PhysRevPhysEducRes.14.020105
13. Darmawan, D., Yatimah, D., Sasmita, K., & Syah, R. (2020). Analysis of Non-Formal Education Tutor Capabilities in Exploring Assessment for Science Learning. Jurnal Pendidikan IPA Indonesia, 9(2), 267–275. https://doi.org/10.15294/jpii.v9i2.24025
14. De Cock, M. (2012). Representation Use and Strategy Choice in Physics Problem Solving. Physical Review Special Topics - Physics Education Research, 8(2), 1–15. https://doi.org/10.1103/PhysRevSTPER.8.020117
15. Docktor, J. L., & Mestre, J. P. (2014). Synthesis of Discipline-Based Education Research in Physics. Physical Review Special Topics - Physics Education Research, 10(2), 1–58. https://doi.org/10.1103/PhysRevSTPER.10.020119
16. du Toit, M. (2003). IRT from SSI: Bilog-Mg, Multilog, Parscale, Testfact. Scientific Software International, Inc.
17. Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138
18. George, D., & Mallery, P. (2020). IBM SPSS Statistics 26 Step by Step: A Simple Guide and Reference (Sixteenth). Routledge. https://doi.org/10.4324/9780429056765
19. Gurcay, D., & Gulbas, E. (2015). Development of Three-Tier Heat, Temperature and Internal Energy Diagnostic Test. Research in Science and Technological Education, 33(2), 197–217. https://doi.org/10.1080/02635143.2015.1018154
20. Gurel, D. K., Eryilmaz, A., & McDermott, L. C. (2015). A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a
21. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory Library. Sage Publication, Inc.
22. Heuvelen, A. Van, & Zou, X. (2001). Multiple representations of work–energy processes. American Journal of Physics, 69(2), 184–194. https://doi.org/10.1119/1.1286662
23. Ibrahim, B., & Rebello, N. S. (2012). Representational Task Formats and Problem Solving Strategies in Kinematics and Work. Physical Review Special Topics - Physics Education Research, 8(1), 1–19. https://doi.org/10.1103/PhysRevSTPER.8.010126
24. Johnson, R. B., & Christensen, L. B. (2016). Educational Research: Quantitative, Qualitative, and Mixed Approaches (Sixth). SAGE Publication.
25. Kim, S. (2006). A comparative study of IRT fixed parameter calibration methods. Journal of Educational Measurement, 43(4), 355–381. https://doi.org/10.1111/j.1745-3984.2006.00021.x
26. Kirschner, S., Borowski, A., Fischer, H. E., Gess-Newsome, J., & von Aufschnaiter, C. (2016). Developing and evaluating a paper-and-pencil test to assess components of physics teachers’ pedagogical content knowledge. International Journal of Science Education, 38(8), 1343–1372. https://doi.org/10.1080/09500693.2016.1190479
27. Krawec, J. L. (2014). Problem Representation and Mathematical Problem Solving of Students of Varying Math Ability. Journal of Learning Disabilities, 47(2), 103–115. https://doi.org/10.1177/0022219412436976
28. Kriek, J., & Koontse, R. D. (2017). First year physics students’ expectations of the role of mathematics in physics. International Journal of Innovation in Science and Mathematics Education, 25(2), 1–16.
29. Liang, L. L., Fulmer, G. W., Majerich, D. M., Clevenstine, R., & Howanski, R. (2012). The Effects of a Model-Based Physics Curriculum Program with a Physics First Approach: A Causal-Comparative Study. Journal of Science Education and Technology, 21(1), 114–124. https://doi.org/10.1007/s10956-011-9287-2
30. Lissitz, R. W., & Samuelsen, K. (2007). Further Clarification Regarding Validity and Education. Educational Researcher, 36(8), 482–484. https://doi.org/10.3102/0013189x07311612
31. Loewenthal, K. M., & Lewis, C. A. (2018). An introduction to psychological tests and scales. Psychology Press.
32. Minarni, A., & Napitupulu, E. E. (2017). Developing Instruction Materials Based on Joyful PBL to Improve Students Mathematical Representation Ability. International Education Studies, 10(9), 23. https://doi.org/10.5539/ies.v10n9p23
33. Niss, M. (2016). Obstacles Related to Structuring for Mathematization Encountered by Students when Solving Physics Problems. International Journal of Science and Mathematics Education, 15(8), 1441–1462. https://doi.org/10.1007/s10763-016-9754-6
34. Nitko, A. J., & Brookhart, S. M. (2011). Educational Assessment of Students (Sixth Edit). Pearson Education, Inc. OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. In OECD Publishing: Vol. II. https://doi.org/https://doi.org/10.1787/b5fd1b8f-en
35. Oriondo, L. L., & Dallo-Antonio, E. M. (1984). Evaluating Educational Outcomes (Test, Measurement, and Evaluation) (5th Editio). REX Printing Company, Inc.
36. Pape, S. J., & Tchoshanov, M. A. (2001). The Role of Representation(s) in Developing Mathematical Understanding. Theory Into Practice, 40(2), 118–127. https://doi.org/10.1207/s15430421tip4002_6
37. Park, E.-J., & Choi, K. (2013). Analysis of Student Understanding of Science Concepts Including Mathematical Representations: pH Values and The Relative Differences of pH Values. International Journal of Science and Mathematics Education, 11(3), 683–706. https://doi.org/10.1007/s10763-012-9359-7
38. Permendikbud No. 22 Tahun 2016 tentang Standar Proses Pendidikan Dasar dan Menengah (Minister of Education and Culture Regulation No. 22 of 2016 on Process Standards for Primary and Secondary Education), (2016).
39. Planinic, M., Ivanjek, L., Susac, A., & Milin-Sipus, Z. (2013). Comparison of university students’ understanding of graphs in different contexts. Physical Review Special Topics - Physics Education Research, 9(2), 1–9. https://doi.org/10.1103/PhysRevSTPER.9.020103
40. Pospiech, G., Eylon, B., Bagno, E., Lehavi, Y., & Geyer, M. A. (2016). The Role of Mathematics for Physics Teaching and Understanding. Nuovo Cimento C Geophysics Space Physics C, 38(3), 10. https://doi.org/https://doi.org/10.1393/ncc/i2015-15110-6
41. Pujayanto, Budiharti, R., Radiyono, Y., Rizky, N., Nuraini, A., Putri, H. V., Saputro, D. E., & Adhitama, E. (2018). Pengembangan Tes Diagnostik Miskonsepsi Empat Tahap Tentang Kinematika (Developing Four-Tier Misconception Diagnostic Test about Kinematics). Cakrawala Pendidikan, 37(2), 237–249. https://doi.org/10.21831/cp.v37i2.16491
42. Retnawati, H. (2014). Teori Respon Butir dan Penerapannya: untuk Peneliti, Praktisi Pengukuran dan Pengujian, Mahasiswa Pascasarjana (Item Response Theory and Its Application: for Researchers, Measurement and Testing Practitioners, Postgraduate Students). Nuha Medika.
43. Retnawati, H. (2016). Analisis Kuantitatif Instrumen Penelitian: Panduan Peneliti, Mahasiswa, dan Psikometrian (Quantitative Analysis of Research Instruments: A Guide for Researchers, Students, and Psychometrics). Parama Publishing.
44. Sadaghiani, H. R. (2011). Using multimedia learning modules in a hybrid-online course in electricity and magnetism. Physical Review Special Topics - Physics Education Research, 7(1), 1–7. https://doi.org/10.1103/PhysRevSTPER.7.010102
45. Subali, B., & Pujiati, S. (2012). Pengembangan Item Tes Konvergen dan Divergen dan Penyelidikan Validitasnya secara Empiris (Development of Convergent and Divergent Test Item and Empirical Validity Investigation). Diandra Pustaka Indonesia.
46. Subali, Bambang, & Suyata, P. (2011). Panduan Analisis Data Pengukuran Pendidikan Untuk Memperoleh Bukti Empirik Kesahihan menggunakan program Quest (Guidelines for Educational Measurement Data Analysis to Obtain Empiric Proof of Eligibility using the Quest Program). Lembaga Penelitian dan Pengabdian pada Masyarakat Universitas Negeri Yogyakarta.
47. Supahar, S., & Prasetyo, Z. K. (2015). Pengembangan Instrumen Penilaian Kinerja Kemampuan Inkuiri Peserta Didik Pada Mata Pelajaran Fisika SMA (Developing Assessment Instrument of The Performance of The Inquiry Abilities in Physics Subject of High School Students). Jurnal Penelitian Dan Evaluasi Pendidikan, 19(1), 96–108. https://doi.org/10.21831/pep.v19i1.4560
48. Swaminathan, H., Hambleton, R. K., & Rogers, H. J. (2007). Assessing the Fit of Item Response Theory Models. In Handbook of Statistics (Vol. 26, Issue 06, pp. 683–718). Elsevier B.V. https://doi.org/10.1016/S0169-7161(06)26021-8
49. Tezza, R., Bornia, A. C., & Andrade, D. F. De. (2011). Measuring web usability using item response theory: Principles, features and opportunities. Interacting with Computers, 23(2), 167–175. https://doi.org/10.1016/j.intcom.2011.02.004
50. Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook. Journal of School Psychology, 14(1), 75. https://doi.org/10.1016/0022-4405(76)90066-2
51. Treagust, D. F., Duit, R., & Fischer, H. E. (2017). Multiple Representations in Physics Education. Springer International Publishing. https://doi.org/10.1007/978-3-319-58914-5_7
52. Tumanggor, A. M. R., Jumadi, J., Wilujeng, I., & Ringo, E. S. (2019). The Profile of Students’ Physics Problem Solving Ability in Optical Instruments. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 5(1), 29–40. https://doi.org/10.21009/1.05104
53. Tumanggor, A. M. R., Supahar, Kuswanto, H., & Ringo, E. S. (2020). Using four-tier diagnostic test instruments to detect physics teacher candidates’ misconceptions : Case of mechanical wave concepts. Journal of Physics: Conference Series, 1440, 1–8. https://doi.org/10.1088/1742-6596/1440/1/012059
54. Tumanggor, A. M. R., Supahar, Ringo, E. S., & Harliadi, M. D. (2020). Detecting Students ’ Misconception in Simple Harmonic Motion Concepts Using Four-Tier Diagnostic Test Instruments. Jurnal Ilmiah Pendidikan Fisika Al-Biruni, 9(1), 21–31. https://doi.org/10.24042/jipfalbiruni.v9i1.4571
55. Uhden, O., Karam, R., Pietrocola, M., & Pospiech, G. (2012). Modelling Mathematical Reasoning in Physics Education. Science & Education, 21(4), 485–506. https://doi.org/10.1007/s11191-011-9396-6
56. Volfson, A., Eshach, H., & Ben-Abu, Y. (2018). Development of a diagnostic tool aimed at pinpointing undergraduate students’ knowledge about sound and its implementation in simple acoustic apparatuses’ analysis. Physical Review Physics Education Research, 14(2), 20127. https://doi.org/10.1103/PhysRevPhysEducRes.14.020127
57. Wang, J., & Chen, S. (2016). Development and Validation of an Online Dynamic Assessment for Raising Students’ Comprehension of Science Text. International Journal of Science and Mathematics Education, 14(3), 373–389.
58. Wati, M., Mahtari, S., Hartini, S., & Amalia, H. (2019). A Rasch model analysis on junior high school students’ scientific reasoning ability. International Journal of Interactive Mobile Technologies, 13(7), 141–149. https://doi.org/10.3991/ijim.v13i07.10760
59. Wei, S., Liu, X., & Jia, Y. (2014). Using Rasch Measurement to Validate The Instrument of Students’ Understanding of Models in Science (SUMS). International Journal of Science and Mathematics Education, 12(5), 1067–1082. https://doi.org/10.1007/s10763-013-9459-z
60. Wilcox, B. R., & Pollock, S. J. (2015). Validation and analysis of the coupled multiple response Colorado upper-division electrostatics diagnostic. Physical Review Special Topics-Physics Education Research, 11(2). https://doi.org/10.1103/PhysRevSTPER.11.020130
61. Yeh, Y. F., Hsu, Y. S., Wu, H. K., & Chien, S. P. (2016). Exploring The Structure of TPACK with Video-Embedded and Discipline-Focused Assessments. Computers and Education, 104, 49–64. https://doi.org/10.1016/j.compedu.2016.10.006
62. Zhe, L. (2012). Survey of Primary Students’ Mathematical Representation Status and Study on the Teaching Model of Mathematical Representation. Journal of Mathematics Education, 5(1), 63–76.