The development of diagnostic test instrument for mathematical representation ability (PhysDTRA) in high school physics learning

Year 2020, Volume: 8 Issue: 4, 1439 - 1455, 15.12.2020

Awal Mulia Rejeki Tumanggor , Supahar Supahar

https://doi.org/10.17478/jegys.777425

Abstract

Assessment in education is part of collecting and processing various information related to student achievements during learning. Good learning quality can be seen from the quality of the assessment. Assessment activities that help educators to find out the difficulty of students on a learning material so that it can be guided to achieve completeness criteria, namely by diagnostic tests. The assessment carried out must follow 21st-century learning that is integrated with the industrial era 4.0, namely how to diagnose difficulties related to the representation ability of students in physics learning by utilizing assessments that have advantages in detecting student difficulties and giving suggestions for appropriate improvement. The purpose of this research was to develop a test instrument (PhysDTRA) that could be used to diagnose students' mathematical representation abilities in high school physics learning. The results of the research were analyzed quantitatively and qualitatively using Item Response Theory (IRT). Based on the content validity, the PhysDTRA instrument was declared to be valid according to the expert judgments who were analyzed using the Aiken's V equation. All items in the test instrument were valid based on the Rasch, INFIT MNSQ, and INFIT t models. The PhysDTRA instrument has also been relied upon based on the reliability of the estimated items and TIC curves so that it can be used to diagnose and determine the profile of students' mathematical representation abilities. Thus, the PhysDTRA instrument developed has fulfilled the test characteristics that are feasible of its content, empirical evidence, validity, and reliability.

Keywords

Diagnostic Instrument, Mathematical Representation, Work and Energy, Validity, Reliability, Development of diagnostic test

Supporting Institution

Directorate of Research and Community Service (DRPM), Deputy for Strengthening Research and Development of the Ministry of Research, Technology, and Higher Education/ National Research and Innovation Agency, Indonesia

Project Number

T/9.65/UN34.21/PT.01.03/2020

Thanks

The author would like to thank for funding support from the Directorate of Research and Community Service (DRPM), Deputy for Strengthening Research and Development of the Ministry of Research, Technology, and Higher Education/ National Research and Innovation Agency through Master Thesis Research, No. T/9.65/UN34.21/PT.01.03/2020.

References

• Adams, R. J., & Khoo, S.-T. (1996). QUEST: The interactive test analysis system version 2.1. Australian Council for Educational Research.
• Adams, W. K., Wieman, C. E., & Adams, W. K. (2015). Analyzing The Many Skills Involved in Solving Complex Physics Problems Analyzing The Many Skills Involved in Solving complex Physics Problems. American Journal of Physics, 83(459). https://doi.org/10.1119/1.4913923
• Adlina, A., & Supahar. (2019). Developing android assisted worked example application on kinematics (weak) to improve mathematical representation ability in high school physics learning. International Journal of Scientific and Technology Research, 8(10), 3790–3793.
• Aiken, L. R. (1985). Three Coefficients for Analysing Reliability and Validity of Ratings. Educational and Psychological Measurement, 45, 131–142. https://doi.org/10.1177/07399863870092005
• Albe, V., Venturini, P., & Lascours, J. (2014). Electromagnetic Concepts in Mathematical Representation of Physics. Journal of Science Education and Technology, 10, 197–203.
• Angell, C., Kind, P. M., Henriksen, E. K., & Guttersrud, Ø. (2008). An Empirical-Mathematical Modelling Approach to Upper Secondary Physics. Physics Education, 43(3), 256–264. https://doi.org/10.1088/0031-9120/43/3/001
• Atkin, J. M., & Coffey, J. E. (2003). Everyday Assessment in The Science Classroom (Issue c). National Science Teachers Association.
• Azwar, S. (2012). Penyusunan Skala Psikologi (Psychological Scale Compilation). Pustaka Pelajar.
• Azwar, S. (2015). Reliability and Validity (fourth). Pustaka Pelajar.
• Bing, T. J., & Redish, E. F. (2014). The Cognitive Blending of Mathematics and Physics Knowledge. AIP Conference Proceedings, 883(February 2007), 26–29. https://doi.org/10.1063/1.2508683
• Bollen, L., van Kampen, P., Baily, C., Kelly, M., & De Cock, M. (2017). Student difficulties regarding symbolic and graphical representations of vector fields. Physical Review Physics Education Research, 13(2), 020109. https://doi.org/10.1103/PhysRevPhysEducRes.13.020109
• Ceuppens, S., Deprez, J., Dehaene, W., & De Cock, M. (2018). Design and validation of a test for representational fluency of 9th grade students in physics and mathematics: The case of linear functions. Physical Review Physics Education Research, 14(2), 20105. https://doi.org/10.1103/PhysRevPhysEducRes.14.020105
• Darmawan, D., Yatimah, D., Sasmita, K., & Syah, R. (2020). Analysis of Non-Formal Education Tutor Capabilities in Exploring Assessment for Science Learning. Jurnal Pendidikan IPA Indonesia, 9(2), 267–275. https://doi.org/10.15294/jpii.v9i2.24025
• De Cock, M. (2012). Representation Use and Strategy Choice in Physics Problem Solving. Physical Review Special Topics - Physics Education Research, 8(2), 1–15. https://doi.org/10.1103/PhysRevSTPER.8.020117
• Docktor, J. L., & Mestre, J. P. (2014). Synthesis of Discipline-Based Education Research in Physics. Physical Review Special Topics - Physics Education Research, 10(2), 1–58. https://doi.org/10.1103/PhysRevSTPER.10.020119
• du Toit, M. (2003). IRT from SSI: Bilog-Mg, Multilog, Parscale, Testfact. Scientific Software International, Inc.
• Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138
• George, D., & Mallery, P. (2020). IBM SPSS Statistics 26 Step by Step: A Simple Guide and Reference (Sixteenth). Routledge. https://doi.org/10.4324/9780429056765
• Gurcay, D., & Gulbas, E. (2015). Development of Three-Tier Heat, Temperature and Internal Energy Diagnostic Test. Research in Science and Technological Education, 33(2), 197–217. https://doi.org/10.1080/02635143.2015.1018154
• Gurel, D. K., Eryilmaz, A., & McDermott, L. C. (2015). A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptions in Science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a
• Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of Item Response Theory Library. Sage Publication, Inc.
• Heuvelen, A. Van, & Zou, X. (2001). Multiple representations of work–energy processes. American Journal of Physics, 69(2), 184–194. https://doi.org/10.1119/1.1286662
• Ibrahim, B., & Rebello, N. S. (2012). Representational Task Formats and Problem Solving Strategies in Kinematics and Work. Physical Review Special Topics - Physics Education Research, 8(1), 1–19. https://doi.org/10.1103/PhysRevSTPER.8.010126
• Johnson, R. B., & Christensen, L. B. (2016). Educational Research: Quantitative, Qualitative, and Mixed Approaches (Sixth). SAGE Publication.
• Kim, S. (2006). A comparative study of IRT fixed parameter calibration methods. Journal of Educational Measurement, 43(4), 355–381. https://doi.org/10.1111/j.1745-3984.2006.00021.x
• Kirschner, S., Borowski, A., Fischer, H. E., Gess-Newsome, J., & von Aufschnaiter, C. (2016). Developing and evaluating a paper-and-pencil test to assess components of physics teachers’ pedagogical content knowledge. International Journal of Science Education, 38(8), 1343–1372. https://doi.org/10.1080/09500693.2016.1190479
• Krawec, J. L. (2014). Problem Representation and Mathematical Problem Solving of Students of Varying Math Ability. Journal of Learning Disabilities, 47(2), 103–115. https://doi.org/10.1177/0022219412436976
• Kriek, J., & Koontse, R. D. (2017). First year physics students’ expectations of the role of mathematics in physics. International Journal of Innovation in Science and Mathematics Education, 25(2), 1–16.
• Liang, L. L., Fulmer, G. W., Majerich, D. M., Clevenstine, R., & Howanski, R. (2012). The Effects of a Model-Based Physics Curriculum Program with a Physics First Approach: A Causal-Comparative Study. Journal of Science Education and Technology, 21(1), 114–124. https://doi.org/10.1007/s10956-011-9287-2
• Lissitz, R. W., & Samuelsen, K. (2007). Further Clarification Regarding Validity and Education. Educational Researcher, 36(8), 482–484. https://doi.org/10.3102/0013189x07311612
• Loewenthal, K. M., & Lewis, C. A. (2018). An introduction to psychological tests and scales. Psychology Press.
• Minarni, A., & Napitupulu, E. E. (2017). Developing Instruction Materials Based on Joyful PBL to Improve Students Mathematical Representation Ability. International Education Studies, 10(9), 23. https://doi.org/10.5539/ies.v10n9p23
• Niss, M. (2016). Obstacles Related to Structuring for Mathematization Encountered by Students when Solving Physics Problems. International Journal of Science and Mathematics Education, 15(8), 1441–1462. https://doi.org/10.1007/s10763-016-9754-6
• Nitko, A. J., & Brookhart, S. M. (2011). Educational Assessment of Students (Sixth Edit). Pearson Education, Inc. OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. In OECD Publishing: Vol. II. https://doi.org/https://doi.org/10.1787/b5fd1b8f-en
• Oriondo, L. L., & Dallo-Antonio, E. M. (1984). Evaluating Educational Outcomes (Test, Measurement, and Evaluation) (5th Editio). REX Printing Company, Inc.
• Pape, S. J., & Tchoshanov, M. A. (2001). The Role of Representation(s) in Developing Mathematical Understanding. Theory Into Practice, 40(2), 118–127. https://doi.org/10.1207/s15430421tip4002_6
• Park, E.-J., & Choi, K. (2013). Analysis of Student Understanding of Science Concepts Including Mathematical Representations: pH Values and The Relative Differences of pH Values. International Journal of Science and Mathematics Education, 11(3), 683–706. https://doi.org/10.1007/s10763-012-9359-7
• Permendikbud No. 22 Tahun 2016 tentang Standar Proses Pendidikan Dasar dan Menengah (Minister of Education and Culture Regulation No. 22 of 2016 on Process Standards for Primary and Secondary Education), (2016).
• Planinic, M., Ivanjek, L., Susac, A., & Milin-Sipus, Z. (2013). Comparison of university students’ understanding of graphs in different contexts. Physical Review Special Topics - Physics Education Research, 9(2), 1–9. https://doi.org/10.1103/PhysRevSTPER.9.020103
• Pospiech, G., Eylon, B., Bagno, E., Lehavi, Y., & Geyer, M. A. (2016). The Role of Mathematics for Physics Teaching and Understanding. Nuovo Cimento C Geophysics Space Physics C, 38(3), 10. https://doi.org/https://doi.org/10.1393/ncc/i2015-15110-6
• Pujayanto, Budiharti, R., Radiyono, Y., Rizky, N., Nuraini, A., Putri, H. V., Saputro, D. E., & Adhitama, E. (2018). Pengembangan Tes Diagnostik Miskonsepsi Empat Tahap Tentang Kinematika (Developing Four-Tier Misconception Diagnostic Test about Kinematics). Cakrawala Pendidikan, 37(2), 237–249. https://doi.org/10.21831/cp.v37i2.16491
• Retnawati, H. (2014). Teori Respon Butir dan Penerapannya: untuk Peneliti, Praktisi Pengukuran dan Pengujian, Mahasiswa Pascasarjana (Item Response Theory and Its Application: for Researchers, Measurement and Testing Practitioners, Postgraduate Students). Nuha Medika.
• Retnawati, H. (2016). Analisis Kuantitatif Instrumen Penelitian: Panduan Peneliti, Mahasiswa, dan Psikometrian (Quantitative Analysis of Research Instruments: A Guide for Researchers, Students, and Psychometrics). Parama Publishing.
• Sadaghiani, H. R. (2011). Using multimedia learning modules in a hybrid-online course in electricity and magnetism. Physical Review Special Topics - Physics Education Research, 7(1), 1–7. https://doi.org/10.1103/PhysRevSTPER.7.010102
• Subali, B., & Pujiati, S. (2012). Pengembangan Item Tes Konvergen dan Divergen dan Penyelidikan Validitasnya secara Empiris (Development of Convergent and Divergent Test Item and Empirical Validity Investigation). Diandra Pustaka Indonesia.
• Subali, Bambang, & Suyata, P. (2011). Panduan Analisis Data Pengukuran Pendidikan Untuk Memperoleh Bukti Empirik Kesahihan menggunakan program Quest (Guidelines for Educational Measurement Data Analysis to Obtain Empiric Proof of Eligibility using the Quest Program). Lembaga Penelitian dan Pengabdian pada Masyarakat Universitas Negeri Yogyakarta.
• Supahar, S., & Prasetyo, Z. K. (2015). Pengembangan Instrumen Penilaian Kinerja Kemampuan Inkuiri Peserta Didik Pada Mata Pelajaran Fisika SMA (Developing Assessment Instrument of The Performance of The Inquiry Abilities in Physics Subject of High School Students). Jurnal Penelitian Dan Evaluasi Pendidikan, 19(1), 96–108. https://doi.org/10.21831/pep.v19i1.4560
• Swaminathan, H., Hambleton, R. K., & Rogers, H. J. (2007). Assessing the Fit of Item Response Theory Models. In Handbook of Statistics (Vol. 26, Issue 06, pp. 683–718). Elsevier B.V. https://doi.org/10.1016/S0169-7161(06)26021-8
• Tezza, R., Bornia, A. C., & Andrade, D. F. De. (2011). Measuring web usability using item response theory: Principles, features and opportunities. Interacting with Computers, 23(2), 167–175. https://doi.org/10.1016/j.intcom.2011.02.004
• Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook. Journal of School Psychology, 14(1), 75. https://doi.org/10.1016/0022-4405(76)90066-2
• Treagust, D. F., Duit, R., & Fischer, H. E. (2017). Multiple Representations in Physics Education. Springer International Publishing. https://doi.org/10.1007/978-3-319-58914-5_7
• Tumanggor, A. M. R., Jumadi, J., Wilujeng, I., & Ringo, E. S. (2019). The Profile of Students’ Physics Problem Solving Ability in Optical Instruments. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 5(1), 29–40. https://doi.org/10.21009/1.05104
• Tumanggor, A. M. R., Supahar, Kuswanto, H., & Ringo, E. S. (2020). Using four-tier diagnostic test instruments to detect physics teacher candidates’ misconceptions : Case of mechanical wave concepts. Journal of Physics: Conference Series, 1440, 1–8. https://doi.org/10.1088/1742-6596/1440/1/012059
• Tumanggor, A. M. R., Supahar, Ringo, E. S., & Harliadi, M. D. (2020). Detecting Students ’ Misconception in Simple Harmonic Motion Concepts Using Four-Tier Diagnostic Test Instruments. Jurnal Ilmiah Pendidikan Fisika Al-Biruni, 9(1), 21–31. https://doi.org/10.24042/jipfalbiruni.v9i1.4571
• Uhden, O., Karam, R., Pietrocola, M., & Pospiech, G. (2012). Modelling Mathematical Reasoning in Physics Education. Science & Education, 21(4), 485–506. https://doi.org/10.1007/s11191-011-9396-6
• Volfson, A., Eshach, H., & Ben-Abu, Y. (2018). Development of a diagnostic tool aimed at pinpointing undergraduate students’ knowledge about sound and its implementation in simple acoustic apparatuses’ analysis. Physical Review Physics Education Research, 14(2), 20127. https://doi.org/10.1103/PhysRevPhysEducRes.14.020127
• Wang, J., & Chen, S. (2016). Development and Validation of an Online Dynamic Assessment for Raising Students’ Comprehension of Science Text. International Journal of Science and Mathematics Education, 14(3), 373–389.
• Wati, M., Mahtari, S., Hartini, S., & Amalia, H. (2019). A Rasch model analysis on junior high school students’ scientific reasoning ability. International Journal of Interactive Mobile Technologies, 13(7), 141–149. https://doi.org/10.3991/ijim.v13i07.10760
• Wei, S., Liu, X., & Jia, Y. (2014). Using Rasch Measurement to Validate The Instrument of Students’ Understanding of Models in Science (SUMS). International Journal of Science and Mathematics Education, 12(5), 1067–1082. https://doi.org/10.1007/s10763-013-9459-z
• Wilcox, B. R., & Pollock, S. J. (2015). Validation and analysis of the coupled multiple response Colorado upper-division electrostatics diagnostic. Physical Review Special Topics-Physics Education Research, 11(2). https://doi.org/10.1103/PhysRevSTPER.11.020130
• Yeh, Y. F., Hsu, Y. S., Wu, H. K., & Chien, S. P. (2016). Exploring The Structure of TPACK with Video-Embedded and Discipline-Focused Assessments. Computers and Education, 104, 49–64. https://doi.org/10.1016/j.compedu.2016.10.006
• Zhe, L. (2012). Survey of Primary Students’ Mathematical Representation Status and Study on the Teaching Model of Mathematical Representation. Journal of Mathematics Education, 5(1), 63–76.