Review of problem-solving measurement: an assessment developed in the Indonesian context

Year 2022, Volume: 9 Issue: 1, 116 - 136, 01.01.2022

Azizul Ghofar Candra Wicaksono , Erzsébet Korom

https://doi.org/10.17275/per.22.7.9.1

Cited By: 2

Abstract

The accuracy of learning results relies on the evaluation and assessment. The learning goals, including problem solving ability must be aligned with the valid standardized measurement tools. The study on exploring the nature of problem-solving, framework, and assessment in the Indonesian context will make contributions to problem solving assessment in Indonesian educational learning system. This review involved 32 studies that focus on problem-solving test development conducted in Indonesia and have the Indonesian version of the test. All tests are in the scope of certain subjects (mathematics, science, physics, and chemistry) and administered grade 7 to undergraduate level. Each test revealed a good value of reliability. Most of them have acceptable reliability score (r-value between .60 and .80) and high-reliability score (r > .80). Besides, they also showed content and construct validity (acceptable r value in Pearson product moment analysis and INFIT MNSQ index), but additional analysis is needed to fully develop the tests’ empirical evidence. All the tests are categorized as domain specific problem solving which focus on mathematics, science for junior high school, physics, chemistry, and biology. In addition, the topic coverage in the test should be improved and further studies about the measurement of problem-solving and test development are needed in the case of the Indonesian context.

Keywords

problem solving, assessment, Indonesian context

References

• Alfika, Z. A., Mayasari, T., & Kurniadi, E. (2019, January). Pengembangan Instrumen Tes Kemampuan Memecahkan Masalah Fisika Berbasis Kearifan Lokal pada siswa SMP/MTs. In Prosiding SNPF (Seminar Nasional Pendidikan Fisika).
• Angriani, A. D., Nursalam, N., Fuadah, N., & Baharuddin, B. (2018). Pengembangan Instrumen Tes untuk Mengukur Kemampuan Pemecahan Masalah Matematika Siswa. AULADUNA: Jurnal Pendidikan Dasar Islam, 5(2), 211-223. https://doi.org/10.24252/auladuna.v5i2a9.2018
• Ariawan, R. (2016). Pengembangan Instrumen Tes Kemampuan Pemecahan Masalah Dan Komunikasi Matematis Pada Materi Bangun Ruang Sisi Datar. AdMathEdu, 6(2), 57206. http://dx.doi.org/10.12928/admathedu.v6i2.5447
• Asyari, M., al Muhdhar, M. H. I., Susilo, H., & . I. (2016). Improving critical thinking skills through the integration of problem based learning and group investigation. International Journal for Lesson and Learning Studies, 5(1), 36–44. https://doi.org/10.1108/IJLLS-10-2014-0042
• Bartley, J. E., Boeving, E. R., Riedel, M. C., Bottenhorn, K. L., Salo, T., Eickhoff, S. B., Brewe, E., Sutherland, M. T., & Laird, A. R. (2018). Meta-analytic evidence for a core problem solving network across multiple representational domains. Neuroscience & Biobehavioral Reviews, 92, 318–337. https://doi.org/10.1016/j.neubiorev.2018.06.009
• Bransford, J., Sherwood, R., Vye, N., & Rieser, J. (1986). Teaching thinking and problem solving: Research foundations. American Psychologist, 41(10), 1078–1089. https://doi.org/10.1037/0003-066X.41.10.1078
• Bidasari, F. (2017). Pengembangan Soal Matematika Model PISA pada Konten Quantity untuk Mengukur Kemampuan Pemecahan Masalah Matematika Siswa Sekolah Menengah Pertama. Jurnal Gantang, 2(1), 63-77. DOI: https://doi.org/10.31629/jg.v2i1.59
• Brookhart, S. M., & Nitko, A. J. (2014). Educational assessment of students. Pearson Higher Ed.
• Butterworth, J., & Thwaites, G. (2013). Thinking skills: Critical thinking and problem solving. Cambridge University Press.
• Care, E., Scoular, C., & Griffin, P. (2016). Assessment of Collaborative Problem Solving in Education Environments. Applied Measurement in Education, 29(4), 250–264. https://doi.org/10.1080/08957347.2016.1209204
• Carlson, M. P., & Bloom, I. (2005). The Cyclic Nature of Problem Solving: An Emergent Multidimensional Problem-Solving Framework. Educational Studies in Mathematics, 58(1), 45–75. https://doi.org/10.1007/s10649-005-0808-x
• Christ, A., Becker, N., & Kröner, S. (2020). Multiple complex problem-solving scenarios: The incremental validity of ability self-concept beyond reasoning in adults. Intelligence, 78, 101421. https://doi.org/10.1016/j.intell.2019.101421
• Csapó, B., & Funke, J. (2017). The Nature of Problem Solving (B. Csapó & J. Funke, Eds.). OECD. https://doi.org/10.1787/9789264273955-en
• Davidson, J. E., Sternberg, R. J., & Sternberg, R. J. (2003). The psychology of problem solving. Cambridge University Press.
• Dermitzaki, I., Leondari, A., & Goudas, M. (2009). Relations between young students’ strategic behaviours, domain-specific self-concept, and performance in a problem-solving situation. Learning and Instruction, 19(2), 144–157. https://doi.org/10.1016/j.learninstruc.2008.03.002
• Dindar, M. (2018). An empirical study on gender, video game play, academic success and complex problem solving skills. Computers & Education, 125, 39–52. https://doi.org/10.1016/j.compedu.2018.05.018
• Docktor, J., & Heller, K. (2009). Robust assessment instrument for student problem solving. In Proceedings of the NARST 2009 Annual Meeting, Garden Grove, CA, 1–9.
• Downing, S. M., & Haladyna, T. M. (1997). Test Item Development: Validity Evidence From Quality Assurance Procedures. Applied Measurement in Education, 10(1), 61–82. https://doi.org/10.1207/s15324818ame1001_4
• Fawcett, L. M., & Garton, A. F. (2005). The effect of peer collaboration on children’s problem-solving ability. British Journal of Educational Psychology, 75(2), 157–169. https://doi.org/10.1348/000709904X23411
• Fathurrahman, M., Permanasari, A., & Siswaningsih, W. (2016). Pengembangan Tes Keterampilan Problem Solving Siswa SMA pada Pokok Bahasan Stoikiometri Larutan. EduChemia (Jurnal Kimia dan Pendidikan), 1(1), 62-75. http://dx.doi.org/10.30870/educhemia.v1i1.440
• Februarini, Y. T., & Suratman, D. Pengembangan perangkat tes untuk mengukur kemampuan problem solving dan penalaran matematis siswa sekolah menengah kejuruan. Jurnal Pendidikan dan Pembelajaran Khatulistiwa, 6(5). (retrieved from http://jurnal.untan.ac.id/index.php/jpdpb/article/view/19934)
• Ferreira, M., & Trudel, A. (2012). The Impact of Problem-Based Learning (PBL) on Student Attitudes Toward Science, Problem-Solving Skills, and Sense of Community in the Classroom. The Journal of Classroom Interaction, 47(1), 23–30.
• Funke, J. (2001). Dynamic systems as tools for analysing human judgement. Thinking & Reasoning, 7(1), 69–89. https://doi.org/10.1080/13546780042000046
• Funke, J. (2010). Complex problem solving: a case for complex cognition? Cognitive Processing, 11(2), 133–142. https://doi.org/10.1007/s10339-009-0345-0
• Glaser, R., Chi, M. T. H., & Farr, M. J. (2009). Measuring complex problem solving: The MicroDYN approach.
• Gliner, J.A., Morgan, G.A., & Leech, N.L. (2017). Research method in applied settings: an integrated approach to design and analysis. New York: Routledge, Taylor and Francis
• Gok, T. (2014). Peer instruction in the physics classroom: Effects on gender difference performance, conceptual learning, and problem solving. Journal of Baltic Science Education, 13(6), 776.
• Greiff, S. (2012). Assessment and Theory in Complex Problem Solving - A Continuing Contradiction? Journal of Educational and Developmental Psychology, 2(1). https://doi.org/10.5539/jedp.v2n1p49
• Greiff, S., Fischer, A., Stadler, M., & Wüstenberg, S. (2015). Assessing complex problem-solving skills with multiple complex systems. Thinking & Reasoning, 21(3), 356–382. https://doi.org/10.1080/13546783.2014.989263
• Greiff, S., & Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learning and Individual Differences, 36, 37–48. https://doi.org/10.1016/j.lindif.2014.08.003
• Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., & Martin, R. (2015). Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence, 50, 100–113. https://doi.org/10.1016/j.intell.2015.02.007
• Greiff, S., Wüstenberg, S., Csapó, B., Demetriou, A., Hautamäki, J., Graesser, A. C., & Martin, R. (2014). Domain-general problem solving skills and education in the 21st century. Educational Research Review, 13, 74–83. https://doi.org/10.1016/j.edurev.2014.10.002
• Griffin, P. (1999). Item response modeling: an introduction to Rasch model. Melbourne: Assessment Research Center, University of Melbourne.
• Harta, J. (2017). Pengembangan Soal Esai Berbasis HOTS Untuk Menyelidiki Keterampilan Pemecahan Masalah Siswa SMA. Jurnal Penelitian, 21(1). (retrieved from https://e-journal.usd.ac.id/index.php/JP/article/view/896)
• Hendriana, H., Johanto, T., & Sumarmo, U. (2018). The Role of Problem-Based Learning to Improve Students' Mathematical Problem-Solving Ability and Self Confidence. Journal on Mathematics Education, 9(2), 291-300.
• Hidayat, S. R., Setyadin, A. H., Hermawan, K., Kaniawati, I., Suhendi, E., Siahaan, P., & Samsudin, A. (2017). Pengembangan instrumen tes keterampilan pemecahan masalah pada materi getaran, gelombang, dan bunyi. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 3(2), 157-166. http://doi.org/10.21009/1
• Hidayat, T., Susilaningsih, E., & Kurniawan, C. (2018). The effectiveness of enrichment test instruments design to measure students’ creative thinking skills and problem-solving. Thinking Skills and Creativity, 29, 161-169. https://doi.org/10.1016/j.tsc.2018.02.011
• Hung, C. M., Hwang, G. J., & Huang, I. (2012). A project-based digital storytelling approach for improving students’ learning motivation, problem-solving competence and learning achievement. Journal of Educational Technology & Society, 15(4), 368–379.
• Irawati, T. N., & Mahmudah, M. (2018). Pengembangan Instrumen Kemampuan Berpikir Analisis Siswa SMP dalam Menyelesaikan Soal Pemecahan Masalah Matematika. Kadikma, 9(2), 1-11.
• Iswandari, D. C., Prayogo, J. A., & Cahyono, B. Y. (2017). Effect of Environmental Problem-based Learning on the Indonesian EFL Students’ Environment-related Vocabulary Mastery and Writing Ability. Theory and Practice in Language Studies, 7(8), 608. https://doi.org/10.17507/tpls.0708.02
• Jonassen, D. H. (2010). Learning to solve problems: A handbook for designing problem-solving learning environments. Routledge.
• Kroner, S., Plass, J., & Leutner, D. (2005). Intelligence assessment with computer simulations. Intelligence, 33(4), 347–368. https://doi.org/10.1016/j.intell.2005.03.002
• Kurniawan, B. R., & Taqwa, M. R. A. (2018). Pengembangan Instrumen Tes Kemampuan Pemecahan Masalah Fisika pada Materi Listrik Dinamis. Jurnal Pendidikan: Teori, Penelitian, dan Pengembangan, 3(11), 1451-1457. http://dx.doi.org/10.17977/jptpp.v3i11.11761
• Lachman, R., Lachman, J. L., & Butterfield, E. C. (2015). Cognitive psychology and information processing: An introduction. Psychology Press.
• Lestari, P. E., Purwanto, A., & Sakti, I. (2019). Pengembangan instrument tes keterampilan pemecahan masalah pada konsep usaha dan energy si SMA. Jurnal Kumparan Fisika, 2(3), 161-168. https://doi.org/10.33369/jkf.2.3.161-168
• Liao, S. H. (2002). Problem solving and knowledge inertia. Expert Systems with Applications, 22(1), 21–31.
• Luthfi, I. A., Muharomah, D. R., Ristanto, R. H., & Miarsyah, M. (2019). Pengembangan tes kemampuan pemecahan masalah pada isu pencemaran lingkungan. Jurnal BIOEDUIN: Program Studi Pendidikan Biologi, 9(2), 11-20. https://doi.org/10.15575/bioeduin.v9i2.5892
• Mukhopadhyay, R. (2013). Problem solving in science learning- some important consideration of teacher. IOSR Journal of Humanities and Social Science (IOSR-JHSS), 4(6), 21–25.
• Nadapdap, A. T. Y., & Istiyono, E. (2017). Developing physics problem-solving skill test for grade X students of senior high school. REiD (Research and Evaluation in Education), 3(2), 114-123. https://doi.org/10.21831/reid.v3i2.14982
• Nickles, T. (1988). Questioning and problems in philosophy of science: Problem-solving versus directly truth-seeking epistemologies. Questions and Questioning, 43–67.
• Novita, R. (2012). Exploring Primary Student's Problem-Solving Ability by Doing Tasks Like PISA's Question. Indonesian Mathematical Society Journal on Mathematics Education, 3(2), 133-150. (retrieved from https://eric.ed.gov/?id=EJ1078578)
• OECD. (2014). PISA 2012 Results: Creative Problem Solving (Volume V). OECD. https://doi.org/10.1787/9789264208070-en
• OECD. (2016). PISA 2015 Results (Volume I). OECD. https://doi.org/10.1787/9789264266490-en
• OECD. (2017). PISA 2015 Results (Volume V). OECD. https://doi.org/10.1787/9789264285521-en
• Pardimin, P., Widodo, S. A., & Purwaningsih, I. E. (2017). Analisis Butir Soal Tes Pemecahan Masalah Matematika. Wacana Akademika: Majalah Ilmiah Kependidikan, 1(1). http://dx.doi.org/10.30738/wa.v1i1.1084
• Partnership for 21st century skills (2009). P21 Framework Definitions. (retrieved from https://files.eric.ed.gov/fulltext/ED519462.pdf)
• Peranginangin, S. A., & Surya, E. (2017). An analysis of students’ mathematics problem solving ability in VII grade at smp negeri 4 pancurbatu. International Journal of Sciences: Basic and Applied Research (IJSBAR), 33(2), 57-67.
• Polya, G. (2004). How to solve it: A new aspect of mathematical method. Princeton University Press.
• Pratiwi, D.N.. (2015). Pengembangan Instrumen Evaluasi Berbasis Taksonomi Structure of the Observed Learning Outcome (SOLO) Untuk Menentukan Profil Kemampuan Siswa dalam Memecahkan Masalah Fluida Statis. Inovasi Pendidikan Fisika, 4(3).
• Putra, H. D. (2017). Pengembangan Instrumen untuk Meningkatkan Kemampuan Mathematical Problem Posing Siswa SMA. Euclid, 4(1). DOI: http://dx.doi.org/10.33603/e.v4i1.211
• Rausch, A., & Wuttke, E. (2016). Development of multi faceted model of domain specific problem solving and its acceptance by different stakeholders in the business domain. Unterrichtswissenschaft, 44(2), 169–184.
• Ridhwan, R., Sumarmi, S., Ruja, I., Utomo, D., & Sari, R. (2020). Measuring Students Environmental Problem Solving Ability Across Gender and School Differences Using Paper Based Testing. International Journal of Emerging Technologies in Learning (iJET), 15(13), 303-320.
• Rifa'i, A., & Lestari, H. P. (2018, March). The effect of think pair share (TPS) using scientific approach on students’ self-confidence and mathematical problem-solving. In Journal of Physics: Conference Series. https://doi. org/10.1088/1742-6596/983/1/012084.
• Rudolph, J., Niepel, C., Greiff, S., Goldhammer, F., & Kröner, S. (2017). Metacognitive confidence judgments and their link to complex problem solving. Intelligence, 63, 1–8. https://doi.org/10.1016/j.intell.2017.04.005
• Savitri, N., & Setiani, I. A. (2020). Pengembangan Instrumen Tes Kemampuan Pemecahan Masalah Fisika pada Materi Momentum dan Impuls. Jurnal Kependidikan Betara, 1(1), 20-26.
• Schoenfeld, A.H. (2013). Reflections on problem solving theory and practice. The Mathematics Enthusiast, 10(1), 9–35. https://scholarworks.umt.edu/tme/vol10/iss1/3
• Schoenfeld, Alan H. (1987). Pólya, Problem Solving, and Education. Mathematics Magazine, 60(5), 283–291. https://doi.org/10.1080/0025570X.1987.11977325
• Schumacker, R. E., & Lomax, R. G. (2014). A beginner’s guide to structural equation modeling. Psychology Press.
• Sinaga, N. A. (2016). Pengembangan tes kemampuan pemecahan masalah dan penalaran matematika siswa SMP kelas VIII. Pythagoras: Jurnal Pendidikan Matematika, 11(2), 169-181. https://doi.org/10.21831/pg.v11i2.10642
• Sinensis, A. R., Firman, H., Hamidah, I., & Muslim, M. (2019). Pengembangan instrument tes termodinamikan untuk mengukur kemampuan pemecahan masalah (PsACAr) pada mahasiswa calon Guru Fisika. Jurnal Inovasi dan Pembelajaran Fisika, 6(2), 122-129. https://doi.org/10.36706/jipf.v6i2.9879
• Sonnleitner, P., Brunner, M., Greiff, S., Funke, J., Keller, U., Martin, R., Hazotte, C., Mayer, H., & Latour, T. (2012). The Genetics Lab. Acceptance and psychometric characteristics of a computer-based microworld to assess complex problem solving. Psychological Test and Assessment Modeling, 54, 54–72.
• Stadler, M., Becker, N., Gödker, M., Leutner, D., & Greiff, S. (2015). Complex problem solving and intelligence: A meta-analysis. Intelligence, 53, 92–101. https://doi.org/10.1016/j.intell.2015.09.005
• Sternberg, R. (2018). Speculations on the Role of Successful Intelligence in Solving Contemporary World Problems †. Journal of Intelligence, 6(1), 4. https://doi.org/10.3390/jintelligence6010004
• Subekti, F. E., & Widiyanti, R. A. (2014). Pengembangan instrumen tes Kalkulus Lanjut 2 berbasis pemecahan masalah. Khazanah Pendidikan, 7(1). DOI: 10.30595/jkp.v7i1.669
• Sugrue, B. (2005). A Theory-Based Framework for Assessing Domainl-Specific Problem-Solving Ability. Educational Measurement: Issues and Practice, 14(3), 29–35. https://doi.org/10.1111/j.1745-3992.1995.tb00865.x
• Sumarmo, U. (2015). Mathematical Problem Posing: Rasional, Pengertian, Pembelajaran, dan Pengukurannya. Pascasarjana STKIP Siliwangi Bandung dan Pascasarjana UPI.
• Sutiadi, A., & Nurwijayaningsih, H. (2016). Konstruksi dan Profil Problem Solving Skill Siswa SMP dalam Materi Pesawat Sederhana. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 2(1), 37-42. https://doi.org/10.21009/1.02106
• Syifauliyah, A. (2019, November). Pengembangan Instrumen Tes Kemampuan Memecahkan Masalah Fisika Bunyi Berbasis Musik. In Seminar Nasional Lontar Physics Forum (pp. 100-105).
• Voskoglou, M. G. (2011). Problem solving from Polya to nowadays: A review and future perspectives. Progress in Education, 22(4), 65–82. https://d1wqtxts1xzle7.cloudfront.net/32888949/Problem_-_Solving.pdf?1391217514=&response-content-disposition=inline%3B+filename%3DProblem_Solving_from_Polya_to_Nowadays_A.pdf&Expires=1598531286&Signature=GK3T3jRs2cyK-Cw8IYirLoBEeC51q10f1eTT3qSDhzEmbHouDK
• Wahyuningrum, E. (2014). Kemampuan pemecahan masalah, komunikasi dan disposisi matematik siswa SMP melalui strategi MEAs. Dissertation. Unpublish. Graduate program Universitas Pendidikan Indonesia
• Walker, F., Link, F., & Nickolaus, N. (2016). A multidimensional structure of domain-specific problem-solving competencies of electronics technicians for automaton technology. Empirical Research in Vocational Education and Training, 8(7), 1–26. https://doi.org/10.1186/s40461 016 0034 z.
• Walsh, L. N., Howard, R. G., & Bowe, B. (2007). Phenomenographic study of students’ problem solving approaches in physics. Physical Review Special Topics-Physics Education Research, 3(2), 020108. https://doi.org/10.1103/PhysRevSTPER.3.020108
• Wardani, A., Arkan, M. N., & Suyudi, A. (2020). Pengembangan Instrumen Tes Keterampilan Pemecahan Masalah Pokok Bahasan Listrik Dinamis. Jurnal Kependidikan Betara, 1(1), 14-19.
• Westen, D., & Rosenthal, R. (2003). Quantifying construct validity: Two simple measures. Journal of Personality and Social Psychology, 84(3), 608–618. https://doi.org/10.1037/0022-3514.84.3.608
• Wolff, K. (2017). Engineering problem-solving knowledge: the impact of context. Journal of Education and Work, 30(8), 840–853. https://doi.org/10.1080/13639080.2017.1380299
• Wüstenberg, S., Greiff, S., & Funke, J. (2012). Complex problem solving — More than reasoning? Intelligence, 40(1), 1–14. https://doi.org/10.1016/j.intell.2011.11.003
• Yu, W., She, H., & Lee, Y. (2010). The effects of Web‐based/non‐Web‐based problem‐solving instruction and high/low achievement on students’ problem‐solving ability and biology achievement. Innovations in Education and Teaching International, 47(2), 187–199. https://doi.org/10.1080/14703291003718927
• Yuliati, L., Riantoni, C., & Mufti, N. (2018). Problem Solving Skills on Direct Current Electricity through Inquiry-Based Learning with PhET Simulations. International Journal of Instruction, 11(4), 123-138.
• Zulkarnain, R., Sugiatno, S., & Bistari, B. (2018). Pengembangan instrument problem solving dengan sajian masalah matematika kontekstual di sekolah menengah pertama. Jurnal Pendidikan dan Pembelajaran, 7(7).