Integrating Assessment and Performance Measurement: A Case of an Academic Course for Quality Improvement Actions at a Saudi University

Abstract

This study aims to assess and measure students’ performances using course-key performance indicators (course-KPIs) in an academic course at a Saudi university. The approach includes three aspects of assessment (i) integrating course components and correlating course learning objectives with the program learning domain, (ii) course evaluation using rubrics, and (iii) performance mesurement using a scientific method. Moreover, it presents a novel approach for performance measurement of the course learning skills. In this study, a course has been taken to demonstrate how the KPIs are measured for evaluating students’ performances. This approach relies on several specific documents that are developed for the course delivery by following the National Qualification Framework (NQF) in Saudi Arabia and the guidelines of the Accreditation Board for Engineering and Technology (ABET). The performance evaluation outcomes are useful indicators that guide the teachers to improve course learning skills. It also helps the teachers in the quality delivery of the courses and ensures continuous improvement in learning and teaching. This study concludes with an emphasis on the measuring performance using course-KPIs which can be adopted for quality improvement for any academic course in higher education irrespective of data size.

Keywords

• Course assessment
• Higher education
• Performance indicators
• Rubrics
• Performance evaluation

References

1. ABET (2017). Criteria for Accrediting Computing Programs, 2016 2017. ABET. https://www.abet.org/accreditation/accreditation criteria/criteria for accrediting computing programs-2016-2017/
2. Alstete, J. W. (1995). Benchmarking in Higher Education: Adapting Best Practices to Improve Quality. ERIC Development Team. http://files.eric.ed.gov/fulltext/ED402800.pdf
3. Aoudia, M., Marji, K., & AlQahsi, D. A.-D. (2015). Assessment of Higher Education Quality by Using Cohort of First-year in University. Procedia - Social and Behavioral Sciences, 191, 330–335. https://doi.org/10.1016/j.sbspro.2015.04.310
4. Azma, F. (2011). The Quality Indicators of Information Technology in Higher Education. Procedia - Social and Behavioral Sciences, 30, 2535 2537. https://doi.org/10.1016/j.sbspro.2011.10.494
5. Baeten, M., Struyven, K., & Dochy, F. (2013). Student-centred Teaching Methods: Can They Optimise Students’ Approaches to Learning in Professional Higher Education? Studies in Educational Evaluation, 39(1), 14–22. https://doi.org/10.1016/j.stueduc.2012.11.001
6. Bradley, K. D., Snyder, E. M., & Tombari, A. K. (2015). Higher Education End-of-Course Evaluations: Assessing the Psychometric Properties Utilizing Exploratory Factor Analysis and Rasch Modeling Approaches. International Journal of Assessment Tools in Education, 3(1). https://ijate.net/index.php/ijate/article/view/90
7. Brown, C. (2012). Application of the Balanced Scorecard in Higher Education Opportunities and Challenges. Society for College and University Planning, 40(4), 40–50.
8. Bruwer, J. (1998). First Destination Graduate Employment as Key Performance Indicator: Outcomes Assessment Perspectives. Journal of Institutional Research in Australasia, 8(2), 61–91. http://www.aair.org.au/app/webroot/media/pdf/AAIR Fora/Forum1998/Bruwer.pdf

9. CCELT. (2020). Writing Course Goals/Learning Outcomes and Measurable Learning Objectives. Iowa State University. https://www.celt.iastate.edu/teaching/preparing-to-teach/tips-on-writing-course-goalslearning-outcomes-and-measur
10. Chan, V. (2015). Implications of Key Performance Indicator Issues in Ontario Universities Explored. Journal of Higher Education Policy and Management, 37(1), 41–51. https://doi.org/10.1080/1360080X.2014.991531
11. Dawson, P. (2017). Assessment Rubrics: Towards Clearer and More Replicable Design, Research and Practice. Assessment & Evaluation in Higher Education, 42(3), 347–360. https://doi.org/10.1080/02602938.2015.1111294
12. Dochy, F., Segers, M., & Sluijsmans, D. (2006). The Use of Self-, Peer and Co-assessment in Higher Education: A Review. Studies in Higher Education, 24(3), 331–350. https://doi.org/10.1080/03075079912331379935
13. Douglas, E. M., & Hines, D. A. (2011). The Helpseeking Experiences of Men Who Sustain Intimate Partner Violence: An Overlooked Population and Implications for Practice. Journal of Family Violence, 26(6), 473–485. https://doi.org/10.1007/s10896-011-9382-4
14. ETEC. (2018). National Framework for Public Education Curricula Standards. Education and Training Evaluation Commission. https://etec.gov.sa/en/productsandservices/NCSEE/Cevaluation/Pages/NATIONALFRAMEWORK-.aspx
15. Fernandes, S., Mesquita, D., Flores, M. A., & Lima, R. M. (2014). Engaging Students in Learning: Findings from a Study of Project-led Education. European Journal of Engineering Education, 39(1), 55–67. https://doi.org/10.1080/03043797.2013.833170
16. Gibbs, C. (2003). Explaining Effective Teaching: Self-efficacy and Thought Control of Action. The Journal of Educational Enquiry, 4(2). https://www.ojs.unisa.edu.au/index.php/EDEQ/article/view/520/0
17. Haertel, E. H. (1999). Performance Assessment and Education Reform. Phi Delta Kappan, 80(9), 662. https://www.questia.com/library/journal/1G1-54618911/performance-assessment-and-education-reform
18. Klenowski, V., Askew, S., & Carnell, E. (2006). Portfolios for Learning, Assessment and Professional Development in Higher Education. Assessment & Evaluation in Higher Education, 31(3), 267–286. https://doi.org/10.1080/02602930500352816
19. Kucsera, J. V., & Svinicki, M. (2010). Rigorous Evaluations of Faculty Development Programs. The Journal of Faculty Development, 24(2), 5. https://eric.ed.gov/?id=EJ897466
20. Light, G., Cox, R., & Calkins, S. (2009). Learning and Teaching in Higher Education : The Reflective Professional. Sage Publications Ltd. https://books.google.com.sa/books?id=BDtdBAAAQBAJ&source=gbs_book_other_versions
21. Martin, M., & Sauvageot, C. (2011). Constructing an Indicator System or Scorecard for Higher Education: A Practical Guide. In Intenational Institute for Educational Planning. http://uis.unesco.org/sites/default/files/documents/constructing-an-indicator-system-or-scorecard-for-higher-education-a-practical-guide-2011-en.pdf
22. Mercer-Mapstone, L. D., & Matthews, K. E. (2017). Student Perceptions of Communication Skills in Undergraduate Science at an Australian Research-intensive University. Assessment and Evaluation in Higher Education, 42(1), 98 114. https://doi.org/10.1080/02602938.2015.1084492
23. Mohiuddin, K., Islam, A., Mohd, S., & Shariff, M. (2019). Evaluation of an Academic Program: The Case of Computing Accreditation Commission Framework in Higher Education. International Journal of Emerging Technologies in Learning, 14(11), 70–91. https://online-journals.org/index.php/i-jet/article/view/10178/5719
24. Mohiuddin, K., Rasool, A. M., Mohd, M. S., & Mohammad, R. H. (2019). Skill-Centered Assessment in an Academic Course: A Formative Approach to Evaluate Student Performance and Make Continuous Quality Improvements in Pedagogy. International Journal of Emerging Technologies in Learning, 14(11), 92–106. https://online-journals.org/index.php/i-jet/article/view/10275/5720
25. NCAAA, S. A. (2012). National Commission for Academic Accreditation and Assessment Handbook for Quality Assurance and Accreditation in Saudi Arabia, Part 1-The System for Quality Assurance and Accreditation. NCAAA. http://www.kfupm.edu.sa/deanships/dad/Documents/AAC/NCAAA Documents/H1. Handbook Part 1.pdf
26. Pedrosa de Jesus, H. (2009). The Role of Students’ Questions in Aligning Teaching, Learning and Assessment: A Case Study from Undergraduate Sciences. Assessment & Evaluation in Higher Education, 34, 193–208. https://doi.org/10.1080/02602930801955952
27. Pereira, D., Flores, M. A., & Niklasson, L. (2016). Assessment Revisited: A Review of Research in Assessment and Evaluation in Higher Education. Assessment & Evaluation in Higher Education, 41(7), 1008 1032. https://doi.org/10.1080/02602938.2015.1055233
28. Popham, W. J. (1997, October). What’s Wrong—and What’s Right—with Rubrics. Educational Leadership, 72 75. http://www.ascd.org/publications/educational leadership/oct97/vol55/num02/What’s-Wrong—and-What’s-Right—with-Rubrics.aspx
29. Prins, F. J., de Kleijn, R., & Tartwijk, J. van. (2017). Students’ Use of a Rubric for Research Theses. Assessment & Evaluation in Higher Education, 42(1), 128–150. https://doi.org/10.1080/02602938.2015.1085954
30. QAAHE. (2017). Guidelines for Preparing Programme Specifications. In The Quality Assurance Agency for Higher Education. https://www.tsu.ge/data/file_db/qa_docs/guidelines for programme specifications.pdf
31. Ramsden, P. (1991). A Performance Indicator of Teaching Quality in Higher Education: The Course Experience Questionnaire. Studies in Higher Education, 16(2), 129–150. https://doi.org/10.1080/03075079112331382944
32. Reddy, Y. M., & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education, 35(4), 435 448. https://doi.org/10.1080/02602930902862859
33. Sizer, J., Spee, A., & Bormans, R. (1992). The Rôle of Performance Indicators in Higher Education. Springier Higher Education, 24(2), 133 135. https://link.springer.com/content/pdf/10.1007/BF00129438.pdf
34. Strydom, F. (2017). Higher Education Learning Outcomes Assessment: International Perspectives. Assessment & Evaluation in Higher Education, 42(3), 492–494. https://doi.org/10.1080/02602938.2016.1139097
35. Süral, S. (2017). The Development Study of Thoughts Scale Towards Measurement and Assessment Course on High Education. International Journal of Assessment Tools in Education, 4(1), 79-95. https://dergipark.org.tr/en/pub/ijate/issue/23899/270300
36. Suryadi, K. (2007). Framework of Measuring Key Performance Indicators for Decision Support in Higher Education Institution. In Journal of Applied Sciences Research, 3(12), 1689-1695
37. Taticchi, P., Tonelli, F., & Cagnazzo, L. (2010). Performance Measurement and Management: A Literature Review and a Research Agenda. Measuring Business Excellence, 14(1), 4–18. https://doi.org/10.1108/13683041011027418
38. University of Nottingham Malaysia. (2017). Key Performance Indicators. University of Nottingham Malaysia. https://www.coursehero.com/file/p18pg87/PROS 1010 Guideline-3-Key-Performance-Indicators-State-of-Victoria-2010-Version/
39. University of Technology Sydney. (2010). Seven Propositions for Assessment Reform in Higher Education. In Australian Learning & Teaching Council. https://www.uts.edu.au/sites/default/files/Assessment-2020_propositions_final.pdf