Rasch-Based Objective Standard Setting for University Placement Test

Year 2019, Volume: 19 Issue: 84, 57 - 70, 30.11.2019

Ado Abdu Bıchı Rohaya Talıb Rahimah Embong Hasnah Binti Mohamed Mohd Sani Ismaıl Abdallah Ibrahım

https://doi.org/10.14689/ejer.2019.84.3

Abstract

Purpose: University placement test is an important admission policy priority in Nigeria, because it serves as a university-based selection criterion for placement of students into undergraduate programs in Nigeria. Although recently attention have been shifted on the call to develop a standard content and standardize the test, yet attention has not been paid on the development of standard setting in which the decisions to select or reject the applicants are made. This study; therefore, investigated the application Rasch-based Objective Standard Setting (OSS) to establish the standards in a university placement test.
Methods: To demonstrate the application of OSS, 9 judges were employed for the conduct of standard ratings; the data used for Rasch calibration with WINSTEP were the responses of 600 students on the 60 items validated Economics Placement Test (EPT).
Findings: The experts’ ratings and Rasch generated logits (item difficulties) were used in quantifying the set of essential items selected. Results of OSS produced the cut-scores of <-0.62 into Basic, -0.62 logits into Proficient and 0.02 logits for Advanced performance levels. The examinees in these categories were, 39% at Basic, 32% at Proficient and 29% at Advanced performance levels.
Implications for Research and Practice: The results of this OSS provide performance levels with a clear content related description to informed decision on students’ mastery of the content in EPT. It is recommended that, results from the OSS should be compared with other existing IRT-based methods in similar study to ascertain its external validity.

Keywords

rasch measurement, placement test, cut-scores, objective standard setting

References

• Akanwa, U. N. & Nkwocha, P. C. (2015). Prediction of South Eastern Nigerian students under graduate scores with their UME and Post-UME Scores. IOSR Journal of Research & Method in Education, 5(5), PP 36-39
• Angoff, W. (1971). Scales, norms and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (pp 506-600). Washington, DC: American Council on Education.
• Atkinson, R. C., & Geiser, S. (2009). Reflections on a century of college admissions tests. Educational Researcher, 38(9), 665-676.
• Barlow, P. B. (2014). Development of the Biostatistics and clinical epidemiology skills assessment for medical residents. Doctoral thesis, University of Tennessee
• Bejar, I. I. (2008). Standard setting: What is it? Why is it important. R& D Connections, 7,1-6.
• Bichi, A. A. (2015). Analysis of UTME and Post-UTME scores of education students at Northwest University Kano-Nigeria. In 1st International Conference on Education. Held on 9th-10th April 2015 at Novotel Beijing Xinqiao, Beijing China
• Bichi, A. A., Talib, R., Atan, N. A., Ibrahim, H. & Yusof, S. M. (2019). Validation of a developed university placement test using classical test theory and Rasch measurement approach. International Journal of Advanced and Applied Sciences, Volume-6, Issue-6, Pp: 22-29
• Bichi, A. A., Talib, R., Mohamed, H., Ahamad, J. & Khairuddin, N. (2019). Exploratory sequential design to develop and validate Economics placement test for Nigerian universities. International Journal of Recent Technology and Engineering (IJRTE), Volume-7, Issue-6S5, Pp: 769-772
• Carey, K., & Manwaring, R. (2011). Growth Models and Accountability: A recipe for remaking ESEA. Education Sector Reports. Education Sector.
• Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. SAGE Publications Ltd.
• Ebel, R. L. (1979). Essentials of educational measurement, Prentice-Hall. Englewood Cliffs, NJ.
• Ikoghode, A. (2015). Post-UTME screening in Nigerian universities: How relevant today?. International Journal of Education and Research, 3(8), 101-116.
• Karasar, N. (2016). Scientific research method. Ankara: Nobel Akademik Yayincilik
• Khatimin, N., Aziz, A. A., Zaharim, A., & Yasin, S. H. M. (2013). Development of objective standard setting using Rasch measurement model in Malaysian institution of higher learning. International Education Studies, 6(6), 151-160.
• Khatimin, N., Zaharim, A., & Aziz, A. A. (2014, December). Standard setting in students’ assessment of higher education institution in Malaysia. In 2014 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE) (pp. 499-504). IEEE.
• Lewis, D. M. (1996). Standard setting: A bookmark approach. In IRT-based standard setting procedures utilizing behavioural anchoring, DR Green (Chair), Symposium conducted at the Council of Chief State School Officers National Conference on Large-Scale Assessment, Phoenix, AZ, 1996.
• Linacre, J. M. (2019). A User’s Guide to WINSTEPS MINISTEP: Rasch-Model Computer Programme. Online http://www.winsteps.com/winsteps.htm
• MacDougall, M., & Stone, G. E. (2015). Fortune-tellers or content specialists: Challenging the standard setting paradigm in medical education programmes. J Contemp Med Edu, 3(3), 135.
• Nedelsky, L. (1954). Absolute grading standards for objective tests. Educational and Psychological Measurement, 14(1), 3-19.
• Sanz, I & Fernández, M. (2005). The university entrance exam in Spain: Present situation and possible solutions. Presented as a poster in the 2005 EALTA Conference
• Silber, K. H., & Foshay, W. R. (Eds.). (2010). Handbook of improving performance in the workplace (I). San Francisco, CA: Pfeiffer.
• Sondergeld, T. A., Stone, G. E., & Kruse, L. M. (2018). Objective standard setting in educational assessment and decision making. Educational Policy, 0895904818802115.
• Stone, G. E. (1996). The construction of meaning: Replicating objectively derived criterion-referenced standards. In annual meeting of the American Educational Research Association. New York, NY.
• Stone, G. E. (2001). Objective standard setting (or truth in advertising). Journal of applied measurement, 2(2), 187-201.
• Stone, G. E., Koskey, K. L., & Sondergeld, T. A. (2011). Comparing construct definition in the Angoff and Objective Standard Setting models: Playing in a house of cards without a full deck. Educational and psychological measurement, 71(6), 942-962.
• Tas, H., & Minaz, M. B. (2019). An investigation into examination-type preferences of primary school students in relation to various variables. Eurasian Journal of Educational Research (EJER), (81).
• Uhunmwuangho, S. O., & Ogunbadeniyi, O. (2014). The university matriculation examination as a predictor of performance in post university matriculation examination: A model for educational development in the 21st century. African Research Review, 8(1), 99-111.