The Psychometric Properties of the Raven Intelligence Test (RIT): Insights into Validity, Reliability, Item Difficulty, and Item Discrimination

Abstract

The research aimed to validate the Raven Intelligence Test (RIT) in the Jordanian context using a sample of 414 students selected from secondary schools in Amman, Jordan aged between 15 and 17 years. The study yielded several findings. Notably, in terms of validity, the study assessed predictive validity through school achievement, concurrent validity using the School and College Ability Test (SCAT), construct validity by comparing scores of normal and gifted students, discrimination power through extreme score differences, and factorial validity via confirmatory factor analysis (CFA). The reliability of the test was evaluated using Cronbach’s (α), split-halves, test-retest method, and the correlation between each item and the total score. We also calculated item difficulty and item discrimination in order to evaluate the quality of the test as well as its ability to discriminate among the intelligence levels. These favorable outcomes provide additional support for the reliability and validity of the test in measuring cognitive abilities which may further prove useful in contexts such as talent identification and recruitment.

Keywords

• Raven Intelligence Test
• RIT
• Validity
• Reliability
• Confermatory Factor Analysis

Supporting Institution

This study did not receive any financial funding

Ethical Statement

the authors do not possess a letter from the ethical committee as there is no such committee at the university from which sampled to conduct The Psychometric Properties of the Raven Intelligence Test (RIT): Insights into Validity, Reliability, Item Difficulty, and Item Discrimination

Thanks

No one

References

1. Abdalgadr, A. (2009). Standardization of Raven's Standard Progressive Matrices Test for a Libyan sample (Unpublished doctoral dissertation). University of Salford.
2. Abdel-Khalek, A. M. (1988). Egyptian results on the Standard Progressive Matrices. Personality and Individual Differences, 9(1), 193–195. https://doi.org/dfxbpd
3. Abu Al-Qasim, M. (2014). Standardization of the RIT of John Raven to measure general intelligence on a sample of students who are gifted, intellectually talented, and have learning difficulties in the state of Khartoum (Unpublished doctoral dissertation). Umm Durman Islamic University, College of Arts.
4. Abu Ghali, E., & Abu Mustafa, N. (2014). Standardization of the RIT of Raven for the age group (8–18) years for general education students in the governorates of Gaza. Taibah University Journal for Educational Sciences, 9(1), 90–108.
5. Alarcón Paz, C., Díaz Valenzuela, V., Hernández Rosales, J., & Estrada Goic, C. (2012). Estudio sobre la pertinencia del uso de las normas disponibles del Raven en adultos mayores chilenos [Study on the relevance of using available Raven norms in Chilean older adults]. Psico-USF, 17(3), 387–395. https://doi.org/d22m
6. Al-Bokai, H. (2017). A gifted education model for Jordan: A study of Johns Hopkins University Center for Talented Youth. Dirasat, Educational Sciences, 44(4, Supplement 1), 413–427. Jordan University.
7. Al-Bokai, H., Al-Omari, J., & Al-Rahmaneh, A. (2014). Development of the School and College Ability Tests advanced form in Jordan. The Educational Journal, Ain Shams University, 38, 1–12. Egypt.
8. Al-Manizel, A., & Al-Atoum, Y. (2010). Research methods in educational and psychological sciences. Sharjah: University Library.

9. Al-Nefaie, A. (2001). Standardisation of the Advanced Progressive Matrices (APM) for intermediate and secondary stages' students in Makkah Al-Mukarramah region (Unpublished master's thesis). Umm Al Qura University.
10. Al-Shahomee, A. A. (2012). A standardization of the Standard Progressive Matrices for adults in Libya. Personality and Individual Differences, 53(2), 142–146. https://doi.org/gftztx
11. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Prentice-Hall.
12. Angelini, A. L., Alves, I. C. B., Custódio, E. M., Duarte, W. F., & Duarte, J. L. M. (1999). Manual. Matrizes progressivas coloridas de Raven: Escala especial. São Paulo: Centro Editor de Testes e Pesquisas em Psicologia.
13. Bennett, G. K., Seashore, H. G., & Wesman, A. G. (1997). Differential aptitude tests (5th ed.). The Psychological Corporation.
14. Bilker, W. B., Hansen, J. A., Brensinger, C. M., Richard, J., Gur, R. E., & Gur, R. C. (2012). Development of abbreviated nine-item forms of the Raven's Standard Progressive Matrices Test. Assessment, 19(3), 354–369. https://doi.org/f34ssx
15. Bors, D. A., & Stokes, T. L. (1998). Raven's Advanced Progressive Matrices: Norms for first-year university students and the development of a short form. Educational and Psychological Measurement, 58(3), 382–398. https://doi.org/10.1177/0013164498058003002
16. Brand, C. (1996). The g factor: General intelligence and its implications. Wiley.
17. Brookhart, S. M., & Nitko, A. J. (2019). Educational assessment of students. Pearson.
18. Cardwell, M. (2014). Dictionary of psychology. Routledge.
19. Carpenter, P. A., Just, M. A., & Shell, P. (1990). What one intelligence test measures: A theoretical account of the processing in the Raven Progressive Matrices Test. Psychological Review, 97(3), 404–431. https://doi.org/10.1037/0033-295X.97.3.404
20. Cattell, R. B. (1973). Culture fair intelligence test. Journal of Educational Psychology.
21. Chopra, D. (2024). Science behind the Raven's Progressive Matrices Test. Adaface. https://t.ly/6SWiG
22. Clemens, N., Ragan, K., & Prickett, C. (2018). Predictive validity. In The SAGE encyclopedia of educational research, measurement, and evaluation (Vol. 4, p. 1289). SAGE Publications, Inc. https://doi.org/10.4135/9781506326139
23. Deary, I. J., Cox, S. R., Hill, W. D. (2021). Genetic variation, brain, and intelligence differences. Molecular Psychiatry. https://doi.org/10.1038/s41380-021-01027-y
24. DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE Publications. Eid, Gh. (2005). Evaluate the structure of the Advanced Progressive Matrices (APM) and the short form of it using factor analysis and the Rush model. Journal of Psychological and Educational Research, 3rd Issue, 256–283.
25. Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11
26. Flores-Mendoza, C., Ardila, R., Gallegos, M., & Reategui-Colareta, N. (2021). General intelligence and socioeconomic status as strong predictors of student performance in Latin American schools: Evidence from PISA items. Frontiers in Education, 6, Article 632289. https://doi.org/10.3389/feduc.2021.632289
27. Fosnacht, K., Sarraf, S., Howe, E., & Peck, L. K. (2017). How important are high response rates for college surveys? The Review of Higher Education, 40(2), 245–265. https://doi.org/10.1353/rhe.2017.0003
28. Golan, D., Wilken, J., Doniger, G. M., Fratto, T., Kane, R., Srinivasan, J., Zarif, M., Bumstead, B., Buhse, M., Fafard, L., Topalli, I., & Gudesblatt, M. (2019). Validity of a multi-domain computerized cognitive assessment battery for patients with multiple sclerosis. Multiple Sclerosis and Related Disorders, 30, 154–162. https://doi.org/gf2z
29. Greenfield, P. M. (1998). The cultural evolution of IQ. In U. Neisser (Ed.), The rising curve: Long-term gains in IQ and related measures (pp. 81–123). American Psychological Association. https://doi.org/10.1037/10270-003
30. Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modeling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60. Retrieved from www.ejbrm.com. Irrgang, M., Dorenkamp, M., Reohr, P., & Vik, P. (2019). C-28 Raven's Progressive Matrices: Validation of a short form. Archives of Clinical Neuropsychology, 34(6), 1057. https://doi.org/f6tr
31. Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., & DeYoung, C. G. (2016). Openness to experience and intellect differentially predict creative achievement in the arts and sciences. Journal of Personality, 84(2), 248-258. https://doi.org/10.1111/jopy.12163
32. Khalifa, H. (2011). Standardization of the RIT in the state of Khartoum. Psychological Studies - Al-Basrah Center for Research, Consulting, and Educational Services, 5, 41-76.
33. Kline, R. B. (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.
34. Levine, R., & Lord, F. M. (1959). An index of the discriminating power of a test at different parts of the score range. Educational and Psychological Measurement, 19(4), 497-503. https://doi.org/10.1177/001316445901900401
35. Michael, M. (2006). Measurement and evaluation in modern education (5th ed.). Damascus University Publications.
36. Michael, M. (2006a). Psychometric measurement: First part. Damascus University Publications.
37. Michael, M. (2015). Psychological and educational measurement and evaluation of the normal and special needs. Al-Easar Al-Elmi House for Publishing and Distribution.
38. Moran, A. P. (1986). The reliability and validity of Raven's Standard Progressive Matrices for Irish apprentices. Applied Psychology, 35(4), 533–538. https://doi.org/dc9mvh
39. Muhammad, A. (2017). Children's performance scores on the RIT to measure intelligence: A field study of the El Fasher and Nyalas' cities. Journal of Educational and Psychological Sciences, 1(1), 127-143.
40. Murphy, P., Foley, J., Mole, J., Van Harskamp, N., & Cipolotti, L. (2023). Lifespan normative data (18–89 years) for Raven's Advanced Progressive Matrices Set I. Journal of Neuropsychology, 17(2), 417-429. https://doi.org/10.1111/jnp.12294.
41. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
42. Nurhudaya, A., Taufik, E. S., Yudha, D., & Suryana, D. (2019). The Raven’s advanced progressive matrices in educational assessment with a Rasch analysis. Universal Journal of Educational Research, 7(9), 1996–2002. https://doi.org/10.13189/ujer.2019.070914
43. Peter, J. P. (1981). Construct validity: A review of basic issues and marketing practices. Journal of Marketing Research, 18(2), 133-145. https://doi.org/10.1177/002224378101800202
44. Piedmont, R. L. (2023). Factorial validity. In F. Maggino (Ed.), Encyclopedia of quality of life and well-being research (pp. 3466–3471). Springer. https://doi.org/10.1007/978-3-031-17299-1_984
45. Pind, J., Gunnarsdóttir, E. K., & Jóhannesson, H. S. (2003). Raven's Standard Progressive Matrices: New school age norms and a study of the test's validity. Personality and Individual Differences, 34(3), 375–386. https://doi.org/10.1016/S0191-8869(02)00096-9
46. Powers, S., & Barkan, J. H. (1986). Concurrent validity of the Standard Progressive Matrices for Hispanic and non-Hispanic seventh-grade students. Psychology in the Schools, 23(4), 333–336. https://doi.org/10.1002/1520-6807(198610)23:4<333::AID-PITS2310230405>3.0.CO;2-X
47. Rahma, A. (2004). The effectiveness of using time series analysis and regression analysis in studying intelligence in individuals from seven to eighteen years old (Unpublished doctoral dissertation). University of Damascus.
48. Raven, J. C. (1938). Progressive matrices: A perceptual test of intelligence: Individual form. Lewis.
49. Raven, J., Raven, J. C., & Court, I. H. (1998). Manual for Raven's Progressive Matrices and Vocabulary Scales: Section 1: General overview. Oxford Psychologists Press.
50. Revicki, D. (2014). Internal consistency reliability. In A. C. Michalos (Ed.), Encyclopedia of quality of life and well-being research (pp. 1881–1883). Springer. https://doi.org/10.1007/978-94-007-0753-5_1494
51. Roid, G. H. (2003). Stanford-Binet intelligence scales for early childhood (Early SB5) (5th ed.). Riverside Publishing.
52. Rushton, J. P., Skuy, M., & Bons, T. A. (2004). Construct validity of Raven's Advanced Progressive Matrices for African and non-African engineering students in South Africa. International Journal of Selection and Assessment, 12(3), 220–229. https://doi.org/10.1111/j.0965-075X.2004.00260.x
53. Savage, M. (2017). Reliability, split-half. In The Sage Encyclopedia of Communication Research Methods (Vol. 4, p. 1421). SAGE Publications, Inc. https://doi.org/10.4135/9781483381411
54. Tigza, B. (2012). Exploratory and confirmatory factor analysis. Dar Al-Maseera.
55. Ursachi, G., Horodnic, I. A., & Zait, A. (2015). How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Economics and Finance, 20, 679-686. https://doi.org/10.1016/S2212-5671(15)00068-X
56. Van der Ven, A. H. G. S., & Ellis, J. L. (2000). A Rasch analysis of Raven’s standard progressive matrices. Personality and Individual Differences, 29(1), 45–64. https://doi.org/10.1016/S0191-8869(99)00154-X
57. Vilagut, G. (2014). Test-retest reliability. In A. C. Michalos (Ed.), Encyclopedia of quality of life and well-being research (pp. 6656–6659). Springer. https://doi.org/10.1007/978-94-007-0753-5_3001
58. Wechsler, D. (1997). Wechsler adult intelligence scale (3rd ed.). Psychological Corporation.
59. Wechsler, D. (2003). Wechsler intelligence scale for children: Technical manual (4th ed.). Psychological Corporation.
60. Wechsler, D. (2012). Wechsler preschool and primary scale of intelligence—fourth edition. Psychological Corporation.
61. Wicherts, J. M., Dolan, C. V., Carlson, J. S., & van der Maas, H. L. (2010). Raven's test performance of sub-Saharan Africans: Average performance, psychometric properties, and the Flynn effect. Learning and Individual Differences, 20(3), 135-151. https://doi.org/10.1016/j.lindif.2009.08.002
62. Williams, J. E., & McCord, D. M. (2006). Equivalence of standard and computerized versions of the Raven Progressive Matrices Test. Computers in Human Behavior, 22(5), 791–800. https://doi.org/10.1016/j.chb.2005.12.010
63. Wongupparaj, P., Kumari, V., & Morris, R. G. (2015). A cross-temporal meta-analysis of Raven's Progressive Matrices: Age groups and developing versus developed countries. Intelligence, 49, 1-9. https://doi.org/10.1016/j.intell.2015.02.003
64. Yahya, A., Ibrahim, A., & Jalal, S. (2007). Standardization of the RIT in the Omani environment. Childhood Studies, 35, 35-46.