Examination of Differential Item Function for Resilience Scale Items with Latent Classes Based on Intolerance of Uncertainty

Year 2023, , 367 - 386, 01.10.2023

Emine Burcu Tunç , Müge Uluman

https://doi.org/10.37898/spc.2023.8.3.193

Abstract

The concept of resilience encompasses various elements such as spirituality, cultural heritage, adverse life events, and family lineage. Due to this diversity, examining the items measuring resilience, which is one of the concepts evaluated within the scope of positive psychology, differential item function (DIF), is considered important in terms of revealing the structure. As well as determining DIF, there is a need to reveal the reasons for its sources. At this point, the variable intolerance of uncertainty, which is highly related to resilience, is addressed. In this context, the general purpose of this research is to examine whether the resilience scale items show DIF before and after the latent classes have been created within the scope of intolerance of uncertainty. The research, in which the Brief Resilience and Intolerance of Uncertainty scales were used, was conducted with 718 university students. In the first stage of data analyses, likelihood ratio, one of the DIF determination methods, was used. In the second stage, the latent class analysis was carried out to create latent classes within the scope of intolerance of uncertainty. According to the results of this research, all items within the scope of gender for the Brief Resilience scale show a middle level of DIF. Within the scope of Latent Class analysis, it was determined that the four- class model was compatible with the data. After the groups were formed, DIF was examined in terms of gender for the Brief Resilience scale within each group. DIF was not determined in any of the items in class 1 and class 4. However, in class 3, all items showed moderate DIF. It was determined that the DIF results changed after the created latent classes. All these results show that intolerance of uncertainty may be the source of DIF determined in resilience scale items. Therefore, it is recommended to study the interrelated variables together when studying DIF.

Keywords

Resilience, Intolerance of Uncertainty, Differential Item Functioning, Latent Class Analyses

References

• Akbaş, D., & Kahraman, N. (2019). Bireysel farklılıkların kategorik degişkenler olarak modellenmesinde örtük sınıf analizi kullanımı için uygulama kılavuzu: Psikolojik dayanıklılık örneği [A Primer on applied latent class analysis for modeling qualitative differences: An application on resilience data]. Akdeniz Eğitim Araştırmaları Dergisi, 13(29), 356-382. http://doi.org/10.29329/mjer.2019.210.19.
• Arslan, F., & Tuncay, A. (2018). Din ve psikoloji eğitimi alan öğrencilerin psikolojik sağlamlık ve affedicilik düzeylerinin çeşitli değişkenler açısından incelenmesi [Investigation of psychological strength and forgiveness levels of students receiving religion and psychology education in terms of various variables]. Sakarya Üniversitesi Eğitim Fakültesi Dergisi, 19(2), 1-29. https:// dergipark.org.tr/en/pub/sakaefd/issue/51605/651012
• Arslan, Ş.A., & Topal, M. (2021). Covid-19 süresince psiko-sosyal destek hattında çalışan psikologların psikolojik sağlamlık düzeyleri ile erken dönem uyumsuz şemaları arasındaki ilişki [Relationship between psychological resilience and early maladaptive schemas of psychologists who worked at psychological support line during covid-19 pandemic]. Gelişim ve Psikoloji Dergisi, 2(4), 101-118. https://doi.org/10.51503/gpd.977511
• Aydın, İ., Erman, Ö.., Akbulut, V., & Kiliç, S.K. (2019). Öğretmen adaylarinda boş zaman sıkılma algısı ve psikolojik sağlamlık ilişkisi [Psychological resilience and leisure boredom perception in pre-service teacher]. Beden Eğitimi ve Spor Bilimleri Dergisi, 21(1), 39-53. Retrieved from https://dergipark.org.tr/en/pub/ataunibesyo/issue/43861/518563.
• Aydın, M., & Egemberdiyeva, A. (2018). Üniversite öğrencilerinin psikolojik sağlamlık düzeylerinin incelenmesi [An investigation of university students’ resilience levels]. Türkiye Eğitim Dergisi, 3(1), 37-53. Retrieved from https://dergipark.org.tr/en/pub/turkegitimdergisi/issue/37897/333333.
• Bindal, G. (2018). Ergenlerin psikolojik sağlamlığın (resilience), çocukluk çağındaki travma ve bağlanma stilleri ile ilişkisinin incelenmesi [The analysis of the relationship between the resilience of adolescents and trauma in childhood or attachment styles] (Publication No. 498780) [Yüksek lisans tezi, Hasan Kalyoncu Üniversitesi]. Ulusal Tez Merkezi.
• Boelen, P.A., & Lenferink, L.I. (2018). Latent class analysis of indicators of intolerance of uncertainty. Scandinavian Journal of Psychology, 59(3), 243-251.https://doi.org/10.1111/ sjop.12440
• Bozdağ, F. (2020). Pandemi sürecinde psikolojik sağlamlık [Psychological resilience during pandemic]. Electronic Turkish Studies, 15(6)., 247-257.
• Brown, S., & Nagel, L. (2004). Preparing future teachers to respond to stress: Sources and solutions. Action in Teacher Education, 26(1), 34-42. https://doi.org/10.1080/01626620.2004.10463311
• Buhr, K., & Dugas, M.J. (2002). The Intolerance of Uncertainty Scale: Psychometric properties of the English version. Behaviour Research and Therapy, 40(8), 931-945. https://doi.org/10.1016/ S0005-7967(01)00092-4
• Camilli G., & Shepard L.A. (1994). Methods for identifying biased test items (volume 4). SAGE Publications.
• Cantez, E. (2018). Üniversite öğrencilerinin mutluluk, psikolojik sağlamlık ve öz yeterlik düzeyleri arasındaki ilişki [Investigation of happiness, resilience and self efficacy levels in university students].Aydın İnsan ve Toplum Dergisi, 4(2), 61-76.
• Carleton, R.N. (2012). The intolerance of uncertainty construct in the context of anxiety disorders: Theoretical and practical perspectives. Expert Review of Neurotherapeutics, 12(8), 937-947. https://doi.org/10.1586/ern.12.82
• Carleton, R.N., Norton, M.P.J., & Asmundson, G.J. (2007). Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders, 21(1), 105-117. https:// doi.org/10.1016/j.janxdis.2006.03.014
• Çelebi, G.Y. (2020). Covid 19 salgınına ilişkin tepkilerin psikolojik sağlamlık açısından incelenmesi [Investigation of reactions to the Covid 19 outbreak in terms of psychological resilience]. IBAD Sosyal Bilimler Dergisi, 8, 471-483. https://doi.org/10.21733/ibad.737406.
• Çelik, O.B., Demir, G.T., İlhan, E.L., Cicioğlu, İ., & Esentürk, O.K. (2019). Sporcu ergenlerde psikolojik sağlamlık [Psychological resilience in athlete adolescents]. CBÜ Beden Eğitimi ve Spor Bilimleri Dergisi, 14(2), 296-303. https://doi.org/10.33459/cbubesbd.614548.
• Çevik, S., & Yağmur, Y. (2018). Impact of intolerance of uncertainty on psychological well–being in pregnant women with or without miscarriage risk. Perspectives in Psychiatric Care, 54(3),1- 5. https://doi.org/10.1111/ppc.12297.
• Chen, W., Xie, E., Tian, X., & Zhang, G. (2020). Psychometric properties of the Chinese version of the Resilience Scale (RS-14): Preliminary results. PloS One, 15(10), 241606. https://doi. org/10.1371/journal.pone.0241606
• Cheng, Z. (2012). The Relation between uncertainty in latent class membership and outcomes in a latent class signal detection model. [Master’s thesis, Columbia University].
• Cleveland, M.J., Collins, L.M., Lanza, S.T., Greenberg, M.T., & Feinberg, M.E. (2010). Does individual risk moderate the effect of contextual-level protective factors? A latent class analysis of substance use. Journal of Prevention & Intervention in The Community, 38(3), 213-228. https://doi.org/10.1080/10852352.2010.486299
• Collins, L.M., & Lanza, S.T. (2010). Latent class and latent transition analysis: with applications in the social, behavioral, and health sciences. John Wiley & Sons.
• Conner, K.M., & Davidson, J.R. (2003). Development of a new Resilience Scale: The Conner- Davidson Resilience Scale (CD-RISC). Depression and Anxiety, 18, 76-82. https://doi. org/10.1002/da.10113
• Cooke, G.P., Doust, J.A., & Steele, M.C. (2013). A survey of resilience, burnout, and tolerance of uncertainty in Australian general practice registrars. BMC Medical Education, 13(1), 1-6.
• Coutu, D.L. (2002). How resilience works. Harvard Business Review, 80(5), 46-56.
• Day, C., & Gu, Q. (2014). Resilient teachers, resilient schools: Building and sustaining quality in testing times. Routledge.
• Degirmenci, S., Kosger, F., Altinoz, A.E., Essizoglu, A., & Aksaray, G. (2020). The relationship between separation anxiety and intolerance of uncertainty in pregnant women. The Journal of Maternal-Fetal & Neonatal Medicine, 33(17), 2927-2932. https://doi.org/10.1080/1 4767058.2018.1564030
• Doğan, T. (2015). Kısa psikolojik sağlamlık ölçeği’nin Türkçe uyarlaması: Geçerlik ve güvenirlik çalışması [Adaptation of the Brief Resilience Scale into Turkish: A validity and reliability study]. The Journal of Happiness & Well-Being, 3(1), 93-102.
• Doğan, T., & Yavuz, K. (2020). Yetişkinlerde psikolojik sağlamlık, olumlu çocukluk deneyimleri ve algılanan mutluluk [Resilience, positive childhood experiences and perceived happiness among adults]. Psikiyatride Güncel Yaklaşımlar, 12, 312-330.
• Dugas, M. J., Buhr, K., & Ladouceur, R. (2004). The role of intolerance of uncertainty in etiology and maintenance. In R. G. Heimberg, C.L. Turk, & D. S. Mennin (Eds.), Generalized anxiety disorder: Advances in research and practice. The Guilford Press.
• Dugas, M. J., Gosselin, P., & Ladouceur, R. (2001). Intolerance of uncertainty and worry: Investigating specificity in a nonclinical sample. Cognitive Therapy and Research, 25, 551-558.
• Dugas, M. J., Marchand, A., & Ladouceur, R. (2005). Further validation of a cognitive-behavioral model of generalized anxiety disorder: Diagnostic and symptom specificity. Journal of Anxiety Disorders, 19(3), 329-343. https://doi.org/10.1016/j.janxdis.2004.02.002
• Durna, N. B., Durna, D., & Seçer, İ. (2022). The mediating role of resilience in the relationship between emotional reactivity, intolerance of uncertainty and psychological maladjustment in children receiving orthodontic treatment. In Healthcare, 10(8), 1505. https://doi.org/10.3390/ healthcare10081505
• Einstein, D.A. (2014). Extension of the transdiagnostic model to focus on intolerance of uncertainty: a review of the literature and implications for treatment. Clinical Psychology: Science and Practice, 21(3), 280. https://doi.org/10.1111/cpsp.12077
• Ella, G.., Brody, H., Carrington, C.J.,& Rhonda, M. (2021). A psychometric evaluation of the Connor–Davidson Resilience Scale for Australian Aboriginal youth, Australian Psychologist, 56:1, 25-37. https://doi.org/10.1080/00050067.2020.1829453.
• Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates, Inc.
• Erata, F., & Özbey, S (2020). Okul öncesi eğitim kurumuna devam eden çocuklarin psikolojik sağlamlik düzeylerinin bazi demografik değişkenlere göre incelenmesi [Examining the psychological resilience levels of children attending pre-school education institutions according to some demographic variables]. Pearson Journal, 5(9), 125-151. https://doi.org/10.46872/pj.171
• Ergün, O. (2016). Ergenlerde duygusal zeka özellikleri ile psikolojik sağlamlık arasındaki ilişkinin incelenmesi [Relationship between resilience and emotional ı̇ntelligence of adolescent] (Publication No. 424962) [Yüksek Lisans Tezi, İstanbul Arel Üniversitesi]. Ulusal Tez Merkezi.
• Francis, K., Dugas, M. J., & Ricard, N. C. (2016). An exploration of Intolerance of Uncertainty and memory bias. Journal of Behavior Therapy and Experimental Psychiatry, 52, 68-74. https://doi. org/10.1016/j.jbtep.2016.03.011
• Gentes, E. L., & Ruscio, A.M. (2011). A meta-analysis of the relation of intolerance of uncertainty to symptoms of generalized anxiety disorder, major depressive disorder, and obsessive–compulsive disorder. Clinical Psychology Review, 31(6), 923-933. https://doi.org/10.1016/j.cpr.2011.05.001
• Gorman, E., Heritage, B., Shepherd, C.C., & Marriott, R. (2021). A psychometric evaluation of the Connor–Davidson Resilience Scale for Australian Aboriginal youth. Australian Psychologist, 56(1), 25-37.
• Greer, T.G. (2004). Detection of differential item functioning (DIF) on the SATV: A comparison of four methods: Mantel-Haenszel, logistic regression, simultaneous item bias, and likelihood ratio test. [Master’s thesis, University of Houston].
• Güngör Çulha, D. (2012). Örtük sınıf analizlerinde ölçme eşdeğerliğinin incelenmesi [Investigating measurement equivalence with latent class analysis] (Publication No. 724982) [Doktora Tezi, Ege Üniversitesi]. Ulusal Tez Merkezi.
• Holaway, R.M., Heimberg, R.G., & Coles, M.E. (2006). A comparison of intolerance of uncertainty in analogue obsessive-compulsive disorder and generalized anxiety disorder. Journal of Anxiety Disorders, 20(2), 158-174. https://doi.org/10.1016/j.janxdis.2005.01.002
• Hoşoğlu, R., Kodaz, A.F., Bingöl, T.Y., & Batik, M.V. (2018). Öğretmen adaylarında psikolojik sağlamlık [The resilient levels of preservice teachers]. OPUS International Journal of Society Researches, 8(14), 217-239.
• Kalaycıoğlu, D.B., & Kelecioğlu, H. (2011). Öğrenci Seçme Sınavı’nın madde yanlılığı açısından incelenmesi [Item bias analysis of the university entrance examination]. Eğitim ve Bilim, 36(161), 3-13.
• Kankaras, M., Moors, G., & Vermunt, J.K. (2010). Testing for measurement invariance with latent class analysis. In E. Davidov, P. Schmidt, & J. B. Billiet (Eds), Cross-Cultural analysis: Methods and applications. Routledge.
• Karakaya, İ., & Kutlu, Ö. (2012). Seviye belirleme sınavındaki Türkçe alt testlerinin madde yanlılığının incelenmesi [An investigation of item bias in Turkish sub tests in level determination exam]. Eğitim ve Bilim, 37(165), 348-362.
• Karal, E., & Biçer, B.G. (2021). Salgın hastalık dönemindeki üniversite öğrencilerinin psikolojik sağlamlık düzeylerinin bazı değişkenler açısından incelenmesi [Examining the psychological resilience levels of university students in the epidemic period in terms of some variables]. Uluslararası Sosyal Bilimler ve Eğitim Dergisi, 3(4), 17-34.
• Karataş, Z. & Tagay, Ö. (2021). The relationships between resilience of the adults affected by the covid pandemic in Turkey and Covid-19 fear, meaning in life, life satisfaction, intolerance of uncertainty and hope. Personality and Individual Differences, 172, 110592. https://doi. org/10.1016/j.paid.2020.110592.
• Kimter, N. (2020). Covid-19 günlerinde bireylerin psikolojik sağlamlık düzeylerinin bazı değişkenler açısından incelenmesi [Examining the psychological resilience levels of individuals in the days of Covid-19 in terms of some variables].IBAD Sosyal Bilimler Dergisi, Özel Sayı, 574-605. http://doi.org/10.21733/ibad.805481.
• Kobasa, S.C. (1979). Personality and resistance to illness. American Journal of Community Psychology, 7(4), 413-423. https://doi.org/10.1007/BF00894383
• Lanza, S.T., Flaherty, B.P., & Collins, L.M. (2003). Latent class and latent transition analysis. John A. Liu, V.Y.Y., & Lim, S.M. (2022). A psychometric evaluation of the brief resilience scale among tertiary students in Singapore. Asia Pacific Journal of Education, 42(3), 464-477.
• Lukočienė, O., Varriale, R., & Vermunt, J.K. (2010). The simultaneous decision(s) about the number of lower and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40(1), 247-283. http://doi.org/10.1111/j.1467-9531.2010.01231.
• Majid, A., & Pragasam, J. (1997). Interactions of intolerance of ambiguity and of contingent liability on auditors’ avoidance of litigation. Psychological Reports, 81(3), 935-944. https://doi. org/10.2466/pr0.1997.81.3.935
• Masten, A. S. (1994). Resiliency in individual development. Successful adaptation despite risk and adversity. Wang M.C. Gordon, E.W. (Eds.) Educational Resiliency in Inner City America. Challenges and Prospects. Hillssdale.
• Masten, A., & Reed, M. (2005). Resilience in development. In C.R. Snyder & S.J. Lopez (Eds.), Handbook of positive psychology (pp. 74-88). Oxford University Press.
• Moors, G., & Wennekers, C. (2003). Comparing moral values in Western European countries between 1981 and 1999. A multiple group latent-class factor approach. International Journal of Comparative Sociology, 44(2), 155-172. https://doi.org/10.1177/002071520304400203
• Neenan, M. (2009). Developing resilience: A cognitive-behavioural approach. Routledge.
• Nylund, K.L., Asparouhov, T., & Muthen, B.O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14(4), 535-569. https://doi.org/10.1080/10705510701575396
• Oktan, V., Odacı, H., & Çelik, Ç.B. (2014). Psikolojik doğum sırasının psikolojik sağlamlığın yordanmasındaki rolünün incelenmesi [Investigating the role of psychological birth order in predicting resilience]. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 14(1), 141-152.
• Önder, A., & Gülay, H. (2008). İlköğretim 8. sınıf öğrencilerinin psikolojik sağlamlığının çeşitli değişkenler açısından incelenmesi [Resilience of 8 grade students in relation to various variables]. Dokuz Eylül Üniversitesi Buca Eğitim Fakültesi Dergisi, 23, 192-197.
• Osterlind, J.S. (1983). Test İtem Bias. Sage Publications.
• Özkapu, Y. (2019). Suriyeli çocuklarla çalışan psikolojik danışmanların, kişilik özellikleri ve öz yeterlik algılarının psikolojik sağlamlıkları üzerindeki etkisi [The effect of psychological counselors working with Syrian children on resilience of personality and self-efficiency perceptions] (Publication No. 565728) [Doktora Tezi, Marmara Üniversitesi]. Ulusal Tez Merkezi.
• Polat Başpınar, D. (2021). Pamukkale Üniversitesi kadın hastalıkları ve doğum polikliniğine başvuran gebelerin, gebeliğe uyum, psikolojik sağlamlık ve bunları etkileyen faktörlerin değerlendirilmesi [Assessment of pregnancy, psychological stability and the factors affecting them of the pregnant women’s consulted women diseases and maternity clinic of Pamukkale University] (Publication No. 379400) [Yüksek Lisans Tezi, Pamukkale Üniversitesi]. Ulusal Tez Merkezi.
• Roever, C. (2005) Testing ESL Pragmatics. Peter Lang.
• Rutter, M. (2006). Implications of resilience concepts for scientific understanding. Annals of the New York Academy of Sciences, 1094, 1-12.http:// doi:10.1196/annals.1376.002.
• Sarıçam, H., Deveci, M., & Ahmetoğlu, E. (2020). The examination of hope, intolerance of uncertainty and resilience levels in parents having disabled children. Global Journal of Psychology Research: New Trends and Issues, 10(1), 118-131. http://doi.org/10.18844/gjpr.v10i1.4398.
• Sarıçam, H., Erguvan, F.M., Akın, A., & Akça, M.Ş. (2014). The Turkish short version of the intolerance of uncertainty (IUS-12) scale: The study of validity and reliability. Route Educational and Social Science Journal, 1(3), 148-157.
• Schinka, J. A., Velicer, W. F., & Weiner, I. B. (2013). Handbook of psychology: Research methods in psychology, Vol. 2. John Wiley & Sons, Inc.
• Seong, H., Resnick, B., Holmes, S., Galik, E., Breman, R. B., Fortinsky, R. H., & Zhu, S. (2023). Psychometric properties of The Resilience Scale in older adults post-hip fracture. Journal of Aging and Health, 13 https://doi.org/08982643231184098.
• Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Jennifer Bernard, J. (2008). The brief resilience scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine, 15, 194–200. https://doi.org/10.1080/10705500802222972.
• Thissen, D. (2001). Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Chapel Hill.
• Tuck, I., & Anderson, L. (2014). Forgiveness, flourishing, and resilience: the influences of expressions of spirituality on mental health recovery. Issues in Mental Health Nursing, 35, 277– 282. https://doi.org/10.3109/01612840.2014.885623.
• Tura, G., & Doğan, B.B. (2020). Okul psikolojik danışmanlarının (rehber öğretmenlerin) psikolojik sağlamlığının demografik değişkenler açısından incelenmesi [Examination of the resilience of school psychological counselors (guide teachers) in terms of demographic variables]. In Saracaloğlu, A.S. (Ed.), 6th International congress on social sciences & humanities and education - full and abstract text book (p. 39-51). Güven Plus Grup A.Ş. Publications.
• Turgut, Ö. (2016). Ergenlerin psikolojik sağlamlık düzeylerinin, önemli yaşam olayları, algılanan sosyal destek ve okul bağlılığı açısından incelenmesi [Investigation of resilience of adolescents in terms of major life events, perceived social support and school engagement] (Publication No. 395175) [Doktora Tezi, Anadolu Üniversitesi]. Ulusal Tez Merkezi.
• Tusaie, K., & Dyer, J. (2004). Resilience: A historical review of the construct. Holistic Nursing Practice, 18(1), 3-8.
• Ulukan, M. (2020). Öğretmenlerin mutluluk ile psikolojik sağlamlik düzeyleri arasindaki ilişkinin incelenmesi [Investigation of the relationship between happiness and psychological resilience levels of teachers]. Journal of International Social Research, 13(73), 620-631.
• Vermunt, J.K. (2003). Multilevel latent class models. Sociological Methodology, 33(1), 213-239. https://doi: 10.1111/j.0081-1750.2003.t01- 1-00131.
• Vermunt, J.K., & Magidson, J. (2004). Latent class analysis. The Sage Encyclopedia of Social Sciences Research Methods, 2, 549-553.
• Volarov, M., Saulnier, K.G., Allan, N.P., Shapiro, M.O., & Mihić, L. (2021). Are we still uncertain about the latent structure of intolerance of uncertainty: Results from factor mixture modeling in a Serbian sample. Journal of Affective Disorders, 294, 505-512. https://doi.org/10.1016/j. jad.2021.07.081.
• Weick, K.E., & Sutcliffe, K.M. (2011). Managing the unexpected: Resilient performance in an age of uncertainty (8th ed.). CA: John Wiley & Sons.
• Weston, K.J., & Parkin, J.R. (2010). From at-risk to at-promise: The resilience movement. In C. S. Clauss-Ehlers (Eds.), Encyclopedia of Cross-Cultural School Psychology (pp. 808-815). Springer.
• Wongpakaran, T., Yang, T., Varnado, P., Siriai, Y., Mirnics, Z., Kövi, Z., & Wongpakaran, N. (2023). The development and validation of a new resilience inventory based on inner strength. Scientific Reports, 13(1), 2506. https://doi.org/10.1038/s41598-023-29848-7
• Yildiz, B., & Iskender, M. (2021). The secure attachment style oriented psycho-educational program for reducing intolerance of uncertainty and academic procrastination: Yıldız & Iskender: Secure attachment, academic procrastination & intolerance of uncertainty. Current Psychology, 40(4), 1850-1863.
• Yüksel, B. (2014). Attachment, positive and negative emotion regulation, and intolerance of uncertainty in anxiety: searching for an integrative model. (Publication No. 368995) [Master Thesis, University of Hacettepe]. Ulusal Tez Merkezi.
• Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic Regression modeling as a unitary framework for binary and likert-type (ordinal) item scores. ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.