Yıl 2019, Cilt 10 , Sayı 3, Sayfalar 235 - 248 2019-09-04

The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters

Osman TAT [1] , İlhan KOYUNCU [2] , Selahattin GELBAL [3]


Uluslararası Öğrenci Değerlendirme Programı (PISA) ve Uluslararası Matematik ve Fen Eğilimleri Çalışması (TIMSS) gibi geniş ölçekli uygulamalarda öğrenci yeteneğine ilişkin kestirimler olarak makul değerler kullanılır. Bu çalışmalarda katılımcılar tabakalı örnekleme yöntemi ile çekilmektedir. Bu durum elde edilen verilerin, çok sayıdaki tabakadan oluşan hiyerarşik bir yapıda olmasının önünü açmaktadır. Geniş ölçekli değerlendirme çalışmaları bağlamında makul değerler, sonsal yetenek dağılımından rastgele elde edilen değerler olarak tanımlanmaktadır. Doğrusal modellerde tek bir makul değerin veya tüm makul değerlerin ortalamasının bağımsız değişken olarak kullanılmasının yanlı sonuçlara sebep olabildiği bilinmektedir. Aynı zamanda bu geniş ölçekli çalışmaların verileri ile analizler yapılırken örnekleme ağırlıklarının göz ardı edildiği sıkça gözlenmektedir. Bu çalışmanın amacı, çoklu doğrusal regresyon ve hiyerarşik doğrusal modellerde 1) tek makul değer kullanımının, 2) tüm makul değerlerin kullanımının, 3) ağırlık kullanma durumunun parametre kestirimlerine etkisini araştırmaktır. Çalışmada PISA 2015 uygulamasının Türkiye verilerinden yararlanılmıştır. Araştırmada, örnekleme ağırlıkları kullanma durumunun ve makul değerlerin kullanım şeklinin kat sayıların, standart hataların ve açıklanan varyans oranının kestirilmesinde önemli rolleri olduğu belirlenmiştir. Bulgular detaylı bir biçimde tartışılmış ve uygulama ve gelecek araştırmalar için bazı öneriler sunulmuştur.

In large-scale assessments like Programme for International Students Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS), plausible values are often used as students’ ability estimations. In those studies, stratified sampling method is employed in order to draw participants, and hence, the data gathered has a hierarchical structure. In the context of large-scale assessments, plausible values refer to randomly drawn values from posterior ability distribution. It is reported that using one of plausible values or mean of those values as independent variable in linear models may lead to some estimation errors. Moreover, it is observed that sampling weights sometimes are not used during analysis of large-scale assessment data. This study aims to investigate the influence of three approaches on the parameters of linear and hierarchical linear regression models: 1) using only one plausible value, 2) using all plausible values, 3) incorporating sampling weights or not. Data used in the present study is obtained from school and student questionnaires in PISA (2015) Turkey database. Results revealed that the use of sampling weights and number of plausible values has significant effects on regression coefficients, standard errors and explained variance for both regression models. Findings of the study were discussed in details and some conclusions were drawn for practice and further research.

  • Adams, R.J., & Wu, M.L. (Eds.) (2002) PISA 2000 technical report. Paris: OECD Publications.
  • Beaton, A.E. (1987). Implementing the new design. The NAEP 1983-84 technical report. (Report No. 15-TR-20). Princeton, NJ: Educational Testing Service.
  • Bock, R.D. & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of the EM algorithm. Psychometrika 46, 443-459.
  • Bryk, A. S., & Raudenbush, S. W. (1988). Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical linear model. American Journal of Education 97(1), 65-108.
  • Bryk, A. S., & Raudenbush, S. W. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Sage Publications, Thousand Oaks, CA.
  • Bryk, A. S., Raudenbush, S. W., & Congdon, R. (2010). HLM7 for Windows [Computer software]. Chicago, IL: Scientific Software International, Inc.
  • Chowa, G. A., Masa, R. D., Ramos, Y., & Ansong, D. (2015). How do student and school characteristics influence youth academic achievement in Ghana? A hierarchical linear modelling of Ghana YouthSave baseline data. International Journal of Educational Development, 45, 129-140.
  • Cochran, W. G. (1977). Sampling techniques (3rd ed.). John Wiley and Sons, New York, NY.
  • Gelman, A. (2006). Multilevel (hierarchical) modelling: What it can and cannot do. Technometrics 48(3), 432-435.Goldstein, H. (2011). Multilevel statistical models (Vol. 922). John Wiley & Sons.
  • IBM Corp. Released 2015. IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.IEA (2016) Help Manual for the IDB Analyzer. Hamburg, Germany. (Available fromwww.iea.nl/data)
  • Kreft, I. G., De Leeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30(1), 1-21.
  • Lohr, S. (2010). Sampling: Design and Analysis (2nd edition). Brooks/Cole, Boston, MA.
  • Maas, C. J. M., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1(3), 86-92. doi:10.1027/1614-2241.1.3.86
  • Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56(2), 177–196.
  • Mislevy, R. J. (1993). Should “multiple imputations” be treated as “multiple indicators”? Psychometrika, 58(1), 79–85.
  • OECD (2009). Analyses with plausible values. In PISA Data Analysis Manual: SPSS, Second Edition, OECD Publishing. http://dx.doi.org/10.1787/9789264056275-9-en
  • OECD (2017). PISA 2015 technical report. Paris: OECD.
  • Osborne, J. W. (2000). Advantages of hierarchical linear modeling. Practical Assessment, Research & Evaluation, 7(1), 1-3.
  • Rasch, G. (1960). Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Oxford, England: Nielsen & Lydiche.
  • Raudenbush, S. W. (1988). Educational applications of hierarchical linear models: A review. Journal of Educational Statistics, 13(2), 85-116.
  • Raudenbush, S. W., & Bryk, A. S. (1986). A hierarchical model for studying school effects. Sociology of Education, 1-17.
  • Roberts, J. K. (2004). An introductory primer on multilevel and hierarchical linear modeling. Learning Disabilities: A Contemporary Journal 2, 30-38.
  • Rubin, D.B. (1987). Multiple imputations for non-response in surveys. New York: Wiley.
  • Särndal, C., B. Swensson, & J. Wretman (1992). Model assisted survey sampling. Springer-Verlag, New York, NY.
  • Snijders, T., & Bosker, R. (2003). Multilevel analysis: An introduction to basic and applied multilevel analysis. Sage Publications, Thousand Oaks, CA.
  • Stipek, D., & Valentino, R. A. (2015). Early childhood memory and attention as predictors of academic growth trajectories. Journal of Educational Psychology, 107(3), 771.
  • Von Davier, M., Gonzalez, E., & Mislevy, R. (2009). What are plausible values and why are they useful. IERI monograph series 2, 9-36.
  • Warm, T. A. (1985). Weighted maximum likelihood estimation of ability in item response theory with tests q/jinite length. Technical Report CGI-TR-85-08. Oklahoma City: U.S. Coast Guard Institute.
  • Woltman, H., Feldstain, A., MacKay, J. C., & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52-69.
  • Wright. B.D., & Stone, M. H. (1979). Best test design. Chicago: MESA Press.
  • Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2), 114-128.
Birincil Dil en
Konular Sosyal
Yayımlanma Tarihi Sonbahar
Bölüm Makaleler
Yazarlar

Orcid: 0000-0003-2950-9647
Yazar: Osman TAT (Sorumlu Yazar)
Kurum: Hacettepe University
Ülke: Turkey


Orcid: 0000-0002-0009-5279
Yazar: İlhan KOYUNCU
Kurum: Hacettepe University
Ülke: Turkey


Orcid: 0000-0001-5181-7262
Yazar: Selahattin GELBAL
Kurum: Hacettepe University
Ülke: Turkey


Tarihler

Yayımlanma Tarihi : 4 Eylül 2019

Bibtex @araştırma makalesi { epod486999, journal = {Journal of Measurement and Evaluation in Education and Psychology}, issn = {1309-6575}, eissn = {1309-6575}, address = {}, publisher = {Eğitimde ve Psikolojide Ölçme ve Değerlendirme Derneği}, year = {2019}, volume = {10}, pages = {235 - 248}, doi = {10.21031/epod.486999}, title = {The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters}, key = {cite}, author = {TAT, Osman and KOYUNCU, İlhan and GELBAL, Selahattin} }
APA TAT, O , KOYUNCU, İ , GELBAL, S . (2019). The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters. Journal of Measurement and Evaluation in Education and Psychology , 10 (3) , 235-248 . DOI: 10.21031/epod.486999
MLA TAT, O , KOYUNCU, İ , GELBAL, S . "The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters". Journal of Measurement and Evaluation in Education and Psychology 10 (2019 ): 235-248 <https://dergipark.org.tr/tr/pub/epod/issue/48297/486999>
Chicago TAT, O , KOYUNCU, İ , GELBAL, S . "The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters". Journal of Measurement and Evaluation in Education and Psychology 10 (2019 ): 235-248
RIS TY - JOUR T1 - The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters AU - Osman TAT , İlhan KOYUNCU , Selahattin GELBAL Y1 - 2019 PY - 2019 N1 - doi: 10.21031/epod.486999 DO - 10.21031/epod.486999 T2 - Journal of Measurement and Evaluation in Education and Psychology JF - Journal JO - JOR SP - 235 EP - 248 VL - 10 IS - 3 SN - 1309-6575-1309-6575 M3 - doi: 10.21031/epod.486999 UR - https://doi.org/10.21031/epod.486999 Y2 - 2019 ER -
EndNote %0 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters %A Osman TAT , İlhan KOYUNCU , Selahattin GELBAL %T The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters %D 2019 %J Journal of Measurement and Evaluation in Education and Psychology %P 1309-6575-1309-6575 %V 10 %N 3 %R doi: 10.21031/epod.486999 %U 10.21031/epod.486999
ISNAD TAT, Osman , KOYUNCU, İlhan , GELBAL, Selahattin . "The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters". Journal of Measurement and Evaluation in Education and Psychology 10 / 3 (Eylül 2019): 235-248 . https://doi.org/10.21031/epod.486999
AMA TAT O , KOYUNCU İ , GELBAL S . The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters. Journal of Measurement and Evaluation in Education and Psychology. 2019; 10(3): 235-248.
Vancouver TAT O , KOYUNCU İ , GELBAL S . The Influence of Using Plausible Values and Survey Weights on Multiple Regression and Hierarchical Linear Model Parameters. Journal of Measurement and Evaluation in Education and Psychology. 2019; 10(3): 248-235.