Drawing a Sample with Desired Properties from Population in R Package “drawsample”

Year 2020, Volume 11, Issue 4, 405 - 429, 30.12.2020

Kübra ATALAY KABASAKAL Tuba GÜNDÜZ

https://doi.org/10.21031/epod.790449

Abstract

The aim of this study is to develop an R package called drawsample, which will be used to draw samples with the desired properties from a real data set. In accordance with the aim of the study, a sample with the desired properties can be drawn by purposive sampling with determining several conditions, such as deviation from normality (skewness and kurtosis) and sample size. Different applications of the package drawsample are illustrated using real data from the “Science and Technology(Score_1)” and “Social Studies (Score_2)” subtests of 6th Grade Public Boarding and Scholarship Examinations (PBSE). As the importance given to research with real data has increased in recent years, a good approach would be to draw a sample of the population. With this package, it is expected that researchers will draw samples as close as possible to the desired properties from the population or a large sample. It is thought that using the drawn samples obtained from real data with package drawsample will provide an alternative to simulation studies as well as a complement for these studies.

Keywords

R package “drawsample", distribution, real data, simulation

References

• Abdel-fattah, A.-F. A. (1994, April). Comparing BILOG and LOGIST estimates for normal, truncated normal and beta ability distributions. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
• American Educational Research Association. (2020). Journal of Educational and Behavioral Statistics. Retrieved from https://journals.sagepub.com/description/jeb
• Bahry L. M. (2012). Polytomous item response theory parameter recovery: An investigation of non-normal distributions and small sample size (Unpublished Master’s disertation). University of Alberta, Edmonton, Canada.
• Bıkmaz Bilgen, Ö., & Doğan, N. (2017). Çok kategorili parametrik ve parametrik olmayan madde tepki kuramı modellerinin karşılaştırılması [Comparison of Polytomous Parametric and Nonparametric Item Response Theory Models]. Journal of Measurement and Evaluation in Education and Psychology, 8(4), 354-372. DOI: 10.21031/epod.346650
• Blanca, M. J., Arnau, J., López-Montiel, D., Bono, R., & Bendayan, R. (2013). Skewness and kurtosis in real data samples. Methodology,9(2), 78–84. DOI: 10.1027/1614-2241/a000057
• Blanca, M., Alarcón, R., Arnau, J., Bono, R., & Bendayan, R. (2017). Non-normal data: Is ANOVA still a valid option?. Psicothema, 29(4), 552-557. DOI: 10.7334/psicothema2016.383
• Blest, D. C. (2003). A new measure of kurtosis adjusted for skewness. Australian & New Zealand Journal of Statistics, 45(2), 175-179.
• Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
• Büyüköztürk, Ş., Çokluk, Ö., & Köklü, N. (2014). Sosyal Bilimler için istatistik. Ankara: Pegem Akademi.
• Çelikten, S., & Çakan, M. (2019). Bayesian ve nonbayesian kestirim yöntemlerine dayali olarak siniflama indekslerinin TIMSS 2015 matematik testi üzerinde incelenmesi. [Investigation of classification indices on TIMSS 2015 mathematic-subtest through Bayesian and nonbayesian estimation methods]. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi, 13(1), 105-124.
• Courville, T. G. (2004). An empirical comparison of item response theory and classical test theory item/person statistics (Doctoral dissertation, Texas A&M University).
• Custer, M., Omar, M. H., & Pomplun, M. (2006). Vertical scaling with the Rasch model utilizing default and tight convergence settings with WINSTEPS and BILOG-MG. Applied Measurement in Education, 19(2), 133-149. DOI: 10.1207/s15324818ame1902_4
• D'agostino, R. B., Belanger, A., & D'Agostino Jr, R. B. (1990). A suggestion for using powerful and informative tests of normality. The American Statistician, 44(4), 316-321. DOI: 10.2307/2684359
• Doğan, N. & Tezbaşaran, A. A. (2003). Klasik test kuramı ve örtük özellikler kuramının örneklemler bağlamında karşılaştırılması. [Comparison of classical test theory and latent traits theory by samples]. Hacettepe University Journal of Education, 25, 58–67. DOI: 10.17860/efd.86348
• Doğan, N., & Kılıç, A. F. (2018). The Effects of Sample Size, Correlation Technique, and Factor Extraction Method on Reliability Coefficients. Kastamonu Eğitim Dergisi, 26(3), 697-706. DOI: 10.24106/kefdergi.413303
• Dolma, S. (2009). Çok ihtimalli rasch modeli ile derecelendirilmiş yanıt modelinin örtük özellikleri tahminleme performansı açısından simülasyon yöntemiyle karşılaştırılması [A simulation study for the comparison of the polytomous Rasch model and graded response model according to their performance on recovering the latent traits]. (Unpublished Doctoral dissertation). İstanbul Üniversitesi Sosyal Bilimler Enstitüsü, İstanbul, Turkey.
• Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern robust statistical methods: An easy way to maximize the accuracy and power of your research. American Psychologist, 63(7), 591–601. doi: 10.1037/0003-066X.63.7.591
• Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58, 357-381. doi: 10.1177/0013164498058003001
• Fialkowski, A. C. (2018). SimMultiCorrData: Simulation of Correlated Data with Multiple. Retrieved from: https://cran.r-project.org/web/packages/SimMultiCorrData/index.html
• Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In Hancock, G.R. & Mueller R. O. (Eds.), Structural equation modeling: A second course, (pp. 269-314). Information Age Publishing, U.S.A.
• Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521-532. doi: 10.1007/BF02293811
• Flott, L. W. (1995). Quality control: Measurement error. Metal Finishing, 93(9), 72-75.
• Geary, R. C. (1947). Testing for normality. Biometrika, 34(3/4), 209-242. DOI: 10.1093/biomet/34.3-4.209
• Gelbal, S. (1994). P madde güçlük indeksi ile Rasch modelinin b parametresi ve bunlara dayalı yetenek ölçüleri üzerine bir karşılaştırma [A comparison of item difficulty index P and Rasch model b parameters and their ability measures based on them].Doctoral disertation, Hacettepe University, Ankara. Retrieved from
• Gotzmann, A. J. (2011). Comparison of vertical scaling methods in the context of NCLB. (Doctoral dissertation, University of Alberta, Alberta). Retrieved from https://era.library.ualberta.ca/items/04a8d59c-791d-435b-bde6-7a6de3012169
• Hallgren, K. A. (2013). Conducting simulation studies in the R programming environment. Tutorials in Quantitative Methods for Psychology, 9(2), 43–60. DOI:10.20982/tqmp.09.2.p043
• Han, K. T. (2007). WinGen: Windows software thatgenerates IRT parameters and item responses. Applied Psychological Measurement, 31(5), 457-459. doi: 10.1177/0146621607299271
• Han, K. T., & Hambleton, R. K. (2007). User's Manual: WinGen (Center for Educational Assessment Report No. 642). Amherst, MA: University of Massachusetts, School of Education.
• Headrick, T. C. 2002. Fast fifth-order polynomial transforms for generating univariate and multivariate non-normal distributions. Computational Statistics & Data Analysis. 40(1),685–711. doi: 10.1016/S0167-9473(02)00072-5
• Huck, S. W. (2012). Reading statistics and research (6th ed). Boston: Pearson.
• John Wiley & Sons, Inc.-a. (2019). Educational Measurement: Issues and Practice. Retrieved from https://onlinelibrary.wiley.com/page/journal/17453992/homepage/productinformation.html
• Kaya, Y., Leite, W. L., & Miller, M. D. (2015). A comparison of logistic regression models for DIF detection in polytomous items: The effect of small sample sizes and non-normality of ability distributions. International Journal of Assessment Tools in Education, 2(1), 22-39. doi: 10.21449/ijate.239563
• Kieftenbeld, V., & Natesan, P. (2012). Recovery of graded response model parameters: A comparison of marginal maximum likelihood and Markov chain Monte Carlo estimation. Applied Psychological Measurement, 36(5), 399-419. DOI: 10.1177/0146621612446170
• Kim, H. Y. (2013). Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restorative dentistry & endodontics, 38(1), 52-54.
• Kirisci, L., Hsu, T. C., & Yu, L. (2001). Robustness of item parameter estimation programs to assumptions of unidimensionality and normality. Applied Psychological Measurement, 25(2), 146-162. doi: 10.1177/01466210122031975
• Kline, R. B. (2005). Principles and practice of structural equations modeling. New York: Guilford.
• Kogar, H . (2018). Effects of Various Simulation Conditions on Latent-Trait Estimates: A Simulation Study. International Journal of Assessment Tools in Education , 5 (2) , 263-273. DOI: 10.21449/ijate.377138
• Kolen, M. J. (1985). Standard errors of Tucker Equating. Applied Psychological Measurement, 9(2), 209-223, doi: 10.1177/014662168500900209.
• Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(1), 156–166. DOI: 10.1037/0033-2909.105.1.156
• Ministry of National Education. (2013). Parasız Yatılılık Ve Bursluluk Sınavı (PYBS) Sınav Kılavuzu [Guide of Public Boarding And Scholarship Examination (PBSE)]. Retrieved from http://www.meb.gov.tr/sinavlar/dokumanlar/2013/kilavuz/2013_PYBS_2.pdf
• Ministry of National Education. (2020). Ortaöğretim Kurumlarına İlişkin Merkezi Sınav Kılavuzu [Guide for Central Examination Secondary Education Institution]. Retrieved from: http://www.meb.gov.tr/meb_iys_dosyalar/2020_07/17104126_2020_Ortaogretim_Kurumlarina_Iliskin_Merkezi_Sinav.pdf
• Nartgün, Z. (2002). Aynı tutumu ölçmeye yönelik likert tipi ölçek ile metrik ölçeğin madde ve ölçek özelliklerinin klasik test kuramı ve örtük özellikler kuramına göre incelenmesi [ Examination of item and scale properties of likert type scale and metric scale to measure the same attitude according to classical test theory and item response theory]. (Unpublished doctoral dissertation). Hacettepe University Social Sciences Institute, Ankara.
• Olivier, J., & Norberg, M. M. (2010). Positively skewed data: revisiting the box-cox power transformation. International Journal of Psychological Research, 3(1), 68-95. DOI: 10.21500/20112084.846
• Pearson, E.S. (1932). The analysis of variance in cases of non-normal variation. Biometrika. 23, 114-133.
• Pomplun, M., Omar, M. H., & Custer, M. (2004). A comparison of WINSTEPS and BILOG-MG for vertical scaling with the Rasch model. Educational and Psychological Measurement, 64(4), 600-616. doi: 10.1177/0013164403261761
• R Core Team. (2014). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. Retrieved September 10, 2019, from http://www.R-project.org/
• Ramos, C., Costa, P. A., Rudnicki, T., Marôco, A. L., Leal, I., Guimarães, R., ... & Tedeschi, R. G. (2018). The effectiveness of a group intervention to facilitate posttraumatic growth among women with breast cancer. Psycho‐oncology, 27(1), 258-264. DOI:10.1002/pon.4501
• Revelle, W. (2018) psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA, https://CRAN.R-project.org/package=psych Version = 1.8.12.
• Reyhanlıoğlu Keceoğlu, Ç. (2018). Parametrik ve Parametrik Olmayan Madde Tepki Kuramında Farklı Örneklem Büyüklüklerine ve Boyutluluklarına Göre Parametre Değişmezliğinin İncelenmesi. Unpublished doctoral dissertation). Hacettepe University Social Sciences Institute, Ankara.
• Şahin, M. G., & Yıldırım, Y. (2018). The examination of item difficulty distribution, test length and sample size in different ability distribution. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 9(3), 277-294. DOI: 10.21031/epod.385000
• Sarkar, D. (2008). lattice: Multivariate Data Visualization with R. Springer-Verlag, New York.
• SAS Institute Inc. (2009). SAS/Stat User‘s Guide, version 9.2, (Version 9.2). Cary, NC.
• Sen, S., Cohen, A. S., & Kim, S. H. (2014, November). Robustness of mixture IRT models to violations of latent normality. In Quantitative Psychology Research: The 78th Annual Meeting of the Psychometric Society (Vol. 89, p. 27). Springer.
• Seong, T. J. (1990). Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions. Applied Psychological Measurement, 14(3), 299-311. DOI: 10.1177/014662169001400307
• Sireci, S. G. (1991). "Sample-Independent" Item Parameters? An Investigation of the Stability of IRT Item Parameters Estimated from Small Data Sets. Paper presented at the annual Conference of Northeastern Educational Research Association, New York, NY.
• SSCP (2019). 2019 YKS Değerlendirme Raporu [2019 Examinations of the Council of Higher Education Assessment Report]. Retrieved from https://dokuman.osym.gov.tr/pdfdokuman/2019/GENEL/yksDegRaporweb03092019.pdf
• Stone, C. A. (1992). Recovery of marginal maximum likelihood estimates in the two-parameter logistic response model: An evaluation of MULTILOG. Applied Psychological Measurement, 16(1), 1-16. doi: 10.1177/014662169201600101
• Urry, V. W. (1974). Approximations to item parameters of mental test models and their uses. Educational and Psychological Measurement, 34, 253-269. doi: 10.1177/001316447403400206
• Uysal, İ. (2014). Comparison of irt test equating methods for mixed format tests. [Madde tepki kuramına dayalı test eşitleme yöntemlerinin karma modeller üzerinde karşılaştırılması]. (Master disertation, Bolu Abant Izzet Baysal University, Bolu). Retrieved from https://tez.yok.gov.tr/UlusalTezMerkezi/ Variable Types. R package version 0.2.2.
• West, S. G., Finch, J. F., & Curran, P. J. (1995). Structural equation models with non-normal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (p. 56–75). Newbery Park, CA: Sage
• Wickham, H., François, R., Henry, L., & Müller, K. (2016) tibble: Simple data frames. Retrieved from https://CRAN.Rproject.org/package=tibble. R package version 3.0.3
• Wickham, H., François, R., Henry, L., & Müller, K. (2019). dplyr: A Grammar of data manipulation. R package version 0.8.0.1. https://CRAN.R-project.org/package=dplyr
• Wicklin, R. (2013). Simulating data with SAS. SAS Institute.
• Wuensch, K. L. (2005). Kurtosis. Encyclopedia of Statistics in Behavioral Science. doi:10.1002/0470013192.bsa334
• Yıldırım, H., Uysal-Saraç, M., & Büyüköztürk, Ş. (2018). Farklı örneklem büyüklüğü ve dağılımı Koşullarında WLS ve Robust WLS yöntemlerinin karşılaştırılması. Ilkogretim Online, 17(1), 431-439. doi: 10.17051/ilkonline.2018.413794
• Yıldırım, Y. (2015). Derecelendirilmiş tepki modeli temelli parametre kestiriminde normallik sayıltısı ihlalinin ölçme kesinliğine etkisi [The effect of normality violation in the process of parameter estimation based upon Graded Response Model on measurement precision]. (Master disertation, Gazi University, Ankara). Retrieved from https://tez.yok.gov.tr/UlusalTezMerkezi/
• Yoes, M. E. (1993). A comparison of the effectiveness of item parameter estimation techniques used with the three-parameter logistic item response theory model. (Volumes I and II). Unpublished Ph.D., University of Minnesota, Minneapolis/St. Paul, MN.