Missing Data Management Practices in L2 Research: The Good, The Bad and The Ugly

Abstract

Missing data are one of the frequently encountered problems in quantitative research. When neglected or handled improperly, this problem can have adverse impact on research results. However, the issue of missing data in quantitative second language (L2) research has largely been ignored when compared to the other sister disciplines such as education and psychology. The purpose of this methodological synthesis was, therefore, to investigate the issue of missing data in L2 research, with a particular focus on L2 researchers’ current missing data management practices. A total of 143 studies published in six leading L2 journals were reviewed in this synthesis. The results indicated that missing data were indeed quite common in L2 research in that 41% of the studies indicated evidence of missing data, but L2 researchers’ management and reporting of missing data was often less than optimal. In light of the results, several directed suggestions were made to improve the rigor and quality of L2 research.

Keywords

• Missing data,quantitative research methods,statistical literacy,L2

İkinci Dil Araştırmalarında Kayıp Veri Yönetim Uygulamaları: İyi, Kötü ve Çirkin

Öz

Kayıp veriler nicel araştırmalarda sıklıkla karşılaşılan sorunlardan biridir. İhmal edildiğinde ya da yanlış şekilde ele alındığında, kayıp veriler araştırma sonuçları üzerinde olumsuz etki yaratabilir. Ancak, eğitim ve psikoloji gibi diğer yakın alanlarla karşılaştırıldığında, ikinci dil araştırmalarında kayıp verilerin durumu göz ardı edilmiştir. Bu nedenle, bu metodolojik sentezin amacı ikinci dil araştırmalarındaki mevcut kayıp veri yönetim uygulamalarını araştırmaktır. Bu sentezde ikinci dil araştırmaları dergisinde yayınlanan toplam 143 çalışma ele alındı. Sonuçlar, ikinci dil araştırmalarında kayıp verilerin gerçekten oldukça yaygın olduğunu gösterdi. İncelenem çalışmaların %41’inde kayıp veri bulgusuna rastlanmıştır. Ancak, ikinci dil araştırmacılarının kayıp veri yönetimi ve sunumu genel olarak çok yetersiz. Sonuçların ışığında, ikinci dil araştırmalarının kalitesini artırmak için çözüm odaklı bazı öneriler sunulmuştur.

Anahtar Kelimeler

• Kayıp veri,nicel araştırma yöntemleri,istatistiksel okur-yazarlık,ikinci dil

References

1. Author (2016)
2. Author (2018)
3. Author & co-author (2015).
4. Author & co-authors (2017).
5. Author & co-authors (forthcoming).
6. Bennett, D. A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25(5), 464-469.
7. Cheema, J. R. (2014). A review of missing data handling methods in education research. Review of Educational Research, 84(4), 487-508.
8. Cohen, J., & Cohen, P. (1983). Applied multiple regression/ correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

9. Denies, K., Yashima, T., & Janssen, R. (2015). Classroom versus societal willingness to communicate: Investigating French as a second language in Flanders. The Modern Language Journal, 99(4), 718-739.
10. Derrick, D. J. (2016). Instrument reporting practices in second language research. TESOL Quarterly, 50(1), 132-153.
11. Dong, Y., & Peng, C. Y. J. (2013). Principled missing data methods for researchers. SpringerPlus, 2(1), 1-17.
12. Enders, C. (2010). Applied missing data analysis. New York, NY: Guilford Press.
13. Goodwin, A. P., August, D., & Calderon, M. (2015). Reading in multiple orthographies: Differences and similarities in reading in Spanish and English for English Learners. Language Learning, 65(3), 596-630.
14. Karanja, E., Zaveri, J., & Ahmed, A. (2013). How do MIS researchers handle missing data in survey-based research: A content analysis approach. International Journal of Information Management, 33(5), 734-751.
15. Khajavy, G. H., Ghonsooly, B., Hosseini Fatemi, A., & Choi, C. W. (2016). Willingness to communicate in English: A microsystem model in the Iranian EFL classroom context. TESOL Quarterly, 50(1), 154-180.
16. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
17. Loewen, S., Lavolette, E., Spino, L. A., Papi, M., Schmidtke, J., Sterling, S., & Wolff, D. (2014). Statistical literacy among applied linguists and second language acquisition researchers. TESOL Quarterly, 48, 360–388.
18. Mahboob, A., Paltridge, B., Phakiti, A., Wagner, E., Starfield, S., Burns, A., Jones, R.H. & De Costa, P. I. (2016). TESOL Quarterly research guidelines. TESOL Quarterly, 50(1), 42-65.
19. Marsden, E., Thompson, S., & Plonsky, L. (2018). A methodological synthesis of self-paced reading in second language research. Applied Psycholinguistics. doi:10.1017/S0142716418000036
20. McKnight, P. E., McKnight, K. M., Sidani, S., & Figueredo, A. J. (2007). Missing data: A gentle introduction. New York, NY: Guilford Press.
21. Newman, D. A. (2014). Missing data: Five practical guidelines. Organizational Research Methods, 17(4), 372-411.
22. Norris, J. M., Ross, S. J., & Schoonen, R. (2015). Improving second language quantitative research. Language Learning, 65(S1), 1-8.
23. Peng, C., Harwell, M., Liou, S., & Ehman, L. (2006). Advances in missing data methods and implications for educational research. In S. S. Sawilowsky (Ed.), Real data analysis (pp. 31–78). Charlotte, NC: New Information Age.
24. Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of educational research, 74(4), 525-556.
25. Plonsky, L. (2011). Study quality in SLA: A cumulative and developmental assessment of designs, analyses, reporting practices, and outcomes in quantitative L2 research (Unpublished doctoral dissertation). Michigan State University, East Lansing, MI.
26. Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655-687.
27. Plonsky, L. (2014). Study quality in quantitative L2 research (1990–2010): A methodological synthesis and call for reform. The Modern Language Journal, 98(1), 450-470.
28. Plonsky, L. (2015). Advancing quantitative methods in second language research. New York, NY: Routledge.
29. Pichette, F., Béland, S., Jolani, S., & Lesniewska, J. (2015). The handling of missing binary data in language research. Studies in Second Language Learning and Teaching, 5(1), 153–169.Roth, P. L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47(3), 537-560.
30. Osborne, J. W. (2013). Best practices in data cleaning: A complete guide to everything you need to do before and after collecting your data. Thousand Oak, CA: Sage.
31. Rousseau, M., Simon, M., Bertrand, R., & Hachey, K. (2012). Reporting missing data: a study of selected articles published from 2003–2007. Quality & Quantity, 46(5), 1393-1406.
32. Schafer, J. L. (1999). Multiple imputation: a primer. Statistical Methods in Medical Research, 8(1), 3-15.
33. Schafer, J. L., & Graham, J. W. (2002). Missing data: our view of the state of the art. Psychological Methods, 7(2), 147-177.
34. Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling psychology, 57(1), 1-10.
35. Tabachnick B., & Fidell, L. (2013). Using multivariate statistics. Boston: Pearson Education Inc.
36. Teng, L. S., & Zhang, L. J. (2016). A questionnaire‐based validation of multidimensional models of self‐regulated learning strategies. The Modern Language Journal, 100(3), 674-701.
37. Wilkinson, L. & Task Force on Statistical Inference, (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594-604.
38. Winke, P. (2014). Testing hypotheses about language learning using structural equation modeling. Annual Review of Applied Linguistics, 34, 102-122.