Can We Trust The Data Collected Through Survey? : Ideal Response Time and Delusion of Cronbach’s Alpha

Abstract

Much of the studies in the social and behavioral sciences is based on the data collected through survey. To obtain unbiased knowledge, survey respondents are required to answer survey items/questions in a careful way. However, past research demonstrated that research projects are under the threat of careless/inattentive respondents. This paper provides some evidences regarding the rate of careless/inattentive respondents within Turkish respondents. To this end, this paper comprises of two survey data—154 Turkish undergraduate students (face-to-face survey) and 148 online Turkish respondents (via Facebook)—to provide some descriptive evidences. Furthermore, a new approach toward calculating ideal response time was proposed in the current paper. Lastly, the delusion of Cronbach’s Alpha was discussed.

Keywords

• Survey data,data quality,careless respondents,ideal response time

Anket Aracılığıyla Toplanan Veriye Güvenebilir Miyiz? : İdeal Cevaplama Süresi ve Cronbach’s Alpha Yanılgısı

Abstract

Sosyal ve davranışsal araştırma disiplinlerinde birçok araştırma anket verisine dayanmaktadır. İdeal olarak anketi cevaplayıcıların dikkatli şekilde cevap verdikleri varsayılmaktadır. Fakat yakın geçmişte araştırmaların sonuçları dikkatsiz cevaplayıcı tehditinin düşünülenden daha büyük olduğunu göstermektedir. Bu araştırma Türk kültürü kapsamında kesitsel olarak dikkatsiz anket cevaplayıcılarının oranının ne düzeyde olduğuna dair keşifsel bulgular sunmakta, ideal cevaplama süresi hesaplamaya yönelik yeni bir metodolojik yaklaşım önermekte ve Cronbach’s Alpha yanılgısı kavramına değinmektedir. Bu amaçla, 154 Türk üniversite öğrencisinden sınıf ortamında ve 148 Türk yetişkinden çevrim içi ortamda toplanan anket verisi üzerinden analizler gerçekleştirilmiştir.

Keywords

• anket verisi,veri kalitesi,dikkatsiz cevaplayıcılar,ideal cevaplama süresi

References

1. Anduiza, E., & Galais, C. (2017). Answering without reading: IMCs and strong satisficing in online surveys. International Journal of Public Opinion Research, basım aşamasında.
2. Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48(2), 491-509.
3. Beach, D. A. (1989). Identifying the random responder. The Journal of psychology, 123(1), 101-103.
4. Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from the workers? Making sure respondents pay attention on self‐administered survey. American Journal of Political Science, 58(3), 739-753.
5. Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4-19.
6. Doğan, V., Özkara, B. Y., & Doğan, M. (2016). Luxury Consumption Tendency: Conceptualization, Scale Development, and Validation. Paper presented at the 2016 annual meeting of the American Marketing Association, Atlanta, GA.
7. Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59(2), 93-104.
8. Hauser, D. J., & Schwarz, N. (2015). It’s a trap! Instructional Manipulation checks prompt systematic thinking on ‘tricky’ tasks. SAGE Open, 5(2), 1-6.

9. Hayes, A.F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New Jersey: Guilford Press.
10. Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & Deshon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99-114.
11. Huang, J. L., Liu, M., & Bowling, N. A. (2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100(3), 299-311.
12. Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological methods, 17(3), 437-455.
13. Niessen, A. S. M., Meijer, R. R., & Tendeiro, J. N. (2016). Detecting careless respondents in web-based questionnaires: Which method to use?. Journal of Research in Personality, 63, 1-11.
14. Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867-872.
15. Woods, C. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28(3), 186-191.