Can We Trust The Data Collected Through Survey? : Ideal Response Time and Delusion of Cronbach’s Alpha

Year 2017, Volume: 8 Issue: 4, 344 - 353, 28.12.2017

Volkan Doğan

https://doi.org/10.21031/epod.313692

Abstract

Much of the studies
in the social and behavioral sciences is based on the data collected through
survey. To obtain unbiased knowledge, survey respondents are required to answer
survey items/questions in a careful way. However, past research demonstrated
that research projects are under the threat of careless/inattentive
respondents. This paper provides some evidences regarding the rate of
careless/inattentive respondents within Turkish respondents. To this end, this
paper comprises of two survey data—154 Turkish undergraduate students
(face-to-face survey) and 148 online Turkish respondents (via Facebook)—to
provide some descriptive evidences. Furthermore, a new approach toward
calculating ideal response time was proposed in the current paper. Lastly, the
delusion of Cronbach’s Alpha was discussed.

Keywords

Survey data, data quality, careless respondents, ideal response time

Anket Aracılığıyla Toplanan Veriye Güvenebilir Miyiz? : İdeal Cevaplama Süresi ve Cronbach’s Alpha Yanılgısı

Abstract

Sosyal ve
davranışsal araştırma disiplinlerinde birçok araştırma anket verisine
dayanmaktadır. İdeal olarak anketi cevaplayıcıların dikkatli şekilde cevap
verdikleri varsayılmaktadır. Fakat yakın geçmişte araştırmaların sonuçları
dikkatsiz cevaplayıcı tehditinin düşünülenden daha büyük olduğunu
göstermektedir. Bu araştırma Türk kültürü kapsamında kesitsel olarak dikkatsiz
anket cevaplayıcılarının oranının ne düzeyde olduğuna dair keşifsel bulgular
sunmakta, ideal cevaplama süresi hesaplamaya yönelik yeni bir metodolojik yaklaşım
önermekte ve Cronbach’s Alpha yanılgısı kavramına değinmektedir. Bu amaçla, 154
Türk üniversite öğrencisinden sınıf ortamında ve 148 Türk yetişkinden çevrim
içi ortamda toplanan anket verisi üzerinden analizler gerçekleştirilmiştir.

Keywords

anket verisi, veri kalitesi, dikkatsiz cevaplayıcılar, ideal cevaplama süresi

References

• Anduiza, E., & Galais, C. (2017). Answering without reading: IMCs and strong satisficing in online surveys. International Journal of Public Opinion Research, basım aşamasında.
• Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and going to extremes: Black-white differences in response styles. Public Opinion Quarterly, 48(2), 491-509.
• Beach, D. A. (1989). Identifying the random responder. The Journal of psychology, 123(1), 101-103.
• Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from the workers? Making sure respondents pay attention on self‐administered survey. American Journal of Political Science, 58(3), 739-753.
• Curran, P. G. (2016). Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology, 66, 4-19.
• Doğan, V., Özkara, B. Y., & Doğan, M. (2016). Luxury Consumption Tendency: Conceptualization, Scale Development, and Validation. Paper presented at the 2016 annual meeting of the American Marketing Association, Atlanta, GA.
• Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59(2), 93-104.
• Hauser, D. J., & Schwarz, N. (2015). It’s a trap! Instructional Manipulation checks prompt systematic thinking on ‘tricky’ tasks. SAGE Open, 5(2), 1-6.
• Hayes, A.F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New Jersey: Guilford Press.
• Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & Deshon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99-114.
• Huang, J. L., Liu, M., & Bowling, N. A. (2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100(3), 299-311.
• Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological methods, 17(3), 437-455.
• Niessen, A. S. M., Meijer, R. R., & Tendeiro, J. N. (2016). Detecting careless respondents in web-based questionnaires: Which method to use?. Journal of Research in Personality, 63, 1-11.
• Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867-872.
• Woods, C. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology and Behavioral Assessment, 28(3), 186-191.