Examination of a computer-delivered English proficiency test for flight attendants: Test-takers’ experiences

Abstract

International Civil Aviation Organization recognizes English proficiency tests for the aviation personnel. Despite this, no universal test has yet to be developed for this purpose. There are only local tests available on the market. Among them is a computer-delivered English proficiency test which is administered for a globally recognized flight company in Turkey. This test is implemented for flight attendants and has important consequences such as suspending flights until the required level of English is achieved in the test. Thus, test-takers' perceptions of the test are significant as they experience these impacts first-hand. This study, therefore, explored flight attendants' test-taking experiences. 26 flight attendants participated in this study. Semi-structured interviews and two focus group sessions were held to collect data and evidence. The findings showed that the respondents had concerns for test administration including the issues such as physical conditions of the setting, timing of the test, and test format. Although they agreed that the test had a representative sampling of the language, they thought it lacked relevant components in accordance with their needs and level. They also reported the test content did not reflect work-related context. The study, therefore, provided implications and suggestions to improve the test considering the aviation context.

Keywords

• Language testing
• Civil aviation
• Computer-delivered tests

References

1. Aiguo, W. (2007). Teaching aviation English in the Chinese context: Developing ESP theory in a non-English speaking country. English for Specific Purposes, 26, 121-128. https://doi.org/10.1016/j.esp.2005.09.003
2. Alderson, J. C. (2010). A survey of aviation English tests. Language Testing, 27(1), 51-72. https://doi.org/10.1177/0265532209347196
3. Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
4. Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
5. Birebirİngilizce (2018). DLA-Kabin ekibi dil sınavı. [DLA-Language test for flight attendants]. https://www.birebiringilizce.com.tr/dla-kabin-ekibi-dil-sinavi.html
6. Canning-Wilson, C. (2001). Choosing EFL/ESL visual assessments: Image and picture selection on foreign and second language exams. National Center for Research on Teacher Learning. (ERIC Document Reproduction Service No. ED452707).
7. Cobb, T., & Laufer, B. (2021). The nuclear word family list: A list of the most frequent family members, including base and affixed words. Language Learning, 71(3), 834-871. https://doi.org/10.1111/lang.12452
8. Cornwall, T. B., & Srilapung, V. (2013). Senior flight attendants’ English communication needs: A case study of Thai Airways International. US-China Foreign Language, 11(4), 286-291.

9. Çelebi, D. (2018). DLA teknikleri ve İngilizce bilgisi [DLA strategies and English]. Miran Yayıncılık.
10. Çelebi, D. (n.d). DLA nedir? [What is DLA]. https://thyingilizce.com/
11. Dibakanaka, A., & Hiranburana, K. (2012). Developing an e-learning competency-based English course module for chief flight attendants. International Journal of Scientific and Research Publications, 2(8), 313-326.
12. Farris, C. (2016). Aviation language testing. In D. Estival, C. Farris, & B. Molesworth (Eds.), Aviation English: A lingua franca for pilots and air traffic controllers (pp.75-91). Routledge.
13. Fulcher, G., & Davidson, F. (2007). Language testing and assessment: An advanced resource book. Routledge. Galloway, N., & Numajiri, T. (2019). Global Englishes language teaching:Bottom-up curriculum implementation. TESOL Quarterly, 54(1), 118-145. https://doi.org/10.1002/tesq.547
14. Green, A. (2014). Exploring language assessment and testing: Language in action. Routledge.
15. Hughes, A. (2003). Testing for language teachers. Cambridge University Press.
16. International Civil Aviation Community (ICAO). (2009). Language testing criteria for global harmonization.
17. https://www.ealts.com/documents/ICAO%20Cir%20318-AN180%20Global%20Harmonisation%20Testing%20Criteria%202008%20.pdf
18. Karimi, M., & Biria, R. (2017). Impact of risk taking strategies on male and female EFL learners’ test performance: The case of multiple choice questions. Theory and Practice in Language Studies, 7(10), 892-899. http://dx.doi.org/10.17507/tpls.0710.10
19. Kim, H., Baydar, N., Greek, A. (2003). Testing conditions influence the race gap in cognition and achievement estimated by household survey data. Applied Developmental Psychology, 23, 567-582. https://doi.org/10.1016/S0193-3973(02)00142-9
20. Kim, H., & Elder, C. (2009). Understanding aviation English as a lingua franca perceptions of Korean aviation personnel. ARAL, 32(3), 1-17. https://doi.org/0.2104/aral0923
21. Kitzinger, J. (1995). Qualitative research: Introducing focus groups. BMJ, 311(7000), 299–302. https://doi.org/10.1136/bmj.311.7000.299
22. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310 Lewkowicz, J. A. (2000). Authenticity in language testing: Some outstanding questions. Language Testing, 17(1), 43-64.
23. McGrath, C., Palmgren, P. J., & Liljedahl, M. (2019). Twelve tips for conducting qualitative research interviews. Medical Teacher, 41(9), 1002-1006. https://doi.org/10.1080/0142159X.2018.1497149
24. Mullis, I. V. S., Bohrnstedt, G. W., Preuschoff, A. C., Reyes, I.D.L., Stancavage, F., & Martin, M. O. (2012). Examining NAEP achievement in relation to school testing conditions in the 2010 assessments. National Center for Education Statistics. https://www.air.org/sites/default/files/2021-06/NVS_Testing_Conditions_Paper_Final_0.pdf
25. Pocius, K. E. (1991). Personality factors in human-computer interaction: A review of the literature. Computers in Human Behavior, 7, 103-135.
26. Saldana, J. (2013). The coding manual for qualitative researchers. SAGE Publications.
27. Schult, C. A., & McInthosh, J. L. (2004). Employing computer-administered exams in general psychology: Student anxiety and expectations. Teaching of Psychology, 31(3), 209-211. https://doi.org/10.1207/s15328023top3103_7
28. Schuler, J. (2018). Looking at and beyond the lexical surface in L2 reading comprehension: Insights from a video-based study. Language Awareness, 27(1-2), 113-135. https://doi.org/10.1080/09658416.2018.1435672
29. Shohamy, E., Or, L. G., & May, S. (2017). Language testing and assessment. Springer.
30. Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test Impact revisited: Washback effect over time. Language Testing, 13(3), 298-317. https://doi.org/10.1177/026553229601300305
31. Shu’ Aibu, M. G. (2021). The effect of testing conditions on the students’ performance of disatance learning system (DLS) in Jigawa State. International Journal of Contemporary Education Research, 20(8), 71-80. i Stowell, J. R., & Bennett, D. (2010). Effects of online testing on student exam performance and test anxiety. Educational Computing Research, 42(2), 161-171. https://doi.org/10.2190/EC.42.2.b
32. Taylor, C., Jamieson, J., Eignor, D., & Kirsch, I. (1998). The relationship between computer familiarity and performance on computer-based TOEFL test tasks. Educational Testing Service.
33. Yakut, A. D., & Aydın, S. (2014). An experimental study on the effects of the use of blogs on EFL reading comprehension. Innovation in Language Learning and Teaching, 1(11), https://doi.org/10.1080/17501229.2015.1006634
34. Wigglesworth, G., & Frost, K. (2017). Task and performance-based assessment. In E. Shohamy, L. G. Or, & S. May (Eds.), Language testing and assessment (pp.121-133). Springer.