Using Videos in ESL Listening Achievement Tests: Effects on Difficulty

Abstract

The question as to whether an assessment construct of second language (L2) listening comprehension

should include the decoding of visual information remains unanswered (see Buck, 2001; Ockey, 2007).

This study aimed to fill this gap by investigating how audio-only and video-enhanced delivery formats of

listening passages compared in terms of difficulty for English as a second language (ESL) students. It

utilized students’ performance on listening achievement tests developed by the researcher. The

participants were 60 low- and high-proficiency ESL students enrolled in an American intensive English

program. The participants’ scores on the achievement tests were used to compare the difficulty of items of

different formats and determine whether this difficulty related to video type (context versus content) and

students’ proficiency level. The findings suggested that, at least for higher-level students, listening

testlets enhanced with videos containing mostly content-related visuals were significantly easier than

their audio-only counterparts were. On the contrary, the inclusion of videos with mostly context visuals

did not affect the difficulty of testlets in any proficiency category. The findings are discussed in terms of

their practical significance for ESL teachers as well as theoretical implications for the field of ESLlistening assessment.

Keywords

• Audio; second language; listening test; video; content visuals

References

1. Baltova, I. (1994). The impact of video on the comprehension skills of core French students. Canadian Modern Language Review, 50, 507–531.
2. Batty, A. O. (2015). A comparison of video- and audio-mediated listening tests with many-facet Rasch modeling and differential distractor functioning. Language Testing, 32, 3-20. http://doi.org/b4s9
3. Bejar, I., Douglas, D., Jamieson, J., Nissan, S., & Turner, J. (2000). TOEFL 2000 listening framework: A working paper (TOEFL Monograph Series Report No. 19). Retrieved from https://www.ets.org/Media/Research/pdf/RM-00-07.pdf
4. Buck, G. (2001). Assessing listening. Cambridge, UK: Cambridge University Press.
5. Chan, C., Wang, L., & Xu, L. (2014). A study of video effect of English listening comprehension. Studies in Literature and Language, 8, 53-58. http://doi.org/b4tb
6. Copyright Law of the United States and Related Laws Contained in title 17 of the United States Code. (2011). Retrieved from http://www.copyright.gov/title17/circ92.pdf
7. Cubilo, J., & Winke, P. (2013). Redefining the L2 listening construct within an integrated writing task: Considering the impacts of visual-cue interpretation and note-taking. Language Assessment Quarterly, 10, 371–397. http://doi.org/b4tc
8. Ginther, A. (2002). Context and content visuals and performance on listening comprehension stimuli. Language Testing, 19, 133–167. http://doi.org/b9kst8

9. Gruba, P. (1993). A comparison study of audio and video in language testing. JALT Journal, 15, 85–88.
10. Gruba, P. (2004). Understanding digitized second language videotext. Computer Assisted Language Learning, 17, 51-82. http://doi.org/d8jx2b
11. Kintsch, W. (1998). Comprehension. Cambridge: Cambridge University Press.
12. Lado, R. (1961). Language Testing: The construction and use of foreign language tests. London: Longman.
13. Latifi, M., Tavakoli, M., & A’lipour, J. (2013). Investigating the effect of video materials on testing foreign language learners’ listening performance. Middle-East Journal of Scientific Research, 13, 1197-1201. Retrieved from https://www.idosi.org/mejsr/mejsr13(9)13/11.pdf
14. Londe, Z. C. (2009). The effects of video media in English as a second language listening comprehension test. Issues in Applied Linguistics, 17, 41-50. Retrieved from http://escholarship.org/uc/item/0c080191#page-1
15. Mendelsohn, D. J. (1994). Learning to listen: A strategy-based approach for the second language learner. SanDiego: Dominie Press.
16. Ockey, G. J. (2007). Construct implications of including still image or video in computer-based listening tests. Language Testing, 24, 517–537. http://doi.org/dk58d9
17. Shin, D. (1998). Using videotaped lectures for testing academic listening proficiency. International Journal of Listening, 12, 57–80. http://doi.org/b4td
18. Stallings, W. M. (1972). A Comparison of television and audio presentations of the MLA French Listening Examination. The Journal of Educational Research, 65, 472-474.
19. Sueyoshi, A., & Hardison, D. M. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661-699. http://doi.org/fv5v5k
20. Suvorov, R. (2009). Context visuals in L2 listening tests: The effects of photographs and video vs. audio-only format. In C. A. Chapelle, H. G. Jun, & I. Katz (Eds.), Developing and evaluating language learning materials (pp. 53-68). Ames, IA: Iowa State University.
21. Suvorov, R. (2015a). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 21, 1-21. http://doi.org/b4tf
22. Suvorov, R. (2015b). Interacting with visuals in L2 listening tests: An eye-tracking study. In V. Berry (Ed.), ARAGs research reports online (Report #AR-A/2015/1). Retrieved from http://www.britishcouncil.org/sites/default/files/interacting_with_visuals_in_l2_listening_tests_suvorov.pdf
23. Underwood, M. (1989). Teaching listening. New York: Longman.
24. Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning & Technology, 11, 67–86. Retrieved from http://llt.msu.edu/vol11num1/pdf/wagner.pdf
25. Wagner, E. (2010a). Test-takers’ interaction with an L2 video listening test. System, 38, 280-291. http://doi.org/d6s5kd
26. Wagner, E. (2010b). The effect of the use of video texts on ESL listening test-taker performance. Language Testing, 27, 493-513. http://doi.org/cjg7cg
27. Wagner, E. (2013). An investigation of how the channel of input and access to test questions affect L2 listening test performance. Language Assessment Quarterly, 10, 178-195. http://doi.org/b4tg
28. Witkin, B. R., & Trochim, W. W. K. (1997). Toward a synthesis of listening constructs: A concept map analysis. International Journal of Listening, 11, 69-87. http://doi.org/fvmrjx
29. Wolvin, A., & Coakley, C. G. (1996). Listening (5th ed.). Dubuque, IA: Brown & Benchmark.