COH-METRIX: INTRODUCTION AND VALIDATION OF AN ONLINE TOOL FOR TEXT ANALYSIS

Abstract

This is a corpus-based study which tries to compare lexical networks of Turkish EFL learners with native speakers of English; it also tries to verify the validity of an online database (coh-metrix), which is used for lexical computations of texts written in English. The lexical computations are performed through 60 indices such as syntax, general word and text formation and referential and semantic aspects. Three different corpora (one learner, two native) were employed to test the tool’s ability to differentiate learner texts from native ones. The learner text sets were written by 49 intermediate Turkish students learning English, and the other two text sets were written by two different native groups of speakers of English (100 native speakers in total). A number of statistical testing techniques including Kruskal Wallis and Mann Whitney U-test were used. Some of the indices yielded statistically significant differences between three groups; some of them were able to exclude the learner text sets from the native text sets regardless of average number of words in the texts and the prompts for the essays. The results showed that sentences in the essays written by the Turkish EFL learners lacked lexical cohesion when compared to the sentences produced by the native groups.

Keywords

• Lexical networks, Coh-metrix, Corpus

References

1. Allen, D. (1992). Oxford placement test 2 (New edition). Oxford: Oxford University Press.
2. Brown, D. H. (2007). Principles of language learning and teaching. New York: Longman.
3. Connor, U. (1984). A Study of cohesion and coherence in English as a second language students’ writing. Papers in Linguistics 17 (1-4), 301-316.
4. Cook, G. (1989). Discourse. Oxford: Oxford University Press.
5. Crossley, S. A., Salsbury, T. McCarthy, P. M. & McNamara, D. S. (2008). Using Latent Semantic Analysis to explore second language lexical development. In D. Wilson &
6. G. Sutcliffe (Eds.), Proceedings of the 21st International Florida Artificial Intelligence Research Society (pp. 136-141). Menlo Park, California: AAAI Press.
7. Crossley, S. A. & McNamara, D. S. (2009). Computationally assessing lexical differences in second language writing. Journal of Second Language Writing, 17 (2), 119-135.
8. Ellis, R. (1995). Modified oral input and the acquisition of word meanings. Applied Linguistics, 16, 409–435.

9. Ellis, R. & Barkhuizen G. (2005). Analysing Learner Language (pp. 1-15). Oxford: Oxford University Press.
10. Ferris, D.R. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly 28, 414-420.
11. Field, A. (2009). Discovering Statistics Using SPSS. London: Sage Publications.
12. Flowerdew, J. (2009). Use of signalling nouns in a learner corpus. In Mahlberg & Flowerdew. Lexical Cohesion and Corpus Linguistics. Amsterdam: John Benjamins.
13. Frase, L., Faletti, J., Ginther, A., & Grant, L. (1999). Computer analysis of the TOEFL Test of Written English (Research report 64). Princeton, NJ: Educational Testing Service.
14. Gass, S., Selinker, L. (2008). Second Language Acquisition: An Introductory Course. New York, NY: Routledge.
15. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers, 36, 193–202.
16. Granger, S. (1993). The International Corpus of Learner English. In Aarts J., de Haan P. and Oostdijk N. (Eds.) English Language Corpora: Design, Analysis and Exploitation. (pp. 57-69). Amsterdam: Rodopi.
17. Granger, S. (2003). The international corpus of learner English: a new resource for foreign language learning and teaching and second language acquisition research. TESOL Quarterly 37(3), 538-545.
18. Hinkel, E. (2002). Second Language Writers' Text. Mawah, NJ: Lawrence Erlbaum Associates.
19. Landauer, T. K., Foltz, P. W., & Laham, D. (1998). Introduction to latent semantic analysis. Discourse Processes, 25, 259-284.
20. Meara, P. (2002). The discovery of vocabulary. Second Language Research, 18(4), 393-407.
21. Pica, T. (2005). Second Language Acquisition Research and Applied Linguistics. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp.263-280).
22. Mahwah, N.J: Lawrence Erlbaum Associates. Silva, T. (1993). Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications. TESOL Quarterly, 27(4), 657-77.