Testing your Tests: Reliability Issues of Academic English Exams

Abstract

A testing unit or a tester when writing exam questions generally have millions of issues in mind such as, the reliability, validity, clarity of instructions, design, layout, organization, quality, and many more. However, testing units preparing tests for EAP English for Academic Purposes has some other concerns about their exams such as, appropriate topics for listening and reading texts, authenticity, etc. Test writers also have to consider the focus on language skills development, the full coverage of skills, effective learning tasks and the variation in the activity types for all faculties. The main purpose of the study in hand is to investigate the quality of the tests written for the freshman students of Izmir University of Economics. One of the other main purposes of the study is to evaluate the EAP tests written for this institution according to Olaofe’s 1994 eight criterion for good EAP tests.

Keywords

• Testing,Assessment and Evaluation,EAP tests,reliability

References

1. Dudley-Evans, T., & John, M. St.(1998). Developments in English for specific purposes. Cambridge:Cambridge University Press.
2. Dunworth, A. (2008). A Task-Based Analysis of Undergraduate Assessment: A Tool for the EAP Practitioner. TESOL Quarterly.42/2: 315-323.
3. Fulcher, G. (1999). Assessment in English for Academic Purposes: Putting content validity in its place. Applied Linguistics. 20/2: 221-236.
4. Hughes, A. (1988). Achievement and proficiency: the missing link? In A. Hughes (Ed.) Testing English for
5. University Study, ELT Documents 127: 36-42. Lord, F.M. (1952). The Relationship of the Reliability of Multiple-Choice Test to the Distribution of Item
6. Difficulties, Psychometrika, 18, 181-194. Olaofe, I. (1994). Testing English for academic purposes (EAP) in higher education. Assessment & Evaluation in Higher Education. 19/1.
7. Osterlind, S. J. (1997). Constructing Test Items: Multiple-Choice, Constructed-Response, Performance and Other
8. Formats (2nd Ed.). Kluwer Academic Publishers. Tucker, S. (2007). Using Remark Statistics for Test Reliability and Item Analysis. Available at: http://www.umaryland.edu/cits/testscoring/pdf/umbtestscoring_testanditemanalysis.pdf

9. Watering, G. & Rijt. J. (2006). Teachers’ and students’ perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items. Educational Research