Examining the use of bilingual accommodations in digital math assessments: User perceptions

Year 2024, Volume: 11 Issue: 4, 787 - 803, 15.11.2024

Alexis A. Lopez

https://doi.org/10.21449/ijate.1529210

Abstract

Digital math assessments with bilingual accommodations allow multilingual learners to use their entire linguistic repertoire to showcase their knowledge and skills. The bilingual accommodations, which include tools like language translation and audio prompts in both English and Spanish, are designed to be adaptable, giving multilingual learners the freedom to view or listen to the items in either language and to write or say their responses in either language or a combination of both. This study examined how 56 middle school emergent bilingual learners used these bilingual accommodations and explored the perceptions of teachers and students regarding these accommodations. This study provides evidence regarding using bilingual accommodation in math assessments for middle school emergent multilingual learners. The results showed how students used their full linguistic repertoire and language modalities to showcase their math knowledge and skills. Both teachers and students reported having positive perceptions of the bilingual accommodations, reinforcing its responsiveness to different learners’ needs and preferences.

Keywords

Bilingual accommodations, Digital math assessments, Multilingual learners, Linguistic repertoire, Language modalities

Supporting Institution

ETS

Project Number

03081

References

• Abedi, J. (2009). Computer testing as a form of accommodation for English language learners. Educational Assessment, 14(3 4), 195 211.https://doi.org/10.1080/08957347.2014.944310
• Abedi, J. (2014). The use of computer technology in designing appropriate test accommodations for English language learners. Applied Measurement in Education, 27(4), 261–272. https://doi.org/10.1080/08957347.2014.944310
• Abedi, J. (2021). Accommodations and universal design. In G. Fulcher & L. Harding (Eds.), The Routledge handbook of language testing (2nd ed.) (pp. 306–321). Routledge.
• Abedi, J., Hofstetter, C.H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1–28. https://doi.org/10.3102/00346543074001001
• Abedi, J., & Lord, C. (2001). The language factor in mathematics tests. Applied Measurement in Education, 14(3), 219–234. https://doi.org/10.1207/S15324818AME1403_2
• Adie, L.E., Willis, J., & Van der Kleij, F.M. (2018). Diverse perspectives on student agency in classroom assessment. Australian Educational Researcher, 45, 1 12. https://doi.org/10.1007/s13384-018-0262-2
• Ascenzi-Moreno, L. (2018). Translanguaging and responsive assessment adaptations: Emergent bilingual readers through the lens of possibility. Language Arts, 95(6), 355–369. https://doi.org/10.1007/s13384-018-0262-2
• Ascenzi-Moreno, L., García, O., & López, A.A. (2023). Latinx bilingual students’ translanguaging and assessment: A unitary approach. In S. Melo-Pfeifer & C. Ollivier (Eds.). Assessing plurilingual competence and plurilingual students: Theories, educative issues and empirical approaches (pp. 48–61). Routledge. (Routledge Research in Language Education) https://doi.org/10.4324/9781003177197
• Bartlett, H.J. (2021). Assessments and accommodations for English language learners: A literature review. The Nebraska Educator: A Student Led Journal, 60. https://doi.org/10.32873/unl.dc.ne025
• Canagarajah, S. (2013). Literacy as translingual practice between communities and classrooms. Routledge.
• Canagarajah, S. (2018). Translingual practice as spatial repertoires: Expanding the paradigm beyond structuralist orientations. Applied Linguistics, 39(1), 31 54. https://doi.org/10.32873/unl.dc.ne025
• Cenoz J, Gorter D. (Eds.). (2015). Multilingual education: Between language learning and translanguaging. Cambridge University Press. https://doi.org/10.1017/9781009024655
• Council of Chief State School Officers (CCSSO). (2010). Common Core State Standards for Mathematics. CCSSO. Retrieved from https://learning.ccsso.org/wp content/uploads/2022/11/Math_Standards1.pdf
• Donnelly, W.B., & Roe, C.J. (2010). Using sentence frames to develop academic vocabulary for English learners. The Reading Teacher, 64(2), 131 136. https://doi.org/10.1598/RT.64.2.5
• Emirbayer, M., & Mische, A. (1998). What is agency? American Journal of Sociology, 103, 962–1023. https://doi.org/10.1086/231294
• Francis, D., Rivera, M., Lesaux, N., Kieffer, M., & Rivera, H. (2006). Practical guidelines for the education of English language learners: Research-based recommendations for instruction and academic interventions. RMC Research Corporation, Center on Instruction.
• García, O. (2009). Bilingual education in the 21st century: A global perspective. Wiley-Blackwell.
• García, O., & Li, W. (2014). Translanguaging: Language, bilingualism and education. Palgrave Macmillan Pivot.
• Gezer, T., Flowers, C., & Lambert, R. (2023). Effectiveness of English learners computer-based testing accommodations: A meta-analysis. Journal of English Learner Education, 15(1). https://stars.library.ucf.edu/jele/vol15/iss1/2
• Goodrich, J.M., Koziol, N.A., & Yoon, H. (2021). Are translated mathematics items a valid accommodation for dual language learners? Evidence from ECLS-K. Early Childhood Research Quarterly, 57, 89–101. https://doi.org/10.1016/j.ecresq.2021.06.001
• Harmon, J.M., Fraga, L.M., Martin, E., & Wood, K.D. (2013). Revitalizing word walls for high school English learners: Conventional and digital opportunities for learning new words. Georgia Journal of Literacy, 36(1), 20–28. https://doi.org/10.56887/galiteracy.37
• Iliescu, D., & Greiff, S. (2022). Some thoughts and considerations on accommodations in testing. European Journal of Psychological Assessment, 38(4), 239 242. https://doi.org/10.1027/1015-5759/a000732
• Kieffer, M.J., Lesaux, N.K., Rivera, M., & Francis, D.J. (2009). Accommodations for English language learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168 1201. https://doi.org/10.3102/0034654309332490
• King, J., Brundiers, K., & Fischer, D. (2024). Student agency in a sustainability-oriented assessment process: exploring expansive learning in student-led rubric co-design. Assessment & Evaluation in Higher Education, 1 13. https://doi.org/10.1080/02602938.2024.2333031
• Kobrin, J. (2022). Evidence-centered design. In B.B. Frey (Ed.), The SAGE Encyclopedia of Research Design (2nd ed., Vol. 4) (pp. 514 516). SAGE Publications, Inc. https://doi.org/10.4135/9781071812082
• Koran, J., & Kopriva, R.J. (2017). Framing appropriate accommodations in terms of individual need: Examining the fit of four approaches to selecting accommodations of English language learners. Applied Measurement in Education, 30(2), 71 81. https://doi.org/10.1080/08957347.2016.1243539
• Kusters, A., Spotti, M., Swanwick, R., & Tapio, E. (2017). Beyond languages, beyond modalities: Transforming the study of semiotic repertoires. International Journal of Multilingualism, 14(3), 219–232. https://doi.org/10.1080/14790718.2017.1321651
• Li, H., & Suen, H.K. (2012). The effects of test accommodations for English language learners: A meta analysis. Applied Measurement in Education, 25(4), 327 346. https://doi.org/10.1080/14790718.2017.1321651
• Li, W. (2011). Moment analysis and translanguaging space: Discursive construction of identities by multilingual Chinese youth in Britain. Journal of Pragmatics, 43, 1222–1235. https://doi.org/10.1080/14790718.2017.1321651
• Liu, L. (2023). Accommodations for English language learners in large-scale assessments: Testing accommodations for English language learners. The Educational Review, 7(6), 688–692. http://dx.doi.org/10.26855/er.2023.06.006
• López, A.A. (2023). Examining how Spanish-speaking English language learners use their linguistic resources and language modes in a dual language mathematics assessment task. Journal of Latinos and Education, 22(1), 198 210. https://doi.org/10.1080/15348431.2020.1731693
• López, A.A., Guzman-Orth, D.A., & Turkan, S. (2015). How might a translanguaging approach in assessment make tests more valid and fair for emergent bilinguals? In G. Valdés, K. Menken & M. Castro, M. (Eds.), Common core, bilingual and English language learners: A resource for educators (pp. 266–267). Caslon.
• López, A.A., Guzman-Orth, D., & Turkan, S. (2019). Exploring the use of translanguaging to measure the mathematics knowledge of emergent bilingual students. Translation and Translanguaging in Multilingual Contexts, 5(2), 143 164. https://doi.org/10.1075/ttmc.00029.lop
• López, A.A., Turkan, S., & Guzman-Orth, D. (2017). Conceptualizing the use of translanguaging in initial content assessments for newly arrived emergent bilingual students. ETS Research Report, 17(7). https://doi.org/10.1002/ets2.12140
• Lyon, E.G. (2023). Reframing formative assessment for emergent bilinguals: Linguistically responsive assessing in science classrooms. Science Education, 107(1), 203–233. https://doi.org/10.1002/sce.21760
• Martin-Beltrán, M. (2014). “What do you want to say?” How adolescents use translanguaging to expand learning opportunities. International Multilingual Research Journal, 8(3), 208–230. https://doi.org/10.1080/19313152.2014.914372
• Martiniello, M. (2009). Linguistic complexity, schematic representations, and differential item functioning for English language learners in math tests. Educational Assessment, 14(3–4), 160–79. https://doi.org/10.1080/10627190903422906
• Mislevy, R.J., Almond, R.G., & Lukas, J.F. (2003). A brief introduction to evidence-centered design. ETS Research Report, 13(16). https://doi.org/10.1002/j.2333-8504.2003.tb01908.x
• Otheguy, R., García, O., & Reid, W. (2015). Clarifying translanguaging and deconstructing named languages: A perspective from linguistics. Applied Linguistics Review, 6(3), 281–307. https://doi.org/10.1515/applirev-2015-0014
• Otheguy, R., García, O., & Reid, W. (2019). A translanguaging view of the linguistic system of bilinguals. Applied Linguistics Review, 10(4), 625 651. https://doi.org/10.1515/applirev-2018-0020
• Paradis, J., Genesee, F., & Crago, M.B. (2010). Dual language development and disorders: A handbook on bilingualism and second language learning (2nd Ed.). Paul H. Brookes.
• Pennock-Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non-ELLs: A meta-analysis of experimental studies. Educational Measurement: Issues and Practices, 30(3), 10–28. https://doi.org/10.1111/j.1745-3992.2011.00207.x
• Rios, J.A., Ihlenfeldt, & Chavez, C. (2020). Are accommodations for English learners on state accountability assessments evidence-based? A multistudy systematic review and meta-analysis. Educational Measurement: Issues and Practices, 39(4), 65 75. https://doi.org/10.1111/emip.12337
• Rivera, C., & Stansfield, C.W. (2004). The effect of linguistic simplification of science test items on score comparability. Educational Assessment, 9(3 4), 79 105. https://doi.org/10.1080/10627197.2004.9652960
• Roohr, K.C. & Sireci, S.G. (2017). Evaluating computer-based test accommodations for English learners. Educational Assessment, 22(1), 35 53. https://doi.org/10.1080/10627197.2016.1271704
• Roschmann, S., Witmer, S.E., & Volker, M.A. (2021). Examining provision and sufficiency of testing accommodations for English learners. International Journal of Testing, 21(1), 32–55. https://doi.org/10.1080/15305058.2021.1884872
• Sanchez, S.V., Rodriguez, B.J., Soto-Huerta, M.E., Castro Villareal, F., Guerra, N.S., & Bustos Flores, B. (2013). A case for multidimensional bilingual assessment. Language Assessment Quarterly, 10(3), 160–177. https://doi.org/10.1080/15434303.2013.769544
• Sayer, P. (2013). Translanguaging, TexMex, and bilingual pedagogy: Emergent bilinguals learning through the vernacular. TESOL Quarterly, 47(1), 63 88. https://doi.org/10.1002/tesq.53
• Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418 429. https://doi.org/10.1111/j.1540-4781.2011.01210.x
• Smarter Balanced Assessment Consortium (2012). Translation accommodations framework for testing English language learners in Mathematics. Retrieved from https://portal.smarterbalanced.org/library/en/translation-accommodations-framework-for-testing-english-language-learners-in-mathematics.pdf
• Solano-Flores, G. (2008). Who is given tests in what language by whom, when, and where? The need for probabilistic views of language in the testing of English language learners. Educational Researcher, 37(4), 189–199. https://doi.org/10.3102/0013189X08319569
• Turkan, S., López, A.A., Lawless, R.R., & Tolentino, F. (2019). Using pictorial glossaries as an accommodation for English learners: An exploratory study. Educational Assessment, 24(3), 235–265. https://doi.org/10.1080/10627197.2019.1615371
• Velasco, P., & García, O. (2014). Translanguaging and the writing of bilingual learners. Bilingual Research Journal, 37(1), 6–23. https://doi.org/10.1080/15235882.2014.893270
• Walker, M.E., Olivera-Aguilar, M., Lehman, B., Laitusis, C., Guzman-Orth, D., & Gholson, M. (2023). Culturally responsive assessment: Provisional principles. ETS Research Report, 23(11). https://doi.org/10.1002/ets2.12374
• Wise, S.L., & DeMars, C.E. (2005). Examinee motivation in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10, 1 17. https://doi.org/10.1207/s15326977ea1001_1
• Wolf, M.K., Kim, J., & Kao, J. (2012). The effects of glossary and read-aloud accommodations on English language learners’ performance on a mathematics assessment. Applied Measurement in Education, 25(4), 347 374. https://doi.org/10.1080/08957347.2012.714693
• Wolf, M.K., Yoo, H., Guzman-Orth, D., & Abedi, J. (2021). Investigating the effects of test accommodations with process data for English learners in a mathematics assessment. Educational Assessment, 27(1), 27–45. https://doi.org/10.1080/10627197.2021.1982693
• Yang, X. (2024). Linguistically responsive formative assessment for emergent bilinguals: exploration of an elementary teacher’s practice in a math classroom. International Multilingual Research Journal, 1–24. https://doi.org/10.1080/19313152.2024.2339757