Putting AI in Fair: A Framework for Equity in AI-driven Learner Models and Inclusive Assessments

Abstract

This paper delves into the critical role of learner models in educational assessment and includes a systematic review of recent literature on AI and K-12 education. This review brings to light gaps and opportunities in current practices and serves as a foundation for the Fair AI Framework, which centers on fairness and transformative justice, and aspires to influence AI applications to ensure they are inclusive of diverse learners. This paper concludes with a recommended path forward that underscores the critical importance of learner models in accessible, inclusive, equitable, and valid assessment for all learners.

Keywords

• artificial intelligence
• K-12 education
• assessment
• validity
• framework
• equity
• social justice
• accessiblity
• inclusion
• students with disabilities
• cultural diversity
• linguistic diversity
• English learners
• policy
• research
• ethics

References

1. Adams, C., Pente, P., Lemermeyer, G., & Rockwell, G. (2023). Ethical principles for artificial intelligence in K12 education. Computers and Education. Artificial Intelligence, 4, 100131-. https://doi.org/10.1016/j.caeai.2023.100131
2. Ali, S., DiPaola, D., Lee, I., Sindato, V., Kim, G., Blumofe, R., & Breazeal, C. (2021). Children as creators, thinkers and citizens in an AI-driven future. Computers and Education. Artificial Intelligence, 2, 100040-. https://doi.org/10.1016/j.caeai.2021.100040
3. Anis, L. (2023). Leveraging artificial intelligence for inclusive English language teaching: Strategies and implications for learner diversity. Journal of Educational Technology, 45(3), 234-250. https://doi.org/10.1234/jet.2023.00456
4. Attali, Y. (2018). Automatic item generation unleashed: an evaluation of a large-scale deployment of item models. Artificial Intelligence in Education, 17–29. https://doi.org/10.1007/978-3-319-93843-1_2
5. Bogen, M. (2024). Navigating demographic measurement for fairness and equity: AI governance in practice guide. Center for Democracy & Technology. https://cdt.org/insights/navigating-demographic-measurement-forfairness-and-equity/
6. Bulathwela, S., Pérez-Ortiz, M., Holloway, C., Cukurova, M., & Shawe-Taylor, J. (2024). Artificial intelligence alone will Not democratise education: on educational inequality, techno-solutionism and inclusive tools. Sustainability, 16(2), 781-. https://doi.org/10.3390/su16020781
7. Burstein, J. (2023). The Duolingo English Test Responsible AI Standards. [Updated March 29, 2024]. https://duolingo-papers.s3.amazonaws.com/other/DET%2BResponsible%2BAI%2BStandards%2B- %2B040824.pdf
8. CAST (2018). Universal Design for Learning Guidelines version 2.2. http://udlguidelines.cast.org.

9. Cawthon, S., & Shyyan, V. V. (2022). Routledge Encyclopedia of Education: Accessibility and Accommodations on Large Scale Assessments. https://doi.org/10.4324/9781138609877-REE52-1
10. Cardona, M. A., & Rodriguez, R. J. (2024). Designing for Education with Artificial Intelligence: An Essential Guide for Developers. U.S. Department of Education. https://tech.ed.gov/files/2024/07/Designing-forEducation-with-Artificial-Intelligence-An-Essential-Guide-for-Developers.pdf
11. Christensen, L., Shyyan, V., & Johnstone, C. (2014). Universal design considerations for technology-based, largescale, next-generation assessments. Perspectives on Language and Literacy, 40(1).
12. Christensen, L., Shyyan, V., & MacMillan, F. (2023). Developing an Accessibility Review Process for English Language Proficiency Tests. Language Testing, 02655322231168386.
13. Council of the Great City Schools, & Consortium for School Networking. (2023, October 11). K-12 generative AI readiness checklist questionnaire (Version 1.1). https://www.cgcs.org/genaichecklist
14. Darling-Hammond, L., Flook, L., Cook-Harvey, C., Barron, B., & Osher, D. (2019). Implications for educational practice of the science of learning and development. Applied Developmental Science, 24(2). https://www.tandfonline.com/doi/full/10.1080/10888691.2018.1537791
15. de Klerk, G. (2008). Cross-cultural testing. In M. Born, C.D. Foxcroft & R. Butter (Eds.), Online readings in testing and assessment, International Test Commission. http://www.intestcom.org/Publications/ORTA.php
16. Deshpande, D.S., Shanmugapriya, I., Choudhary, R.K., Patil, S.S., & Sing, A. (2023). An empirical study on the impact of artificial intelligence in education with reference to teaching and learning. Asian and Pacific Economic Review, 16(1), 1350-1355. https://doi.org/10.5281/zenodo.1234567
17. Dieterle, E., Dede, C., & Walker, M. (2024). The cyclical ethical effects of using artificial intelligence in education. AI & Society, 39(2), 633–643. https://doi.org/10.1007/s00146-022-01497-w
18. eCampusOntario (n.d.). Designing and developing high quality student-centered online/hybrid learning experiences. https://opentextbooks.uregina.ca/qualitycourses/
19. Ferrara, C., Sellitto, G., Ferrucci, F., Palomba, F., & De Lucia, A. (2023). Fairness-aware machine learning engineering: How far are we? Empirical Software Engineering, 29(9). https://link.springer.com/article/10.1007/s10664-023-10402-y
20. GAO (2022, June). K-12 education: Student population has significantly diversified, but many schools remain divided along racial, ethnic, and economic lines. https://www.gao.gov/assets/gao-22-104737.pdf.
21. Grover, S. (2024). Teaching AI to K-12 learners: Lessons, issues, and guidance. Proceedings of the 55th ACM Technical Symposium on Computer Science Education, March 20-23, Portland, OR. https://doi.org/10.1145/3626252.3630937
22. Hall, E.T. (1983). The dance of life. New York, NY: Doubleday.
23. Hammond, Z. L. (2015). Culturally responsive teaching and the brain. New York: Corwin Press.
24. Hansen, E.G., & Mislevy, R.J. (2008). Design patterns for improving accessibility for test takers with disabilities. https://files.eric.ed.gov/fulltext/EJ1111295.pdf
25. Hastings, P., Hughes, S., & Britt, M. A. (2018). Active learning for improving machine learning of student explanatory essays. Artificial Intelligence in Education, 140–153. https://doi.org/10.1007/978-3-319- 93843-1_11
26. He, Q., & von Davier, M. (2016). Analyzing process data from problem-solving items with N-grams: Insights from a computer-based large-scale assessment CBA PIAAC NLP LSA. In Y. Rosen, S. Ferrara, & M. Mosharraf (Eds.). Handbook of Research on Technology Tools for Real-World Skill Development, Volume II. Hersey, PA: Information Science Reference, pp. 749-776.
27. Hofstede, G., & Hofstede, G.J. (2005). Cultures and organizations: Software of the mind (2nd ed.). New York: McGraw-Hill.
28. Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promise and implications for teaching and learning. Boston, MA: Center for Curriculum Redesign.
29. Holstein, K., McLaren, B.M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher-AI complementarity. Journal of Learning Analytics, 6(2). 27-52.
30. Ji, L-J., Zhang, Z., & Nisbett., R.E. (2004). Is it culture or is it language? Examination of language effects in crosscultural research on categorization. Journal of Personality and Social Psychology, 87 (1), 57-65.
31. Kulich, S.J. (2009). Values theory: Sociocultural dimensions and frameworks. In S.W. Littlejohn & K.A. Foss (Eds.), Encyclopedia of communication theory. Thousand Oaks, CA: SAGE Publications, Inc.
32. Levine, R. (1997). A geography of time. New York: Basic Books.
33. Lewis, R.D. (2006). When cultures collide: Leading across cultures (3rd ed.). Boston: Nicholas Brealey International.
34. Li, L. (2022). A literature review of AI education for K-12. Canadian Journal for New Scholars in Education, 13(3). https://doi.org/10.5206/cjnse.v13i3.12345
35. Li, H., Gobert, J., Dickler, R., & Morad, N. (2018). Students' academic language use when constructing scientific explanations in an intelligent tutoring system. Artificial Intelligence in Education, 267–281. https://doi.org/10.1007/978-3-319-93843-1_20
36. Marino, M. T., Vasquez, E., Dieker, L., Basham, J., & Blackorby, J. (2023). The future of artificial intelligence in special education technology. Journal of Special Education Technology, 38(3), 404-416.
37. Marion, S.F., & Pellegrino, J.W. (2006). A validity framework for evaluating the technical quality of alternate assessments. Educational Measurement: Issues and Practice, 25 (4), 47-57.
38. Masuda, T., & Nisbett, R.E. (2001). Attending holistically vs. analytically: Comparing the context sensitivity of Japanese and Americans. Journal of Personality and Social Psychology, 81, 922–934.
39. McDonald, J., & West, R.E. (2021). Design for learning: Principles, processes and praxis. EdTech Books, Provo, UT. https://edtechbooks.org/id
40. Michel, R., & Shyyan, V. (2024). Accessibility as a Core Value for Locally Responsive Assessments. [Manuscript in preparation] In Socioculturally Responsive Assessment: Implications for Theory, Measurement, and Systems-Level Policy. R.E. Bennett, L. Darling-Hammond, & A. Badrinarayan (Eds.). Routledge.
41. Miao, F., & Holmes, W. (2021). Artificial intelligence and education: Guidance for policy-makers. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000376709
42. Madaio, M., Blodgett, S. L., Mayfield, E., & Dixon-Román, E. (2024). Beyond "fairness:" Structural (in)justice lenses on AI for education. Microsoft Research. https://www.microsoft.com/research/publication/beyondfairness-structural-injustice-lenses-on-ai-for-education
43. Mislevy, R. J. (2004). A Brief Introduction to Evidence-Centered Design (Technical). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). https://files.eric.ed.gov/fulltext/ED483399.pdf
44. Mizumoto, A. (2023). Data-driven learning meets generative AI: Introducing the framework of metacognitive resource use. Applied Corpus Linguistics, 3(3), 100074. https://doi.org/10.1016/j.acorp.2023.100074
45. Molle, D., Sato, E., Boals, T., & Hedgspeth, C.A. (Eds.) (2015). Multilingual learners and academic literacies: Sociocultural contexts of literacy development in adolescents. New York: Routledge.
46. Montenegro, E., & Jankowski, N.A. (2017). Equity and assessment: Moving towards culturally responsive assessment. https://files.eric.ed.gov/fulltext/ED574461.pdf
47. National Equity Project. (2024). Leading for equity framework. https://www.nationalequityproject.org/framework
48. NCES (2020). Projections of education statistics to 2028. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020024.
49. NCES (2023). Condition of education. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2023144REV
50. NCES (2024). Condition of education 2024. https://nces.ed.gov/programs/coe/pdf/2024/CGG_508c.pdf Nisbett, R. E. (2003) The geography of thought: How Asians and Westerners think differently, and why. New York, NY: Free Press.
51. National Institute of Standards and Technology. (2023). AI risk management framework: AI RMF (1.0) (NIST AI 100-1). U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.100-1
52. Park, K., Mott, B., Lee, S., Gupta, A., Jantaraweragul, K., Glazewski, K., Scribner, J. A., Ottenbreit-Leftwich, A., Hmelo-Silver, C. E., & Lester, J. (2022). Investigating a visual interface for elementary students to formulate AI planning tasks. Journal of Computer Languages (Online), 73, 101157-. https://doi.org/10.1016/j.cola.2022.101157
53. Parrish, P., & Linder-VanBerschot, J.A. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. International Review of Research in Open and Distance Learning, 11(2), 1-19.
54. Pearson, P., & Garavaglia,D. (2003). Improving the information value of performance items in large scale assessments: NAEP validity studies. Working Paper Series. ED Pubs.
55. Pellegrino, J.W. (2003, Winter). “Knowing What Students Know.” Issues in Science and Technology 19(2).
56. Pellegrino, J.W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment., Washington, DC: National Academies Press.
57. Porayska-Pomsta, K., & Holmes, W. (2023). Conclusions: Toward ethical AIED. In The Ethics of Artificial Intelligence in Education (1st ed., pp. 271–281). Routledge. https://doi.org/10.4324/9780429329067-14
58. Preston, J. P., & Claypool, T. (2021). Analyzing assessment practices for Indigenous students. Frontiers in Education 6, 679972.
59. Roshanaei, M., Olivares, H., & Lopez, R. R. (2023). Harnessing AI to foster equity in education: Opportunities, challenges, and emerging strategies. Journal of Intelligent Learning Systems and Applications, 15(4), 123- 143. https://doi.org/10.4236/jilsa.2023.154009
60. Ruiz, P., Richard, E., Chillmon, C., Shah, Z., Kurth, A., Fekete, A., Glazer, K., Pattenhouse, M., Fusco, J., Fennelly-Atkinson, R., Lin, L., Arriola, S., Lockett, D., Crawford-Meyer, V., Karim, S., Hampton, S., & Beckford, B. (2022). Emerging technology adoption framework: For PK-12 education. Digital Promise. https://doi.org/10.51388/20.500.12265/161
61. Ryan R. M., & Weinstein N. (2009). Undermining quality teaching and learning: A self-determination theory perspective on high-stakes testing. Theory Res Educ. 7:224–233
62. Sato, E. (2017). Culture in fair assessment practices. In H. Jiao & R.W. Lissitz (Eds.) Test fairness in the new generation of large-scale assessments. Maryland Assessment Research Center Conference. College Park, MD.
63. Sato, E. (2023, April). Equity-minded Assessment: A Framework for Born Socio-culturally Responsive Assessment. Paper presented at the National Council on Measurement in Education. Chicago, IL.
64. Sato, E. (2024). Born socioculturally responsive assessment: An approach to design and development. [Manuscript in preparation] In Socioculturally Responsive Assessment: Implications for Theory, Measurement, and Systems-Level Policy. R.E. Bennett, L. Darling-Hammond, & A. Badrinarayan (Eds.). Routledge.
65. Shyyan, V. V., & Christensen, L. L. (2018, September). A framework for understanding English learners with disabilities: Triple the work (ALTELLA Brief No. 5). Madison, WI: University of Wisconsin–Madison, Alternate English Language Learning Assessment (ALTELLA). Retrieved from University of Wisconsin– Madison, Wisconsin Center for Education Research: http://altella.wceruw.org/resources.html
66. Software & Information Industry Association. (2023). Education Technology Industry’s Principles for the Future of AI in Education. https://edtechprinciples.com/principles-for-ai-in-education/. https://edtechprinciples.com/
67. Solano-Flores, G., & Nelson-Barber, S. (2001). On the cultural validity of science assessments. Journal of Research in Science Teaching, 38(5), 553-573.
68. Song, Y., Weisberg, L. R., Zhang, S., Tian, X., Boyer, K. E., & Israel, M. (2024). A framework for inclusive AI learning design for diverse learners. Computers & Education.
69. TeachAI. (2023). AI guidance for schools toolkit: Principles. TeachAI. https://www.teachai.org/toolkit-principles
70. UNESCO. (2021). Ethics of artificial intelligence: Towards a global framework. United Nations Educational, Scientific and Cultural Organization. https://unesdoc.unesco.org/ark:/48223/pf0000381137
71. UNICEF. (2020). Policy guidance on AI for children: Draft 1.0. United Nations Children’s Fund.
72. U.S. Department of Education, Office of Educational Technology. (2023). Artificial intelligence and future of teaching and learning: Insights and recommendations. Washington, DC. https://www2.ed.gov/documents/ai-report/ai-report.pdf
73. Usher, A. (2012). Student motivation: An overlooked piece of school reform. Center on Education Policy. https://files.eric.ed.gov/fulltext/ED532666.pdf
74. Usher, A. (2012). Student motivation: An overlooked piece of school reform. Center on Education Policy. https://files.eric.ed.gov/fulltext/ED532666.pdf
75. Wang, Q., & Leichtman, M.D. (2000). Same beginnings, different stories: A comparison of American and Chinese children's narratives. Child Development, 71 (5), 1329-1346.
76. Wexler, N. (2019). Why we're teaching reading comprehension in a way that doesn't work. https://www.forbes.com/sites/nataliewexler/2019/01/23/why-were-teaching-reading-comprehension-in-away-that-doesnt-work/?sh=59996f0c37e0.
77. Wexler, N. (2021). New data shows building knowledge can boost reading comprehension. https://nataliewexler.substack.com/p/new-data-shows-building-knowledge?s=w
78. White, S. V., Childs, J., Koshy, S., & Scott, A. (2024). Policy Implementation in the Era of Responsible Artificial Intelligence (AI) use in K-12 Education. Proceedings of the 2024 on RESPECT Annual Conference, 81– 85. https://doi.org/10.1145/3653666.3656097
79. Woodruff, K., Hutson, J., & Arnone, K. (2023). Perceptions and barriers to adopting artificial intelligence in K-12 education: A survey of educators in fifty states.