Visual-Form Learning Analytics: A Tool for Critical Reflection and Feedback

Year 2019, , 214 - 228, 12.07.2019

Kelly Mckenna Beth Pouska Marcia C. Moraes James E. Folkestad

https://doi.org/10.30935/cet.589989

Abstract

Accessible learning analytics available from the data within learning management systems, can assist with teaching and learning practices, but often this data is difficult to interpret. Learning analytics, specifically those presented in visual-form, can provide information that supports learners’ reflection and guides them to the necessary changes that lead to successful self-regulated learning. This research study utilized photo-elicitation methods to prompt learners’ reflections of their self-regulated retrieval practice activities, quiz-based learning opportunities, which were qualitatively analyzed. A tool, U-Behavior, was created which was designed to extract students attempt data on the retrieval practice activities which were presented to students as opportunities to study the course content rather than as evaluations of understanding. Upon completion of the retrieval practice activities, learners were presented with their personalized learning analytics in visual-form and prompted to reflect on their learning. Visual-form learning analytics create opportunities for feedback and critical reflection for both instructors and learners and improve student learning. Analysis of the visual-form learning analytics and corresponding reflections highlighted learners’ understanding of high-impact learning practices, the realization of intended study behaviors versus engrained behaviors, high score orientation, and a focus on comparisons.

Keywords

Learning analytics, Visualizations, Reflection

References

• Aguilar, D. A. G., Therón, R., & Peñalvo, F. G. (2008). Understanding Educational Relationships in Moodle with ViMoodle. 2008 Eighth IEEE International Conference on Advanced Learning Technologies, 954-956. Santander, Cantabria.
• Alhadad, S. S. (2018). Visualizing data to support judgement, inference, and decision making in learning analytics: Insights from cognitive psychology and visualization science. Journal of Learning Analytics, 5(2), 60-85.
• Bakharia A. & Dawson S. (2011). SNAPP: a bird’s-eye view of temporal participant interaction. 1st International Conference on Learning Analytics and Knowledge, 168-173. Vancouver, British Columbia, Canada.
• Balter, O., Enstrom, E., & Klingenberg, B. (2013). The effect of short formative diagnostic web quizzes with minimal feedback. Computers & Education, 60(1), 234-242.
• Banks, M. (2007). Using visual data in qualitative research. Los Angeles: Sage.
• Beheshitha, S. S., Hatala, M., Gašević, D., Joksimović, S. (2016). The role of achievement goal orientations when studying effect of learning analytics visualizations. LAK ’16 Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, 54-63. Edinburgh, UK.
• Berrais, A. (2015). Using online Moodle quizzes to support the teaching of mathematics to foundation engineering students. QScience Proceedings: Engineering Leaders Conference 2014, 8.
• Brown, P. C., Roediger, H. L., & McDaniel. M. A. (2014). Make it stick: The science of successful learning. Cambridge, MA: Harvard University Press.
• Cohen, D. & Sasson, I. (2016). Online quizzes in a virtual learning environment as a tool for formative assessment. Journal of Technology and Science Education, 6(3), 188-208.
• Dietz-Uhler, B & Hurn, J.E. (2013). Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1), 17-26.
• Dyckhoff, A.L., Zielke, D., Chatti, M.A., & Schroeder, U. (2011, July). eLAT: An exploratory learning analytics tool for reflection and iterative improvement of technology enhanced learning. Paper presented at the 4th International Conference on Educational Data Mining. Eindhoven, The Netherlands.
• Gasevic, D., Jovanovic, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128.
• Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative assessment in higher education: A review of the literature. Computers & Education, 57(4), 2333-2351.
• Govaerts, S., Verbert, K., Duval, E. & Pardo, A. (2012). The student activity meter for awareness and self-reflection. CHI’12 Extended Abstracts on Human Factors in Computing Systems, 869–884.
• Harper, D. (2002). Talking about pictures: A case for photo elicitation. Visual Studies, 17(1), 13-26.
• Kruse, A. & Pongsajapan, R. (2012). Student-centered learning analytics. CNDLS Thought Papers, 1–9.
• McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012). Using quizzes to enhance summative-assessment performance in a web-based class: An experimental study. Journal of Applied Research in Memory and Cognition, 1, 18-26.
• O'Dowd, I. (2018). Using learning analytics to improve online formative quiz engagement. Irish Journal of Technology Enhanced Learning 3(1). Retrieved on 11 November 2018 from https://doi.org/10.22554/ijtel.v3i1.25
• Roediger, H. I., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382-395. doi: 10.1037/a0026252
• Rose, G. (2007). Making photographs as part of a research project: Photo-elicitation, photo-documentation and other uses of photos. In G. Rose (Ed.), Visual methodologies: An introduction to the interpretation of visual materials (2nd ed.) (pp. 237-256). Thousand Oaks, CA: Sage.
• Society for Learning Analytics Research. (2011). Open learning analytics: An integrated & modularized platform [Concept paper]. Retrieved on 11 November 2018 from http:/solaresearch.org/OpenLearningAnalytics.pdf.
• Wagner, J. (2006). Visible materials, visualized theory, and images of social research. Visual Studies, 21(1), 55-69.
• Weinstein, C. E., Husman, J., & Dierking, D. R. (2000). Self-regulation interventions with a focus on learning strategies. In P. R. Pintrich & M. Boekaerts (Eds.), Handbook on self-regulation (pp. 727-747). New York: Academic Press.
• Wise, A.F. (2014). Designing pedagogical interventions to support student use of learning analytics. LAK’14 Proceedings of the fourth International Conference on Learning Analytics & Knowledge. Indianapolis, IN. http://dx.doi.org/10.1145.2567574.2567588