The Effects of Learning Management Systems (LMS) on Mathematics Achievement: A Meta-Analysis Study

Year 2021, Volume: 15 Issue: 2, 341 - 362, 31.12.2021

Hatice Saygılı , Hatice Çetin

https://doi.org/10.17522/balikesirnef.1026534

Abstract

The purpose of this investigation is to examine the effect of LMS (Learning Management System) use on students’ mathematic achievement through a meta-analysis method. 43 experimental studies with a data set including standard deviations, mean scores and sample sizes were incorporated in the analysis. The total number of samples from the studies included in the analysis is 15.296. Data were analyzed using Comprehensive Meta Analysis (CMA) software. After the analysis was completed in accordance with the random effects model, the Cohen d effect value of LMS use on students' mathematics achievement was calculated as 0.363. The results of the subgroup analysis of this effect size value indicated that the effect of LMS use on mathematics achievement did not differ significantly between subgroups with reference to the variables of sample, type of publication, duration of application and method of application. On the other hand, it was found that there was a significant difference between the subgroups for the variables of year, country, subject and education level.

Keywords

Learning management system, mathematics achievement, meta- analysis.

References

• Adzharuddin, N. A. & Ling, L. H. (2013). Learning management system (LMS) among university students: Does it work? International Journal of E-Education, 3(3), https://doi.org/10.7763/IJEEEE.2013.V3.233.
• Aguinis, H., Gottfredson, R. K., & Wright, T. A. (2011). Best‐practice recommendations for estimating interaction effects using meta‐analysis. Journal of Organizational Behavior, 32(8), 1033-1043.
• Anthony, C. (2015). Student Achievement in Mathematics: A Comparison Of Onlıne And Direct Instruction. Doctor of Education. Northern Arizona University. Arizona.
• Applebee, J. (2019). Student Usage of Open Educational Resource Learning Materials in Algebra 2. Doctor of Education. The State University. New Jersey.
• Avgeriou, P., Papasalouros, A., Retalis, S., ve Skordalakis, M. (2003). Towards a pattern language for learning management systems, Journal of Educational Technology & Society, 11-24, https://www.researchgate.net/publication/26393703.
• Beam, P. & Cameron, B. (1998). But what did we learn? Evaluating online learning as process. Paper Presented at the Sixteen Annual International Conference on Computer Documentation, 20(5), 258-264, https://doi.org/10.1145/296336.29639.
• Belanger, M. (2018). Effects of Blended Learning and Gender on Mathematics Assessment in Elementary Fourth and Fifth Grade Students. Doctor of Education. Grand Canyon University. Arizona.
• Borenstein, M., Hedges, L. V., Higgins, J. P. T. & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex: Wiley.
• Boyd, A. C. (2018). Perceived Parent Involvement and a Technology-Enabled Workbook Intervention Effect Analysis on Summer Learning Loss for 6th and 7th Grade Students Attending a Pennsylvania K-12 Virtual School. Doctor of Education. The George Washington University. Washington.
• Bradley, K. (2016). Evaluating The Effects of Mastery Learning in Postsecondary Developmental Mathematics. Doctor of Education. University of Louisiana. Louisiana.
• Card, N. A. (2012). Applied meta-analysis for social science research. New York: The Guilford Press.
• Chadwick, D. K. H. (1997). Computer-Assisted İnstruction in Secondary Mathematics Classrooms: A Meta-Analysis. Master’s Thesis. Drake University. United States-Iowa.
• Chan, K. K. & Leung, S. W. (2014). Dynamic geometry software improves mathematical achievement: Systematic review and meta-analysis, Journal of Educational Computing Research, 51(3), 311–325, https//doi.org/10.2190/EC.51.3.c.
• Chaney, T. A. (2016). The Effect of Blended Learning on Math and Reading Achievement in a Charter School Context. Doctor of Education. Liberty University. Liberty.
• Chang, C., Jhu, K., Liang, C., Tseng, J.ve Hsu, Y. (2014). Is blended 3-learning as measured by an achievement test and self-assessment better than traditional classroom learning for vocational high school students? International Review of Research in Open and Distance Learning,15(2), 213-231, https://www.researchgate.net/publication/271831578.
• Cheung, A. C. K. & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis, Educational Research Review, 9, 88–113, https://doi.org/10.1016/j.edurev.2013.01.001.
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational And Psychological Measurement, 20, 37-46.
• Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th Edition). New York: Routledge.
• Cole, J. & Foster, H. (2008). Using moodle: Teaching with the popular open source course management system. Sebastopol, CA: O'Reilly Media.
• Comfort, J. (2016). An Exploratory Study of the Relationship Between a Blended Learning Approach to İnstruction and 5st Grade Student Performance in a Kansas Public School District. Doctor of Education. Studies and the Graduate Faculty of the University. Kansas.
• Crowley, K. (2018). The Impact of Adaptive Learning on Mathematics Achievement. Doctor of Education. New Jersey City University. New Jersey.
• Day, P. A. (2017). Effectiveness of the Career and College Promise Program in Increasing College Readiness at a Rural North Carolina Community College. Doctor of Education. The Gardner-Webb University School of Education. Gardner.
• Deniz, S. (2019). Teknoloji Destekli Öğretimin Matematik ve Geometri Alanlarında Başarı ve Tutuma Etkisi Üzerine Bir Meta-Analiz Çalışması.(Yayınlanmamış Yüksek Lisans Tezi). Yüzüncü Yıl Üniversitesi. Van.
• Dinçer, S. (2014). Eğitim bilimlerinde uygulamalı meta-analiz. Ankara: Pegem Akademi.
• Duval, S. & Tweedie, R. (2000). Trim and fill: a simple funnel‐plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
• Ellis, P. D. (2010). The essential guide to effect sizes: Statistical power, meta-analysis and the interpretation of research result. Cambridge: Cambridge University Press.
• Francis, J. (2016). Next Generation Online Math I Course Evaluation. Master of Arts in Educational Psychology. West Virginia University. West Virginia.
• Gangaram, J. (2014). Blended and Online Student Performance and Persistence: A Comparative Study. Doctor of Education. North central University Graduate Faculty of the School of Education. Arizona.
• Guzer, B. ve Caner, H. (2014). The past, present and future of blended learning: An in depth analysis of literature. Eastern mediterranean university, Faculty of Education, Department of Educational Sciences, Famagusta, North Cyprus, Social and Behavioral Sciences, 116(14), 4596 – 4603, https//doi.org/10.1016/j.sbspro.2014.01.992.
• Higgins, J., Thompson, S. G., Deeks, J. J. ve Altman, D. G. (2003). Measuring inconsistency in meta-analysis, BMJ, 327, 557-560, https://www.researchgate.net/publication/10580837.
• Huang, K. (2012). College Student Competency and Attitudes in Algebra Classes: A Comparison of Traditional and Online Delivery Methods in Exponents and Polynomials Concepts. Doctor of Education. Idaho State University Instructional Design in the College of Education. Idaho.
• Ichinose, C. L. (2011). Learning Mathematics in The 21St Century: High School Students’ Interactıons While Learning Mathematics Online. Doctor of Philosophy. The Claremont Graduate University. California.
• Kelismail, E. (2019). Eğitim Bilişim Ağı (EBA) Destekli Öğretimin 6. Sınıf Öğrenclerinin Cebirsel İfadeler Alt Öğrenme Alanında Matematik Başarılarına ve Tutumlarına Etkisi. Yüksek Lisans Tezi. Gazi Üniversitesi Eğitim Bilimleri Enstitüsü. Ankara.
• Korucu, A.T. & Kabak, K. (2020). Türkiye hibrit öğrenme uygulamaları ve etkileri: Bir meta-analiz çalışması, Bilgi ve İletişim Teknolojileri Dergisi, 2(2), 88-112, https://dergipark.org.tr/tr/pub/bited/issue/58421/80322.
• Kunzler, J. S. (2012). Exploring Customization in Higher Education: An Experiment in Leveraging Computer Spreadsheet Technology to Deliver Highly Indıvıdualızed Online Instructıon to Undergraduate Business Students. Doctor of Education. Idaho State University. Idaho.
• Kwan lo, C. & Foon, H. K. (2017). Using “First Principles of Instruction” to design secondary school mathematics flipped classroom: The findings of two exploratory studies, Educational Technology and Society, 20(1), 222–236, https://www.researchgate.net/publication/312045900.
• Lee, W. C. (1990). The Effectiveness of Computer-Assisted İnstruction and Computer Programing in Elementary and Secondary Mathematics: A Meta–Analysis. Doctor of Education. University of Massachusetts.
• Lee, J. E. (2019). Examining the Effects of Discussion Strategies and Learner Interactions on Performance in Online Introductory Mathematics Courses: An Application of Learning Analytics. Doctor of Philosophy. Utah State University. Utah.
• Li, Q. & Ma, X. (2010). A meta- analysis of the effects of computer technology on school students mathematics learning, Educational Psychology Review, 22(3), 215-243, https//doi.org/10.1007/s10648-010-9125-8.
• Liu, F. (2010). Factors Influencing Success in Online High School Algebra. Doctor of Philosophy. University of Florida. Florida. Massoud, A., Iqbal, U. ve Stockley, D. (2011). Using blended learning to foster education in a contemporary classroom, Transformative Dialogues: Teaching & LearningJournal, 5(2),111, https://www.researchgate.net/publication/254560902 McCray, M. (2019). FRACTIONVILLE: Impact of Gamification on Learning Foundational Fractions in the Third Grade, Doctor of Education. Kean University. New Jersey.
• Meylani, R. (2016). Short, Medium and Long Term Effects of an Online Learning Activity Based (OLAB) Curriculum on Middle School Students Achievement in Mathematics: A Quasi- Experimental Quantitative Study. Doctor of Philosophy. Arizona State University. Arizona.
• Mills, J. J. (2016). A Mixed Methods Approach to Investigating Cognitive Load and Cognitive Presence in an Online and Face-To-Face College Algebra Course. Doctor of Education. Education at the University of Kentucky. Lexington. Kentucky.
• Newberry, G. W. (2011). E-Learning Styles: A Study of Algebra Achievement for the Mıddle School E-Learner. Doctor of Philosophy. Capella University. Capella.
• Norvell, E. A. (2017). Improving Mathematical Understanding: The Effects of Delivery Modes in Pre-Engineering Math Classes. Doctor of Philosophy. North central University School of School of Education. California.
• Nies, K. (2018). The Impact of the Use of Open Education Resources on College Student Success, Course Evaluation and Course Preference. Doctor of Philosophy. TUI University. California.
• Nwaogu, N. E. (2012). The Effect of Aleks on Student’s Mathematics Achievement in an Online Learning Environment and the Cognitive Complexity of the Initial and Final Assessments. Georgia State University. Doctor of Philosophy. Georgia.
• Odom, S. B. (2006). The Effects of Self-Regulated Learning Strategies and Technology Instructional Strategies On the Math Achievement of Junıor High Students. Doctor of Philosophy. University of South Alabama. Alabama.
• Orwin, R. G. (1983). Afail safe N for effect size in meta-analysis, Journal of Educational Statistics, 8(2), 157-159, https://www.jstor.org/stable/1164923.
• Osborne, S. F. (2020). Using Online Interventions to Address Summer Learning Loss in Rising Sixth-Graders. Doctor of Education. University of Missouri-St. Louis. Columbia.
• Özerbaş, M. A. (2012). WebQuest öğrenme ortamının öğrencilerin akademik başarı ve tutumlarına etkisi, Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi (KEFAD), 13(2), 299-315, https://dergipark.org.tr/tr/pub/kefad/issue/59489/85499.
• Pope, H. (2013). Student Success Rate in Online Learning Support Classes Compared to Traditional Classes. Doctor of Education. Walden University. Washington.
• Renee, R. (2017). Effectiveness of Blended Learning in a Rural Alternative Education School Setting. Doctor of Education. Liberty University a Dissertation Presented in Partial Fulfillment of the Requirements for the Degree. Liberty.
• Schenker, J. D. (2007). The Effectiveness of Technology Use in Statistics Instruction in Higher Education: A Meta-Analysis Using Hierarchical Linear Modeling. Doctor of Education. Retrieved From ProQuest Digital Dissertations. (AAT 3286857).
• Smith, R. N. (2017). Perceptıons and Effects of Classroom Capture Software on Course Performance Among Selected Online Community College Mathematics Students. Doctor of Education. Sam Houston State University. Houston. Steenbergen-Hu, S. & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students mathematical learning, Journal of Educational Psychology, 105(4), 970–987, https//doi.org/10.1037/a0032447.
• Şimşek, Ö. (2010). Web Destekli Matematik Öğretiminde Kullanılan Video Derslerin Öğrenenlerin Türev Başarılarına Etkisi ve Öğrenenlerin Video Derslere İlişkin Görüşleri. Yüksek Lisans Tezi. Ege Üniversitesi Fen Bilimleri Enstitüsü. İzmir.
• Şahinoğlu, E. (2012). Moodle Ders Yönetimi Bilgi Sistemi Destekli Matematik Öğretiminin, Öğrencilerin Matematik Başarısına ve Matematik Dersine Yönelik Tutumlarına Etkisi. Yüksek Lisans Tezi. Gazi Üniversitesi Eğitim Bilimleri Enstitüsü Eğitim Bilimleri Anabilim Dalı Eğitim Teknolojisi Bilim Dalı. Ankara.
• Tarazi, N. (2016). The Influence of the Inverted Classroom on Student Achievement and Motivation for Learning in Secondary Mathematics in the United Arab Emirates: A Quasi-Experimental Study. Doctor of Education. North central University. Arizona.
• Tekin, O. (2018). Ters Yüz Sınıf Modelinin Lise Matematik Dersinde Uygulanması: Bir Karma Yöntem Çalışması. Doktora Tezi. Gaziosmanpaşa Üniversitesi Eğitim Bilimleri Enstitüsü. Tokat.
• Telford, W. D. (2011). Investigating the Use of Destination Math in an Urban School District. Doctor of Philosophy. Texas A ve M University. Texas.
• Thalheimer, W. & Cook, S. (2002). How to calculate effect sizes from published research: A simplified methodology, Work-Learning Research, 1, 1-9, https://www.researchgate.net/publication/253642160.
• Tokpah, C. L. (2008). The Effects of Computer Algebra Systems on Students Achievement in Mathematics. Doctor of Education. Kent State University, United States - Ohio.
• Toth, P. F. (2013). Measuring efficiency of teaching mathematics online: Experiences with WeBWorK, Social and Behavioral Sciences, 89, 276 – 282, https://doi.org/10.1016/j.sbspro.2013.08.846.
• Williamson, K. (2017). A Comparative Study on Web-Based Technology Support and Its Value on Student Success in Middle School Mathematics. Doctor of Education. North central University. Prescott Valley. Arizona.
• Winn, D. (2016). The Effects of Blended Learning on Colorado Measures of Academic Success. Doctor of Education. Northern Arizona University. Arizona.
• Woolley, S. ve Ludwig, S. (2000). Online learning communities: Vehicles for collaboration and learning in online Learning environments. Canada: Association for the Advancement of Computing in Education (AACE).
• You, J. W. (2015). Identifying significant indicators using LMS data to predict course achievement in online learning, Internet and Higher Education, 29, 23–30, https://doi.org/10.1016/j.iheduc.2015.11.003.
• Young, J. & Hamilton, C. (2018). Technology effectiveness in the mathematics classroom: A systematic review of meta-analytic research, Journal of Computers Education, 5(2), 133–148, https://www.researchgate.net/publication/325105360.
• Zenati, L. (2020). Implementing Asynchronous Discussion as an Instructional Strategy in the Developmental Mathematics Courses to Support Student Learning. Doctor of Education. Illinois Institute of Technology. Chicago. http://prisma-statement.org/prismastatement/Checklist.aspx