Unpacking TPACK in Mathematics Education Research: A Systematic Review of Meta-Analyses

Abstract

Teaching with technology is considered a necessity in the U.S. mathematics classroom. However, few studies have established explicit considerations to support technology-enhanced student achievement. The purpose of this study was to characterize the effectiveness of technology in the mathematics classroom by systematically reviewing meta-analytic research. An exhaustive literature search was conducted. After applying a prioi inclusion criteria the pool of 65 initial meta-analyses was reduce to 13 representative studies. Each study was reviewed and characteristics were coded in four categories: (1) sample, (2) measurement, (3) design, and (4) source. An inductive review of the coded studies produced five unique moderators that were the most salient across studies. Overall mean effect sizes were retrieved or calculated from available study data. Hedges g was used as the common effect size metric for comparison across studies.  The Technological Pedagogical Content Knowledge (TPACK) framework was used to interpret the most salient moderators of effects across studies.  Studies were categorized by didactical functionality and technology type. The results suggest that effects vary by didactical functionality from small to medium. The largest variations were observed for the didactical function of developing conceptual understanding.  Implications for research and instructional praxis are provided.

Keywords

• Meta-Analysis, systematic review, TPACK, mathematics, achievement

References

1. Abbitt, J. T. (2011). Measuring Technological Pedagogical Content Knowledge in Preservice Teacher Education: A Review of Current Methods and Instruments. Journal of Research on Technology in Education, 43(4), 281-300.
2. Alavi, M., & Leidner, D. (2001). Research commentary: Technology-mediated learning- A call for greater depth and breadth of research. Information Systems Research, 12, 1- 10.
3. Azevedo, R., & Bernard, R. M. (1995). A meta-analysis of the effects of feedback in computer-based instruction. Journal of Educational Computing Research, 13(2), 111–127.
4. *Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K–12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.
5. Clark, R. E. (1985). Evidence for confounding in computer-based instruction studies: Analyzing the meta-analyses. Educational Communication and Technology Journal, 33(4), 249–262.
6. Close, S., Oldham, E., Shiel, G., Dooley, T., & O’Leary, M. (2012). Effects of calculators on mathematics achievement and attitudes of ninth-grade students. Journal of Educational Research, 105(6), 377–390.
7. Cooper, H., & Patall, E. A. (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14(2), 165-176.
8. Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technologies in high school classrooms: Explaining an apparent paradox. American Educational Research Journal, 38(4), 813-834.

9. DeCoster, J. (2004). Meta-analysis notes. Retrieved from http://www.stat-help.com/Meta%20analysis%202009-06-01.pdf
10. Dexter, S., Anderson, R. E., & Becker, H. J. (1999). Teachers’ views of computers as catalyst for changes in their teaching practice. Journal of Research on Computing in Education, 31(3), 221-239.
11. Drijvers, P., Boon, P., & Van Reeuwijk (2010). Algebra and technology. In P. Drijvers (Ed.), Secondary algebra education, Revisiting topics and themes and exploring the unknown (pp. 179–202). Rotterdam: Sense.
12. Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of pediatric psychology, Retrieved from http://jpepsy.oxfordjournals.org/content/early/2009/02/16/jpepsy.jsp004.short.
13. *Ellington, A. J. (2006). The effects of non-CAS graphing calculators on student achievement and attitude levels in mathematics: A meta-analysis. School Science and Mathematics, 106(1), 16–27.
14. Ertmer, P. A., Addison, P., Lane, M., Ross, E., & Woods, D. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54-72.
15. Guerrero, S. (2010). Technological Pedagogical Content Knowledge in the Mathematics Classroom. Journal of Digital Learning in Teacher Education, 26(4), 132-139.
16. Harris, J. B., Mirsha, P., & Koehler, M. J. (2007, April). Teachers’ technological pedagogical content knowledge: Curriculum-based technology integration reframed. Paper presented at American Educational Research Association conference, Chicago, IL.
17. Hicks, D., & Holden, C. (Eds.), (2007). Teaching the global dimension: Key principles and effective practice. London, England: Routledge.
18. *Hsu, Y. (2003). The effectiveness of computer-assisted instruction in statistics education: A meta-analysis (Ph.D.). The University of Arizona, United States, Arizona. Retrieved from http://search.proquest.com/pqdtglobal/docview/305338759/abstract/9D8837D0D8EB4A14PQ/29
19. John, P., & Sutherland R. (2005). Affordance, opportunity, and the pedagogical implications of ICT. Educational Review, 57(4), 405–413.
20. Kemery, E. R., Mossholder, K. W., & Dunlap, W. P. (1989). Meta-analysis and moderator variables: A cautionary note on transportability. Journal of Applied Psychology, 74(1), 168-170.
21. *Larwin, K., & Larwin, D. (2011). A meta-analysis examining the impact of computer-assisted instruction on postsecondary statistics education: 40 years of research. Journal of Research on Technology in Education, 43(3), 253–278.
22. *Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215–243.
23. Lipsey, M. W. (2003). Those Confounded Moderators in Meta-Analysis: Good, Bad, and Ugly. Annals of the American Academy of Political and Social Science, 587, 69-81.
24. Lipsey, M. W. (2009). Identifying interesting variables and analysis opportunities. In H. Cooper, L. V.
25. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (pp. 147–158). New York, NY: Sage Publications, Inc.
26. Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108(4), 1017-1054.
27. Mishra, P., & Koehler, M. (2008, March). Introducing technological pedagogical content knowledge. Paper presented at the annual meeting of the American Educational Research Association, New York.
28. Mitchell, K., Bakia, M., & Yang, E. (2007). State strategies and practices for educational technology: Volume II-Supporting mathematics instruction with educational technology. Washington, DC: US Department of Education, Office of Planning, Evaluation, and Policy Development.
29. *Moyer-Packenham, P. S., & Westenskow, A. (2013). Effects of virtual manipulates on student achievement and mathematics learning. International Journal of Virtual and Personal Learning Environments, 4(3), 35–50.
30. National Center for Educational Statistics. (2009). The nation’s report card. Retrieved from http://nces.ed.gov/nationsreportcard/mathematics/whatmeasure.asp
31. National Council of Teachers of Mathematics. (2000). Principles and standards of school mathematics. Reston, VA: Author.
32. *Nikolaou, C. (2001). Hand-held calculator use and achievement in mathematics: A meta-analysis (Ph.D.). Georgia State University, United States, Georgia. Retrieved from http://search.proquest.com/pqdtglobal/docview/304696658/abstract/9D8837D0D8EB4A14PQ/3
33. Niess, M. L. (2005). Preparing teachers to teach science and mathematics with technology: Developing a technology pedagogical content knowledge. Teaching and Teacher Education, 2(3), 509-523.
34. Pierson, M. E. (2001). Technology integration practice as a function of pedagogical expertise. Journal of Research on Computing in Education, 33(3), 413-429.
35. Phillips-Bey, C. K. (2004). TI-73 calculator activities. Mathematics Teaching in the Middle School, 9, 500–505.
36. Roschelle, J., Pea, R., Hoadley, C., Gordin, D., & Means, B. (2000). Changing how and what children learn in school with computer-based technologies. The Future of Children, 10, 76-101.
37. *Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1–14.
38. Russell, M., Bebell, D., O'Dwyer, L., & O'Connor, K. (2003). Examining teacher technology use: Implications for preservice and inservice teacher preparation. Journal of Teacher Education, 54(4), 297-310.
39. Russell, C. J., & Gilliland, S. W. (1995). Why Meta-Analysis Doesn't Tell Us What the Data Really Mean: Distinguishing between Moderator Effects and Moderator Processes. Journal of Management, 21(4), 813-831.
40. Sanchez-Meca, J., & Marin-Martinez, F. (1998). Testing continuous moderators in meta-analysis: A comparison of procedures. British Journal of Mathematical & Statistical Psychology, 51, 311-326.
41. Sandholtz, L., Ringstaff, C., & Dwyer, D. (1997). Teaching with technology: Creating student-centered classrooms. New York: Teachers College Press.
42. *Schenker, J. D. (2007). The effectiveness of technology use in statistics instruction in higher education: A meta-analysis using hierarchical linear modeling (Ph.D.). Kent State University, United States, Ohio. Retrieved from http://search.proquest.com/pqdtglobal/docview/304835263/abstract/9D8837D0D8EB4A14PQ/25
43. Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4-14.
44. Sokolowski, A., Li, Y., & Willson, V. (2015). The effects of using exploratory computerized environments in grades 1 to 8 mathematics: A meta-analysis of research. International Journal of STEM Education, 2(1), 1–17.
45. *Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970–987.
46. Thompson, A. D., & Mishra, P. (2008). Breaking news: TPCK becomes TPACK! Journal of Computing in Teacher Education, 24(2), 38-39.
47. *Tokpah, C. L. (2008). The effects of computer algebra systems on students’ achievement in mathematics (Ph.D.). Kent State University, United States, Ohio. Retrieved from http://search.proquest.com/pqdtglobal/docview/304549974/abstract/9D8837D0D8EB4A14PQ/41
48. Wang, G., Oh, I. S., Courtright, S. H., & Colbert, A. E. (2011). Transformational leadership and performance across criteria and levels: A meta-analytic review of 25 years of research. Group & Organizational Management, 36, 223–270.
49. Watson, D. (2001). Pedagogy before technology: Re-thinking the relationship between ICT and teaching. Education and Information Technologies, 6(3), 251-266.
50. Webb, M. (2005). Affordances of ICT in science learning: Implications for an integrated pedagogy. International Journal of Science Education, 27(5), 705–735.
51. Windschitl, M., & Sahl, K. (2002). Tracing teachers’ use of technology in a laptop computer school: The interplay of teacher beliefs, social dynamics, and institutional culture. American Educational Research Journal, 39(1), 162-205.
52. Young, J. R., & Young, J. L. (2012). “But that’s not fair”: Teacher technology readiness and African American Students. The Journal of the Texas Alliance of Black School Educators, 4(1), 19-32.
53. Young, J. R., Young, J. L., & Hamilton, C. (2013). The use of confidence intervals as a meta-analytic lens to summarize the effects of teacher education technology courses on preservice teacher TPACK. Journal of Research on Technology in Education, 46(2), 149-172.
54. Yung, H. I., & Paas, F. (2015). Effects of computer-based visual representation on mathematics learning and cognitive load. Journal of Educational Technology & Society, 18(4), 70–77.
55. Zhao, Y., Pugh, K., Sheldon, S., & Byers, J. L. (2002). Conditions for classroom technology innovations. Teachers College Board, 104(3), 482-515.