Educational Technology Research Past and Present: Balancing Rigor and Relevance to Impact School Learning

Abstract

Today, the exponential growth of technology usage in education, via such applications of distance education, Internet access, simulations, and educational games, has raised substantially the focus and importance of educational technology research. In this paper, we examine the past and present research trends, with emphasis on the role and contribution of research evidence for informing instructional practices and policies to improve learning in schools. Specific topics addressed include: (a) varied conceptions of “effective” technology uses in classroom instruction as topics for research, (b) historical trends in research approaches and topics of inquiry; (c) alternative research designs for balancing internal (rigor) and external (relevance) validity; and (d) suggested directions for future research. Attention is devoted to describing varied experimental designs as options for achieving appropriate rigor and relevance of research evidence, and using mixed-methods research for investigating and understanding technology applications in complex real-life settings.

Keywords

• Educational technology research, Technology and learning, Research designs and trends, Impact of technology on learning.

References

1. Azevedo, R., & Cromley, J. G. (2004). Does training of self-regulated learning facilitate students' learning with hypermedia? Journal of Educational Psychology, 96, 523-535.
2. Azevedo, R., Cuthrie, J. T., & Seibert, D. (2004). The role of self-regulated learning in fostering students' conceptual understanding of complex systems with hypermedia. Journal of Educational Computing Research, 28(1), 15-30.
3. Barab, S. (2006). Design-based research: A methodological toolkit for the learning scientist. In R. K. Sawyer (Ed.), Cambridge handbook for the learning sciences (pp. 153-170). Cambridge, UK: Cambridge University Press.
4. Bernard, R. M., Abrami, P. C., Borokhovski, C., Wade, C. A., Tamin, R. M., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research, 79(3), 1243-1289.
5. Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, C. A., Wozney, L., Wallet,P. W., Fiset, M., & Huang, B. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74, 379-439.
6. Calfee, R. (2006). Educational psychology in the 21st century. In P. A. Alexander & P. H. Winne (Eds.) Handbook of educational psychology (2nd Ed., pp. 29-42). Mahwah, NJ: Erlbaum.
7. Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to work: Employers perspectives on the knowledge and applied skills of new entrants to the 21st Century U.S. workforce. The Conference Board, Inc., the Partnership for 21st Century Skills, Corporate Voices for Working Families, and the Society for Human Resource Management. Retrieved January 21, 2008 from http://www.21stcenturyskills.org/index.php?option=com_content&task=view&id=82&Ite mid=40.
8. Chambers, B., Cheung, A., Gifford, R., Madden, N., & Slavin, R. E. (2006). Achievement effects of embedded multimedia in a Success for All reading program. Journal of Educational Psychology, 98, 232-237.

9. Chambers, B., Slavin, R. E., Madden, N. A., Abrami, P. C., Tucker, B., J., Cheung, A., & Gifford, R. (2008). Technology infusion in Success for All: Reading outcomes for first graders. Elementary School Journal, 109 (1), 1-15.
10. Clark, R. E. (1983). Reconsidering the research on learning from media. Review of Educational Research, 53(4), 445-459.
11. Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development,42, 21-29.
12. Clark, R. E. & Feldon, D. F. (2005). Five common but questionable principles of multimedia learning. In R. E. Mayer (Ed.) The Cambridge Handbook of Multimedia Learning (pp. 97- 115). New York: Cambridge University Press.
13. Dynarski, M., Agodini, R., Heaviside, S., Novak, T., Carey, N., Campuzano, L., Means, B., Murphy, R., Penuel, W., Javitz, H., Emery, D., & Sussex, W. (2007). Effectiveness of reading and mathematics software products: Findings from the first student cohort. Washington, DC: Institute of Education Sciences.
14. Eisenhart, M., & Towne, L. (2003). Contestation and change in national policy on "scientifically based" education research. Educational Researcher, 32(7), 31-38.
15. Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25-39.
16. Ertmer, P. A., Gopalakrishnan, S., & Ross, E. M. (2001). Technology-using teachers: Comparing perceptions of exemplary technology use to best practice. Journal of Research on Technology in Education, 33(5). Retrieved January 4, 2008 from http://www.iste.org/jrte/33/5/ertmer.html.
17. Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246-259.
18. Hannafin, R. D., & Sullivan, H. J. (1996). Learner preferences and learner control over amount of instruction. Journal of Educational Psychology, 88, 162–173.
19. Hannafin, R. D., & Young, M. (2008). Research on educational technologies. In M. Spector, M. D. Merrill, J. V. Merrienboer, & M. Driscoll (Eds.). Handbook of research on educational communications and technology, Third Edition (pp. 731-739). New York: Routledge.
20. Hsieh, P., Acee, T., Chung, W., Hsieh, Y., Kim, H., Thomas, G., Levin, J. R., & Robinson, D. H. (2005). Is educational intervention research on the decline? Journal of Educational Psychology, 97(4), 523-529.
21. Jeong, A., & Davidson-Shivers, G.V. (2006).The effects of gender interaction patterns on participation in computer-supported collaborative argumentation. Educational Technology Research and Development, 54(6), 543-568.
22. Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.
23. Jonassen, D. H. (1994). Thinking technology: Toward a constructivist design model. Educational Technology 34(4), 34-37.
24. Karr, C. L., Weck, B., Sunal, D. W., & Cook, T. M. (2003). Analysis of the effectiveness of online learning in a graduate engineering math course. Journal of Online Interactive Learning, 1(3)., from www.ncolr.org/jiol/achieves/2003/winter/3/ms02023_Karr.
25. Kauffman, D. F. (2004). Self-regulated learning in web-based environments: Instructional tools designed to facilitate cognitive strategy use, metacognitive processing, and motivational beliefs. Journal of Educational Computing Research, 30, 139-161.
26. Keegan, D. (1996). Foundations of distance education (3rd ed.). London: Routledge.
27. Kirby, J. A., Hoadley, C. M., & Carr-Chellman, A. A. (2005). Instructional systems design and the learning sciences: A citation analysis. Educational Technology Research & Development, 53, 37-48.
28. Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42, 7-19.
29. Kozma, R. B. (2003). Technology and classroom practices: An international study. Journal of Research on Technology in Education, 36(1), 1-14.
30. Kulik, J. A. (2003). Effects of using instructional technology in elementary and secondary schools: What controlled evaluation studies say. SRI Project Number P10446.001. Arlington, VA: SRI International.
31. Lee, H. W., Lim, K. Y., & Grabowski, B. L. (2008). Generative learning: Principles and implications for making meaning. In M. J. Spector, D. M. Merrill, J. van Merrienboer & M. P. Driscoll (Eds.), Handbook of research and educational communications and technology (3rd ed.). New York, NY: Taylor & Francis Group.
32. Leu, D. J., O’Byrne, W. I., Zawlinski, L., McVerry, G., & Everett-Cacopardo, H. (2009). Expanding the new literacies conversation. Educational Researcher, 38(4), 264-269.
33. Levin, J. R. (2004). Random thoughts on the (in)credibility of educational-psychological intervention research. Educational Psychologist, 39(3), 173-174.
34. Lowther, D.L., Inan, F.A., Strahl, J.D., Ross, S.M. (2008). Does technology integration “work” when key barriers are removed? Educational Media International, 45(3), 195-206.
35. Lowther, D. L., Inan, F. A., Strahl, D. J., & Ross, S. M., (April, 2009). Do one-to-one initiatives bridge the way to 21st Century Knowledge and Skills? Paper presented at the 2009 American Educational Research Association National Conference, San Diego, CA.
36. Lowther, D. L., Ross, S. M., & Morrison, G. R. (2003). When each one has one: The influences on teaching strategies and student achievement of using laptops in the classroom. Educational Technology Research and Development, 51(03), 23-44.
37. Lowther, D. L., Ross, S. M., Wang, L. W., Strahl, D., & McDonald, A. (2004). Tennessee Department of Educational EdTech Launch 1 2003-2004 Evaluation Report. Memphis, TN: The University of Memphis, Center for Research in Educational Policy.
38. Morrison, G. R. (2001). Equivalent evaluation of instructional media: The next round of media comparison studies. In R. E. Clark (Ed.), Learning from instructional media: Arguments, analysis, and evidence (pp. 319-326). Greenwich, CT: Information Age Publishers.
39. Morrison, G. R., Ross, S. M., Gopalakrishnan, M., & Casey, J. (1995). The effects of feedback and incentives on achievement in computer-based instruction. Contemporary Educational Psychology, 20, 32-50.
40. Morrison, G. R., Ross, S. M., Kemp, J. E., & Kalman, H.(2010). Designing effective instruction: Applications of instructional design (6th. Ed.), New York, NY: Wiley.
41. Nolen, A. L. (2009). The content of educational psychology: An analysis of top-ranked journals from 2003 to 2007. Educational Psychology Review, 21, 279-289.
42. Penuel, W. R., Boscardin, C. K., Masyn, K., & Crawford, V. M. (2007). Teaching with student response systems in elementary and secondary education settings: A survey study. Educational Technology, Research & Development, 55 315-346.
43. Richey, R. C., & Klein, J. D. (2008). Research on design and development. In M. Spector, M. D. Merrill, J. V. Merrienboer, & M. Driscoll (Eds.). Handbook of research on educational communications and technology, Third Edition (pp. 748-757). New York: Routledge.
44. Ringstaff, C., & Kelly, L. (2002). The learning return on our educational technology investment: A review of findings from research. San Francisco, CA: WestEd RTEC.
45. Ross, S.M. & Lowther, D.L. (2009). Effectively using technology in education. Better Evidence-Based Education, 2(1), 20-21.
46. Ross, S. M., Lowther, D. L., Wang, L. W., Strahl, J. D. & McDonald, A. J. (2004). Tennessee Department of Education EdTech Launch 1 2003-2004 Evaluation Report. Memphis, TN: The University of Memphis, Center for Research in Educational Policy.
47. Ross, S. M., & Morrison, G. R. (1989). In search of a happy medium in instructional technology research: issues concerning internal validity, media replications, and learner control. Educational Technology Research and Development, 37, 19-24.
48. Ross, S. M. & Morrison. G. R. (2004). Experimental research methods, In D. J. Jonassen (Ed). Handbook of research on educational communications and technology, 2nd Ed., (pp. 1021- 1043). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
49. Ross, S. M., & Morrison, G. R. (2008). Research on instructional strategies. In M. Spector, M. D. Merrill, J. V. Merrienboer, & M. Driscoll (Eds.). Handbook of research on educational communications and technology, Third Edition (pp. 719-730). New York: Routledge.
50. Ross, S. M., Morrison, G. R., & Lowther, D. L. (2005). Using experimental methods in higher education. Journal of Computing in Higher Education, 16(2), 39-64.
51. Ross, S. M., Morrison, G. R., & O'Dell, J. K. (1989). Uses and effects of learner control of context and instructional support in computer-based instruction. Educational Technology, Research, and Development. 37, 29-39.
52. Russell, T. L. (1999). The no significant difference phenomenon. Chapel Hill: Office of Instructional Telecommunications, North Carolina State University.
53. Slavin, R. (2008). What works? Issues in synthesizing education program evaluations. Educational Researcher, 37 (1), 5-14.
54. Slavin, R. E. (2009). Educational Psychology: Theory into Practice (Ninth Edition). Upper Saddle River, NJ: Pearson.
55. Slavin, R.E., Lake, C., Chambers, B., Cheung, A., & Davis, S. (in press). Effective reading programs for the elementary grades. Review of Educational Research. http://www.edweek.org.
56. Spector, M., Merrill, M. D., Merrienboer, J. V., & Driscoll, M. (2008) Handbook of research on educational communications and technology, Third Edition. New York: Routledge.
57. van den Akker, J., & Kuiper, W. (2008). Research on models for instructional design. In M. J. Spector, D. M. Merrill, J. van Merrienboer & M. P. Driscoll (Eds.), Handbook of research and educational communications and technology (3rd ed., pp. 739-748). New York, NY: Taylor & Francis Group.
58. Videro, D. (2009). New head of U.S. research agency aims for relevance. Education Week, 29(13), 10.
59. Correspondence: Steven M. Ross, Professor, Center for Research and Reform in Education, Johns
60. Hopkins University, 2800 North Charles Street, Baltimore, MD 21218, United States.