Review
BibTex RIS Cite

Methodological gaps and domain imbalances in flipped learning reviews: A critical umbrella review

Year 2026, Volume: 13 Issue: 1, 21 - 44, 02.01.2026
https://doi.org/10.21449/ijate.1692300

Abstract

This critical umbrella review (UR) examined methodological rigor and outcome assessment practices in flipped learning (FL) research syntheses. A systematic analysis of 41 reviews (2017-2023) spanning STEM, medical/health sciences, and social sciences identified significant methodological concerns and domain imbalances in reported outcomes. The analysis revealed widespread inconsistencies in primary source selection, with most reviews excluding grey literature. Inadequate quality assessment practices were evident, with over half lacking formal risk of bias evaluation. The analysis also found variable approaches to publication bias assessment and potential language bias, with most reviews limiting inclusion to English-language publications. The analysis further reveals a pronounced imbalance in outcome domains. Cognitive outcomes dominated, particularly academic achievement. The affective and interpersonal domains received substantially less attention despite their theoretical importance to FL pedagogy. The paucity of specified cognitive metrics in numerous reviews was a matter of particular concern, whilst the near absence of metric reporting for affective and interpersonal outcomes was even more concerning. In addition, the psychometric properties of instruments were rarely critically evaluated. Although reviews consistently identified active learning, self-paced instruction, and increased interaction as key mechanisms for FL effectiveness, many failed to provide substantive exploration of how these mechanisms operate within specific disciplinary contexts. This critical synthesis highlights the need for methodological improvements in FL reviews, including comprehensive literature sourcing, rigorous quality assessment, expanded domain focus beyond cognitive outcomes, and deeper mechanism exploration to strengthen the evidence base guiding educational practice.

References

  • Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336
  • Adams, R.J., Smart, P., & Huff, A.S. (2017). Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4), 432 454. https://doi.org/10.1111/ijmr.12102
  • Aeschbach, L.F., Perrig, S.A.C., Weder, L., Opwis, K., & Brühlmann, F. (2021). Transparency in measurement reporting: A systematic literature review of CHI PLAY. Proceedings of the ACM on Human-Computer Interaction, 5(CHI PLAY), 233. https://doi.org/10.1145/3474660
  • Aguinis, H., Ramani, R.S., & Alabduljader, N. (2020). Best-practice recommendations for producers, evaluators, and users of methodological literature reviews. Organizational Research Methods, 26(1), 46–76. https://doi.org/10.1177/1094428120943281
  • Alqahtani, T.M., Yusop, F.D., & Halili, S.H. (2023). Content validity of the constructivist learning in higher education settings (CLHES) scale in the context of the flipped classroom in higher education. Humanities and Social Sciences Communications, 10(1). https://doi.org/10.1057/s41599-023-01754-3
  • Aromataris, E., Fernandez, R., Godfrey, C.M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-based Healthcare, 13(3), 132–140. https://doi.org/10.1097/xeb.0000000000000055
  • Aromataris, E., Fernandez, R., Godfrey, C., Holly, C., Khalil, H., & Tungpunkom, P. (2020). Chapter 10: Umbrella reviews. JBI Reviewer’s Manual. https://doi.org/10.46658/jbirm-17-08
  • Becker, L.A., & Oxman, A.D. (2008). Overviews of reviews. Cochrane Handbook for Systematic Reviews of Interventions, 607 631. https://doi.org/10.1002/9780470712184.ch22
  • Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8-14. https://doi.org/10.1016/j.npls.2016.01.001
  • Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. International society for technology in education.
  • Benzies, K.M., Premji, S., Hayden, K.A., & Serrett, K. (2006). State‐of‐the‐evidence reviews: Advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 3(2), 55–61. https://doi.org/10.1111/j.1741-6787.2006.00051.x
  • Bialik, M., Fadel, C., Trilling, B., Nilsson, P., & Groff, J. (2015). Skills for the 21st century: What should students learn. Centre for Curriculum Redesign.
  • Biesta, G. (2010). Good education in an age of measurement: Ethics, politics, democracy. Paradigm Publishers.
  • Birgili, B., Seggie, F.N., & Oğuz, E. (2021). The trends and outcomes of flipped learning research between 2012 and 2018: A descriptive content analysis. Journal of Computers in Education, 8(3), 365-394. https://doi.org/10.1007/s40692-021-00183-y
  • Bishop, J., & Verleger, M. (2013). Testing the flipped classroom with model-eliciting activities and video lectures in a mid-level undergraduate engineering course. In 2013 IEEE Frontiers in Education Conference (FIE) (pp. 161–163). https://doi.org/10.1109/fie.2013.6684807
  • Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. David McKay.
  • Borenstein, M., Hedges, L.V., Higgins, J.P., & Rothstein, H.R. (2009). Introduction to meta-analysis. John Wiley & Sons. https://doi.org/10.1002/9780470743386
  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
  • Campbell, M., Katikireddi, S.V., Sowden, A., & Thomson, H. (2019). Lack of transparency in reporting narrative synthesis of quantitative data: A methodological assessment of systematic reviews. Journal of Clinical Epidemiology, 105,1 9. https://doi.org/10.1016/j.jclinepi.2018.08.019
  • Care, E. (2017). 21st century skills: From theory to action. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 3–17). Springer.
  • Castro-Gil, R., & Correa, D. (2021). Transparency in previous literature reviews about blended learning in higher education. Education and Information Technologies, 26(3), 3399 3426. https://doi.org/10.1007/s10639-020-10406-x
  • Chen, H.J., Chen, C.H., & Wu, W.C.V. (2025). Effects of flipped learning on language learning outcomes: A meta analysis investigating moderators. SAGE Open, 15(2). https://doi.org/10.1177/21582440251331298
  • Clark, L.A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412 1427. https://doi.org/10.1037/pas0000626
  • Coman, L., & Richardson, J. (2006). Relationship between self-report and performance measures of function: A systematic review. Canadian Journal on Aging / La Revue Canadienne Du Vieillissement, 25(3), 253–270. https://doi.org/10.1353/cja.2007.0001
  • Creswell, J.W., & Plano Clark, V.L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE Publications.
  • Easterbrook, P.J., Gopalan, R., Berlin, J.A., & Matthews, D.R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867 872. https://doi.org/10.1016/0140 6736(91)90201-y
  • Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997). Language bias in randomised controlled trials published in English and German. The Lancet, 350(9074), 326-329. https://doi.org/10.1016/S0140-6736(97)02419-7
  • Ekinci, F., Bektaş, O., Karaca, M., & Yiğit, K.N. (2023). The flipped learning perception scale: A validity and reliability study. Education and Information Technologies, 28(11), 14139–14166. https://doi.org/10.1007/s10639-023-11731-7
  • Francis, N., Morgan, A., Holm, S., Davey, R., Bodger, O., & Dudley, E. (2020). Adopting a flipped classroom approach for teaching molar calculations to biochemistry and genetics students. Biochemistry and Molecular Biology Education, 48(3), 220 226. https://doi.org/10.1002/bmb.21328
  • Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502 1505. https://doi.org/10.1126/science.1255484
  • Fry, H., Ketteridge, S., & Marshall, S. (2009). A handbook for teaching and learning in higher education: Enhancing academic practice. Routledge. https://doi.org/10.4324/9781315763088
  • Grégoire, G., Derderian, F., & Le Lorier, J. (1995). Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias? Journal of Clinical Epidemiology, 48(1), 159–163. https://doi.org/10.1016/0895-4356(94)00098-b
  • Goedhart, N.S., Westrhenen, N.B., Möser, C., & Zweekhorst, M. (2019). The flipped classroom: Supporting a diverse group of students in their learning. Learning Environments Research, 22(2), 297-310.
  • Grant, M.J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91 108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
  • Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: Audit of primary sources. British Medical Journal, 331(7524), 1064–1065. https://doi.org/10.1136/bmj.38636.593461.68
  • Harrison, R., Jones, B., Gardner, P., & Lawton, R. (2021). Quality assessment with diverse studies (QuADS): An appraisal tool for methodological and reporting quality in systematic reviews of mixed- or multi method studies. BMC Health Services Research, 21(1). https://doi.org/10.1186/s12913-021-06122-y
  • Hartling, L., Vandermeer, B., & Fernandes, R.M. (2014). Systematic reviews, overviews of reviews and comparative effectiveness reviews: A discussion of approaches to knowledge synthesis. Evidence-Based Child Health: A Cochrane Review Journal, 9(2), 486–494. https://doi.org/10.1002/ebch.1968
  • Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D.M., & Vandermeer, B. (2017). Grey literature in systematic reviews: A cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child relevant reviews. BMC Medical Research Methodology, 17(1). https://doi.org/10.1186/s12874-017-0347-z
  • Helbach, J., Pieper, D., Mathes, T., Rombey, T., Zeeb, H., Allers, K., & Hoffmann, F. (2022). Restrictions and their reporting in systematic reviews of effectiveness: An observational study. BMC Medical Research Methodology, 22(1). https://doi.org/10.1186/s12874-022-01710-w
  • Hew, K.F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., & Thankrit, K. (2021). On the use of flipped classroom across various disciplines: Insights from a second-order meta-analysis. Australasian Journal of Educational Technology, 37(2), 132-151.
  • Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., & Welch, V.A. (Eds.). (2023). Cochrane handbook for systematic reviews of interventions (Version 6.4). Cochrane. www.training.cochrane.org/handbook
  • Holly, C., Salmond, S., & Saimbert, M. (Eds.). (2017). Comprehensive systematic review for advanced practice nursing. https://doi.org/10.1891/9780826131867
  • Hopewell, S., McDonald, S., Clarke, M.J., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews, (2), MR000010. https://doi.org/10.1002/14651858.MR000010.pub3
  • Huang, H.L., Chou, C.P., Leu, S., You, H.L., Tiao, M.M., & Chen, C.H. (2020). Effects of a quasi-experimental study of using flipped classroom approach to teach evidence-based medicine to medical technology students. BMC Medical Education, 20(1), https://doi.org/10.1186/s12909-020-1946-7
  • Joseph, J., Firmin, S., Oseni, T., & Stranieri, A. (2023). Decoding employee ambidexterity: Understanding drivers, constraints, and performance implications for thriving in the evolving work landscapes A scoping review. Heliyon, 9(12), e22493. https://doi.org/10.1016/j.heliyon.2023.e22493
  • Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002). Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. International Journal of Epidemiology, 31(1), 115-123. https://doi.org/10.1093/ije/31.1.115
  • Kager, K., Kalinowski, E., Jurczok, A., & Vock, M. (2024). A systematic review of transparency in lesson study research: How do we report on the observation and reflection stages? Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1322624
  • Kamei, F., Pinto, G., Wiese, I., Ribeiro, M., & Soares, S. (2021). What evidence we would miss if we do not use grey literature? Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 1 11. https://doi.org/10.1145/3475716.3475777
  • Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022). Fail, flip, fix, and feed – Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.956416
  • Krathwohl, D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of educational objectives: The classification of educational goals. Handbook II: Affective domain. David McKay.
  • Kreitchmann, R.S., Abad, F.J., Ponsoda, V., Nieto, M.D., & Morillo, D. (2019). Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of Likert items. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.02309
  • Kung, J., Chiappelli, F., Cajulis, O.O., Avezova, R., Kossan, G., Chew, L., & Maida, C.A. (2010). From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. The Open Dentistry Journal, 4(1), 84 91. https://doi.org/10.2174/1874210601004010084
  • Kyngäs, H., Kääriäinen, M., & Elo, S. (2020). The trustworthiness of content analysis. In Kyngäs, H., Mikkonen, K., Kääriäinen, M. (Eds.), The application of content analysis in nursing science research (pp. 41– 48). Springer. https://doi.org/10.1007/978-3-030-30199-6_5
  • Larsen, J.N., Maxwell, K.M.D., & Khalil, M. (2025). Exploring the link between motivational regulation and learning design with learning analytics. Journal of Learning Analytics, 12(1), 215–237. https://doi.org/10.18608/jla.2025.8469
  • Leatherman, J.L., & Cleveland, L.M. (2019). Student exam performance in flipped classroom sections is similar to that in active learning sections, and satisfaction with the flipped classroom hinges on attitudes toward learning from videos. Journal of Biological Education, 54(3), 328–344. https://doi.org/10.1080/00219266.2019.1575266
  • Lewis, C.C., Boyd, M.R., Walsh-Bailey, C., Lyon, A.R., Beidas, R., Mittman, B., Aarons, G.A., Weiner, B.J., & Chambers, D.A. (2020). A systematic review of empirical studies examining mechanisms of implementation in health. Implementation Science, 15(1). https://doi.org/10.1186/s13012-020-00983-3
  • Lo, C.K. (2020). Systematic reviews on flipped learning in various education contexts. In Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K. (eds) Systematic reviews in educational research. Springer. https://doi.org/10.1007/978-3-658-27602-7_8
  • Luchini, C., Veronese, N., Nottegar, A., Shin, J.I., Gentile, G., Granziol, U., Soysal, P., Alexinschi, O., Smith, L., & Solmi, M. (2021). Assessing the quality of studies in meta‐research: Review/guidelines on the most important quality assessment tools. Pharmaceutical Statistics, 20(1), 185–195. https://doi.org/10.1002/pst.2068
  • Luo, S., & Gan, Z. (2022). Validation of a measure of flipped English learning readiness and examination of its relationships with instructional practices, learning self-efficacy, and learning beliefs. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.846781
  • Mesquida, C., Warne, J., & Lakens, D. (2025). A scoping review of the transparency, reporting practices and methodological rigor of meta analyses published in Sports Medicine. https://doi.org/10.51224/srxiv.576
  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
  • Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L.A., & PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
  • Murphy, B.A., & Hall, J.A. (2024). How a strong measurement validity review can go astray: A look at Higgins et al. (2024) and recommendations for future measurement-focused reviews. Clinical Psychology Review, 114, 102506. https://doi.org/10.1016/j.cpr.2024.102506
  • Neuendorf, K.A. (2018). Content analysis and thematic analysis. Advanced Research Methods for Applied Psychology, 211–223. https://doi.org/10.4324/9781315517971-21
  • Ortega, A., Lopez-Briz, E., & Fraga-Fuentes, M.D. (2016). From qualitative reviews to umbrella reviews. In Biondi Zoccai, G. (Ed.), Umbrella reviews (pp. 21 41). Springer. https://doi.org/10.1007/978-3-319-25655-9_3
  • Ozenen, G. (2023). Self-assessment and learning outcome evaluation of interior architecture students using flipped versus traditional classroom education models. SAGE Open, 13(4). https://doi.org/10.1177/21582440231209891
  • Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., ... Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
  • Paré, G., Tate, M., Johnstone, D., & Kitsiou, S. (2016). Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews. European Journal of Information Systems, 25(6), 493–508. https://doi.org/10.1057/s41303-016-0020-3
  • Peters, M.D., Marnie, C., Tricco, A.C., Pollock, D., Munn, Z., Alexander, L., ... & Khalil, H. (2020). Updated methodological guidance for the conduct of scoping reviews. JBI evidence synthesis, 18(10), 2119-2126. https://doi.org/10.11124/jbies-20-00167
  • Pham, B., Klassen, T.P., Lawson, M.L., & Moher, D. (2005). Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. Journal of Clinical Epidemiology, 58(8), 769 776.e2. https://doi.org/10.1016/j.jclinepi.2004.08.021
  • Prevalla, B., & Uzunboylu, H. (2019). Flipped learning in engineering education. TEM Journal, 8(2), 446-450. https://doi.org/10.18421/tem82-46
  • Ramma, Y., Bholoa, A., Jawaheer, S., Gunness, S., Wah, H.T.Y.L.K., Gopee, A.K., & Authelsingh, D. (2021). Teaching and learning science in the 21st century: A study of critical thinking of learners and associated challenges. In J. Naidoo (Ed.), Teaching and learning in the 21st century (pp.139–156). https://doi.org/10.1163/9789004460386_009
  • Reinhold, F., Leuders, T., Loibl, K., Nückles, M., Beege, M., & Boelmann, J.M. (2024). Learning mechanisms explaining learning with digital tools in educational settings: A cognitive process framework. Educational Psychology Review, 36(1). https://doi.org/10.1007/s10648-024-09845-6
  • Robertson, W.H. (2022). The constructivist flipped classroom. The Journal of College Science Teaching, 52(2). https://doi.org/10.1080/0047231x.2022.12290644
  • Roehling, P.V. (2018). Flipping the college classroom: An evidence-based guide. Palgrave. https://doi.org/10.1007/978-3-319-69392-7
  • Rutkienė, A., Kaçar, I.G., Karakuş, E., Baltaci, H.S., Altun, M., Şahintaş, Z.A., Barendsen, R. M., Wierda, R., & García, B.C. (2022). The impact of flipped learning on students' engagement and satisfaction development: A cross-country action research study. Pedagogika, 147(3), 201-220. https://doi.org/10.15823/p.2022.147.12
  • Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage Publications.
  • Sancar-Tokmak, H., & Dagli, Z. (2025). A systematic review of theoretical foundations and learning effects in gamified flipped classroom research. Evaluation Review, 49(5), 880–913. https://doi.org/10.1177/0193841x251320438
  • Schmucker, C.M., Blümle, A., Schell, LK., Schwarzer, G., Oeller, P., Cabrera, L., von Elm, E., Briel, M., & Meerpohl, J.J. (2017). Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLOS ONE, 12(4), e0176210. https://doi.org/10.1371/journal.pone.0176210
  • Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D.A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008. https://doi.org/10.1136/bmj.j4008
  • Shih, H.J., & Huang, S.C. (2019). College students’ metacognitive strategy use in an EFL flipped classroom. Computer Assisted Language Learning, 33(7), 755 784. https://doi.org/10.1080/09588221.2019.1590420
  • Silver, C., & Lewins, A. (2014). Using software in qualitative research: A step-by-step guide. https://doi.org/10.4135/9781473906907
  • Silverajah, V.S.G., Wong, S.L., Govindaraj, A., Khambari, M.N.M., Rahmat, R.W.B.O.K., & Deni, A.R.M. (2022). A systematic review of self-regulated learning in flipped classrooms: Key findings, measurement methods, and potential directions. IEEE Access, 10(08), 20270. https://doi.org/10.1109/ACCESS.2022.3143857
  • Sterne, J.A.C., Sutton, A.J., Ioannidis, J.P.A., Terrin, N., Jones, D.R., Lau, J., Carpenter, J., Rücker, G., Harbord, R.M., Schmid, C.H., Tetzlaff, J., Deeks, J.J., Peters, J., Macaskill, P., Schwarzer, G., Duval, S., Altman, D.G., Moher, D., & Higgins, J.P.T. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343, d4002. https://doi.org/10.1136/bmj.d4002
  • Stern, C., & Kleijnen, J. (2020). Language bias in systematic reviews: You only get out what you put in. JBI Evidence Synthesis, 18(9), 1818–1819. https://doi.org/10.11124/jbies-20-00361
  • Taguma, M., Feron, E., & Lim, M.H. (2018). Future of education and skills 2030: Conceptual learning framework. OECD Publishing. Retrieved February 10, 2025, from https://www.oecd.org/education/2030-project/about/documents/
  • Templier, M., & Paré, G. (2018). Transparency in literature reviews: An assessment of reporting practices across review types and genres in top IS journals. European Journal of Information Systems, 27(5), 503–550. https://doi.org/10.1080/0960085x.2017.1398880
  • Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology, 12(1). https://doi.org/10.1186/1471-2288-12-181
  • Tricco, A.C., Lillie, E., Zarin, W., O'Brien, K.K., Colquhoun, H., Levac, D., ... & Straus, S.E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467-473. https://doi.org/10.7326/m18-0850
  • Urcola‐Pardo, F., Subiron‐Valera, A.B., Anton‐Solanas, I., Orkaizagirre‐Gomara, A., Torres‐Enamorado, D., & Gonzalez‐Sanz, J.D. (2024). Design and validation of the flipped‐learning assessment scale for undergraduate nursing education. Nursing Open, 11(12). https://doi.org/10.1002/nop2.70112
  • Walpole, S.C. (2019). Including papers in languages other than English in systematic reviews: Important, feasible, yet often omitted. Journal of Clinical Epidemiology, 111, 127–134. https://doi.org/10.1016/j.jclinepi.2019.03.004
  • Wang, Z., Brito, J., Tsapas, A., Griebeler, M., Alahdab, F., & Murad, M.H. (2015). Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study. Clinical Epidemiology, 243. https://doi.org/10.2147/clep.s78879
  • Whiting, P., Wolff, R., Mallett, S., Simera, I., & Savović, J. (2017). A proposed framework for developing quality assessment tools. Systematic Reviews, 6, 1 9. https://doi.org/10.1186/s13643-017-0604-6
  • Zeng, X., Zhang, Y., Kwong, J.S.W., Zhang, C., Li, S., Sun, F., Niu, Y., & Du, L. (2015). The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta‐analysis, and clinical practice guideline: A systematic review. Journal of Evidence-Based Medicine, 8(1), 2–10. https://doi.org/10.1111/jebm.12141
  • Zhang, H., Mao, R., Huang, H., Dai, Q., Zhou, X., Shen, H., & Rong, G. (2021). Processes, challenges and recommendations of gray literature review: An experience report. Information and Software Technology, 137, 106607. https://doi.org/10.1016/j.infsof.2021.106607
  • Zhu, L., Lian, Z., & Engström, M. (2020). Use of a flipped classroom in ophthalmology courses for nursing, dental and medical students: A quasi-experimental study using a mixed-methods approach. Nurse Education Today, 85, 104262. https://doi.org/10.1016/j.nedt.2019.104262

Methodological gaps and domain imbalances in flipped learning reviews: A critical umbrella review

Year 2026, Volume: 13 Issue: 1, 21 - 44, 02.01.2026
https://doi.org/10.21449/ijate.1692300

Abstract

This critical umbrella review (UR) examined methodological rigor and outcome assessment practices in flipped learning (FL) research syntheses. A systematic analysis of 41 reviews (2017-2023) spanning STEM, medical/health sciences, and social sciences identified significant methodological concerns and domain imbalances in reported outcomes. The analysis revealed widespread inconsistencies in primary source selection, with most reviews excluding grey literature. Inadequate quality assessment practices were evident, with over half lacking formal risk of bias evaluation. The analysis also found variable approaches to publication bias assessment and potential language bias, with most reviews limiting inclusion to English-language publications. The analysis further reveals a pronounced imbalance in outcome domains. Cognitive outcomes dominated, particularly academic achievement. The affective and interpersonal domains received substantially less attention despite their theoretical importance to FL pedagogy. The paucity of specified cognitive metrics in numerous reviews was a matter of particular concern, whilst the near absence of metric reporting for affective and interpersonal outcomes was even more concerning. In addition, the psychometric properties of instruments were rarely critically evaluated. Although reviews consistently identified active learning, self-paced instruction, and increased interaction as key mechanisms for FL effectiveness, many failed to provide substantive exploration of how these mechanisms operate within specific disciplinary contexts. This critical synthesis highlights the need for methodological improvements in FL reviews, including comprehensive literature sourcing, rigorous quality assessment, expanded domain focus beyond cognitive outcomes, and deeper mechanism exploration to strengthen the evidence base guiding educational practice.

References

  • Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336
  • Adams, R.J., Smart, P., & Huff, A.S. (2017). Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4), 432 454. https://doi.org/10.1111/ijmr.12102
  • Aeschbach, L.F., Perrig, S.A.C., Weder, L., Opwis, K., & Brühlmann, F. (2021). Transparency in measurement reporting: A systematic literature review of CHI PLAY. Proceedings of the ACM on Human-Computer Interaction, 5(CHI PLAY), 233. https://doi.org/10.1145/3474660
  • Aguinis, H., Ramani, R.S., & Alabduljader, N. (2020). Best-practice recommendations for producers, evaluators, and users of methodological literature reviews. Organizational Research Methods, 26(1), 46–76. https://doi.org/10.1177/1094428120943281
  • Alqahtani, T.M., Yusop, F.D., & Halili, S.H. (2023). Content validity of the constructivist learning in higher education settings (CLHES) scale in the context of the flipped classroom in higher education. Humanities and Social Sciences Communications, 10(1). https://doi.org/10.1057/s41599-023-01754-3
  • Aromataris, E., Fernandez, R., Godfrey, C.M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-based Healthcare, 13(3), 132–140. https://doi.org/10.1097/xeb.0000000000000055
  • Aromataris, E., Fernandez, R., Godfrey, C., Holly, C., Khalil, H., & Tungpunkom, P. (2020). Chapter 10: Umbrella reviews. JBI Reviewer’s Manual. https://doi.org/10.46658/jbirm-17-08
  • Becker, L.A., & Oxman, A.D. (2008). Overviews of reviews. Cochrane Handbook for Systematic Reviews of Interventions, 607 631. https://doi.org/10.1002/9780470712184.ch22
  • Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8-14. https://doi.org/10.1016/j.npls.2016.01.001
  • Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. International society for technology in education.
  • Benzies, K.M., Premji, S., Hayden, K.A., & Serrett, K. (2006). State‐of‐the‐evidence reviews: Advantages and challenges of including grey literature. Worldviews on Evidence-Based Nursing, 3(2), 55–61. https://doi.org/10.1111/j.1741-6787.2006.00051.x
  • Bialik, M., Fadel, C., Trilling, B., Nilsson, P., & Groff, J. (2015). Skills for the 21st century: What should students learn. Centre for Curriculum Redesign.
  • Biesta, G. (2010). Good education in an age of measurement: Ethics, politics, democracy. Paradigm Publishers.
  • Birgili, B., Seggie, F.N., & Oğuz, E. (2021). The trends and outcomes of flipped learning research between 2012 and 2018: A descriptive content analysis. Journal of Computers in Education, 8(3), 365-394. https://doi.org/10.1007/s40692-021-00183-y
  • Bishop, J., & Verleger, M. (2013). Testing the flipped classroom with model-eliciting activities and video lectures in a mid-level undergraduate engineering course. In 2013 IEEE Frontiers in Education Conference (FIE) (pp. 161–163). https://doi.org/10.1109/fie.2013.6684807
  • Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. David McKay.
  • Borenstein, M., Hedges, L.V., Higgins, J.P., & Rothstein, H.R. (2009). Introduction to meta-analysis. John Wiley & Sons. https://doi.org/10.1002/9780470743386
  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
  • Campbell, M., Katikireddi, S.V., Sowden, A., & Thomson, H. (2019). Lack of transparency in reporting narrative synthesis of quantitative data: A methodological assessment of systematic reviews. Journal of Clinical Epidemiology, 105,1 9. https://doi.org/10.1016/j.jclinepi.2018.08.019
  • Care, E. (2017). 21st century skills: From theory to action. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills: Research and applications (pp. 3–17). Springer.
  • Castro-Gil, R., & Correa, D. (2021). Transparency in previous literature reviews about blended learning in higher education. Education and Information Technologies, 26(3), 3399 3426. https://doi.org/10.1007/s10639-020-10406-x
  • Chen, H.J., Chen, C.H., & Wu, W.C.V. (2025). Effects of flipped learning on language learning outcomes: A meta analysis investigating moderators. SAGE Open, 15(2). https://doi.org/10.1177/21582440251331298
  • Clark, L.A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412 1427. https://doi.org/10.1037/pas0000626
  • Coman, L., & Richardson, J. (2006). Relationship between self-report and performance measures of function: A systematic review. Canadian Journal on Aging / La Revue Canadienne Du Vieillissement, 25(3), 253–270. https://doi.org/10.1353/cja.2007.0001
  • Creswell, J.W., & Plano Clark, V.L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE Publications.
  • Easterbrook, P.J., Gopalan, R., Berlin, J.A., & Matthews, D.R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867 872. https://doi.org/10.1016/0140 6736(91)90201-y
  • Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997). Language bias in randomised controlled trials published in English and German. The Lancet, 350(9074), 326-329. https://doi.org/10.1016/S0140-6736(97)02419-7
  • Ekinci, F., Bektaş, O., Karaca, M., & Yiğit, K.N. (2023). The flipped learning perception scale: A validity and reliability study. Education and Information Technologies, 28(11), 14139–14166. https://doi.org/10.1007/s10639-023-11731-7
  • Francis, N., Morgan, A., Holm, S., Davey, R., Bodger, O., & Dudley, E. (2020). Adopting a flipped classroom approach for teaching molar calculations to biochemistry and genetics students. Biochemistry and Molecular Biology Education, 48(3), 220 226. https://doi.org/10.1002/bmb.21328
  • Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502 1505. https://doi.org/10.1126/science.1255484
  • Fry, H., Ketteridge, S., & Marshall, S. (2009). A handbook for teaching and learning in higher education: Enhancing academic practice. Routledge. https://doi.org/10.4324/9781315763088
  • Grégoire, G., Derderian, F., & Le Lorier, J. (1995). Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias? Journal of Clinical Epidemiology, 48(1), 159–163. https://doi.org/10.1016/0895-4356(94)00098-b
  • Goedhart, N.S., Westrhenen, N.B., Möser, C., & Zweekhorst, M. (2019). The flipped classroom: Supporting a diverse group of students in their learning. Learning Environments Research, 22(2), 297-310.
  • Grant, M.J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91 108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
  • Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: Audit of primary sources. British Medical Journal, 331(7524), 1064–1065. https://doi.org/10.1136/bmj.38636.593461.68
  • Harrison, R., Jones, B., Gardner, P., & Lawton, R. (2021). Quality assessment with diverse studies (QuADS): An appraisal tool for methodological and reporting quality in systematic reviews of mixed- or multi method studies. BMC Health Services Research, 21(1). https://doi.org/10.1186/s12913-021-06122-y
  • Hartling, L., Vandermeer, B., & Fernandes, R.M. (2014). Systematic reviews, overviews of reviews and comparative effectiveness reviews: A discussion of approaches to knowledge synthesis. Evidence-Based Child Health: A Cochrane Review Journal, 9(2), 486–494. https://doi.org/10.1002/ebch.1968
  • Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D.M., & Vandermeer, B. (2017). Grey literature in systematic reviews: A cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child relevant reviews. BMC Medical Research Methodology, 17(1). https://doi.org/10.1186/s12874-017-0347-z
  • Helbach, J., Pieper, D., Mathes, T., Rombey, T., Zeeb, H., Allers, K., & Hoffmann, F. (2022). Restrictions and their reporting in systematic reviews of effectiveness: An observational study. BMC Medical Research Methodology, 22(1). https://doi.org/10.1186/s12874-022-01710-w
  • Hew, K.F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., & Thankrit, K. (2021). On the use of flipped classroom across various disciplines: Insights from a second-order meta-analysis. Australasian Journal of Educational Technology, 37(2), 132-151.
  • Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., & Welch, V.A. (Eds.). (2023). Cochrane handbook for systematic reviews of interventions (Version 6.4). Cochrane. www.training.cochrane.org/handbook
  • Holly, C., Salmond, S., & Saimbert, M. (Eds.). (2017). Comprehensive systematic review for advanced practice nursing. https://doi.org/10.1891/9780826131867
  • Hopewell, S., McDonald, S., Clarke, M.J., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database of Systematic Reviews, (2), MR000010. https://doi.org/10.1002/14651858.MR000010.pub3
  • Huang, H.L., Chou, C.P., Leu, S., You, H.L., Tiao, M.M., & Chen, C.H. (2020). Effects of a quasi-experimental study of using flipped classroom approach to teach evidence-based medicine to medical technology students. BMC Medical Education, 20(1), https://doi.org/10.1186/s12909-020-1946-7
  • Joseph, J., Firmin, S., Oseni, T., & Stranieri, A. (2023). Decoding employee ambidexterity: Understanding drivers, constraints, and performance implications for thriving in the evolving work landscapes A scoping review. Heliyon, 9(12), e22493. https://doi.org/10.1016/j.heliyon.2023.e22493
  • Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002). Direction and impact of language bias in meta-analyses of controlled trials: Empirical study. International Journal of Epidemiology, 31(1), 115-123. https://doi.org/10.1093/ije/31.1.115
  • Kager, K., Kalinowski, E., Jurczok, A., & Vock, M. (2024). A systematic review of transparency in lesson study research: How do we report on the observation and reflection stages? Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1322624
  • Kamei, F., Pinto, G., Wiese, I., Ribeiro, M., & Soares, S. (2021). What evidence we would miss if we do not use grey literature? Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 1 11. https://doi.org/10.1145/3475716.3475777
  • Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022). Fail, flip, fix, and feed – Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.956416
  • Krathwohl, D.R., Bloom, B.S., & Masia, B.B. (1964). Taxonomy of educational objectives: The classification of educational goals. Handbook II: Affective domain. David McKay.
  • Kreitchmann, R.S., Abad, F.J., Ponsoda, V., Nieto, M.D., & Morillo, D. (2019). Controlling for response biases in self-report scales: Forced-choice vs. psychometric modeling of Likert items. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.02309
  • Kung, J., Chiappelli, F., Cajulis, O.O., Avezova, R., Kossan, G., Chew, L., & Maida, C.A. (2010). From systematic reviews to clinical recommendations for evidence-based health care: validation of revised assessment of multiple systematic reviews (R-AMSTAR) for grading of clinical relevance. The Open Dentistry Journal, 4(1), 84 91. https://doi.org/10.2174/1874210601004010084
  • Kyngäs, H., Kääriäinen, M., & Elo, S. (2020). The trustworthiness of content analysis. In Kyngäs, H., Mikkonen, K., Kääriäinen, M. (Eds.), The application of content analysis in nursing science research (pp. 41– 48). Springer. https://doi.org/10.1007/978-3-030-30199-6_5
  • Larsen, J.N., Maxwell, K.M.D., & Khalil, M. (2025). Exploring the link between motivational regulation and learning design with learning analytics. Journal of Learning Analytics, 12(1), 215–237. https://doi.org/10.18608/jla.2025.8469
  • Leatherman, J.L., & Cleveland, L.M. (2019). Student exam performance in flipped classroom sections is similar to that in active learning sections, and satisfaction with the flipped classroom hinges on attitudes toward learning from videos. Journal of Biological Education, 54(3), 328–344. https://doi.org/10.1080/00219266.2019.1575266
  • Lewis, C.C., Boyd, M.R., Walsh-Bailey, C., Lyon, A.R., Beidas, R., Mittman, B., Aarons, G.A., Weiner, B.J., & Chambers, D.A. (2020). A systematic review of empirical studies examining mechanisms of implementation in health. Implementation Science, 15(1). https://doi.org/10.1186/s13012-020-00983-3
  • Lo, C.K. (2020). Systematic reviews on flipped learning in various education contexts. In Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K. (eds) Systematic reviews in educational research. Springer. https://doi.org/10.1007/978-3-658-27602-7_8
  • Luchini, C., Veronese, N., Nottegar, A., Shin, J.I., Gentile, G., Granziol, U., Soysal, P., Alexinschi, O., Smith, L., & Solmi, M. (2021). Assessing the quality of studies in meta‐research: Review/guidelines on the most important quality assessment tools. Pharmaceutical Statistics, 20(1), 185–195. https://doi.org/10.1002/pst.2068
  • Luo, S., & Gan, Z. (2022). Validation of a measure of flipped English learning readiness and examination of its relationships with instructional practices, learning self-efficacy, and learning beliefs. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.846781
  • Mesquida, C., Warne, J., & Lakens, D. (2025). A scoping review of the transparency, reporting practices and methodological rigor of meta analyses published in Sports Medicine. https://doi.org/10.51224/srxiv.576
  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
  • Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L.A., & PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1
  • Murphy, B.A., & Hall, J.A. (2024). How a strong measurement validity review can go astray: A look at Higgins et al. (2024) and recommendations for future measurement-focused reviews. Clinical Psychology Review, 114, 102506. https://doi.org/10.1016/j.cpr.2024.102506
  • Neuendorf, K.A. (2018). Content analysis and thematic analysis. Advanced Research Methods for Applied Psychology, 211–223. https://doi.org/10.4324/9781315517971-21
  • Ortega, A., Lopez-Briz, E., & Fraga-Fuentes, M.D. (2016). From qualitative reviews to umbrella reviews. In Biondi Zoccai, G. (Ed.), Umbrella reviews (pp. 21 41). Springer. https://doi.org/10.1007/978-3-319-25655-9_3
  • Ozenen, G. (2023). Self-assessment and learning outcome evaluation of interior architecture students using flipped versus traditional classroom education models. SAGE Open, 13(4). https://doi.org/10.1177/21582440231209891
  • Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., ... Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
  • Paré, G., Tate, M., Johnstone, D., & Kitsiou, S. (2016). Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews. European Journal of Information Systems, 25(6), 493–508. https://doi.org/10.1057/s41303-016-0020-3
  • Peters, M.D., Marnie, C., Tricco, A.C., Pollock, D., Munn, Z., Alexander, L., ... & Khalil, H. (2020). Updated methodological guidance for the conduct of scoping reviews. JBI evidence synthesis, 18(10), 2119-2126. https://doi.org/10.11124/jbies-20-00167
  • Pham, B., Klassen, T.P., Lawson, M.L., & Moher, D. (2005). Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. Journal of Clinical Epidemiology, 58(8), 769 776.e2. https://doi.org/10.1016/j.jclinepi.2004.08.021
  • Prevalla, B., & Uzunboylu, H. (2019). Flipped learning in engineering education. TEM Journal, 8(2), 446-450. https://doi.org/10.18421/tem82-46
  • Ramma, Y., Bholoa, A., Jawaheer, S., Gunness, S., Wah, H.T.Y.L.K., Gopee, A.K., & Authelsingh, D. (2021). Teaching and learning science in the 21st century: A study of critical thinking of learners and associated challenges. In J. Naidoo (Ed.), Teaching and learning in the 21st century (pp.139–156). https://doi.org/10.1163/9789004460386_009
  • Reinhold, F., Leuders, T., Loibl, K., Nückles, M., Beege, M., & Boelmann, J.M. (2024). Learning mechanisms explaining learning with digital tools in educational settings: A cognitive process framework. Educational Psychology Review, 36(1). https://doi.org/10.1007/s10648-024-09845-6
  • Robertson, W.H. (2022). The constructivist flipped classroom. The Journal of College Science Teaching, 52(2). https://doi.org/10.1080/0047231x.2022.12290644
  • Roehling, P.V. (2018). Flipping the college classroom: An evidence-based guide. Palgrave. https://doi.org/10.1007/978-3-319-69392-7
  • Rutkienė, A., Kaçar, I.G., Karakuş, E., Baltaci, H.S., Altun, M., Şahintaş, Z.A., Barendsen, R. M., Wierda, R., & García, B.C. (2022). The impact of flipped learning on students' engagement and satisfaction development: A cross-country action research study. Pedagogika, 147(3), 201-220. https://doi.org/10.15823/p.2022.147.12
  • Saldaña, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage Publications.
  • Sancar-Tokmak, H., & Dagli, Z. (2025). A systematic review of theoretical foundations and learning effects in gamified flipped classroom research. Evaluation Review, 49(5), 880–913. https://doi.org/10.1177/0193841x251320438
  • Schmucker, C.M., Blümle, A., Schell, LK., Schwarzer, G., Oeller, P., Cabrera, L., von Elm, E., Briel, M., & Meerpohl, J.J. (2017). Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLOS ONE, 12(4), e0176210. https://doi.org/10.1371/journal.pone.0176210
  • Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D.A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008. https://doi.org/10.1136/bmj.j4008
  • Shih, H.J., & Huang, S.C. (2019). College students’ metacognitive strategy use in an EFL flipped classroom. Computer Assisted Language Learning, 33(7), 755 784. https://doi.org/10.1080/09588221.2019.1590420
  • Silver, C., & Lewins, A. (2014). Using software in qualitative research: A step-by-step guide. https://doi.org/10.4135/9781473906907
  • Silverajah, V.S.G., Wong, S.L., Govindaraj, A., Khambari, M.N.M., Rahmat, R.W.B.O.K., & Deni, A.R.M. (2022). A systematic review of self-regulated learning in flipped classrooms: Key findings, measurement methods, and potential directions. IEEE Access, 10(08), 20270. https://doi.org/10.1109/ACCESS.2022.3143857
  • Sterne, J.A.C., Sutton, A.J., Ioannidis, J.P.A., Terrin, N., Jones, D.R., Lau, J., Carpenter, J., Rücker, G., Harbord, R.M., Schmid, C.H., Tetzlaff, J., Deeks, J.J., Peters, J., Macaskill, P., Schwarzer, G., Duval, S., Altman, D.G., Moher, D., & Higgins, J.P.T. (2011). Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ, 343, d4002. https://doi.org/10.1136/bmj.d4002
  • Stern, C., & Kleijnen, J. (2020). Language bias in systematic reviews: You only get out what you put in. JBI Evidence Synthesis, 18(9), 1818–1819. https://doi.org/10.11124/jbies-20-00361
  • Taguma, M., Feron, E., & Lim, M.H. (2018). Future of education and skills 2030: Conceptual learning framework. OECD Publishing. Retrieved February 10, 2025, from https://www.oecd.org/education/2030-project/about/documents/
  • Templier, M., & Paré, G. (2018). Transparency in literature reviews: An assessment of reporting practices across review types and genres in top IS journals. European Journal of Information Systems, 27(5), 503–550. https://doi.org/10.1080/0960085x.2017.1398880
  • Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology, 12(1). https://doi.org/10.1186/1471-2288-12-181
  • Tricco, A.C., Lillie, E., Zarin, W., O'Brien, K.K., Colquhoun, H., Levac, D., ... & Straus, S.E. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467-473. https://doi.org/10.7326/m18-0850
  • Urcola‐Pardo, F., Subiron‐Valera, A.B., Anton‐Solanas, I., Orkaizagirre‐Gomara, A., Torres‐Enamorado, D., & Gonzalez‐Sanz, J.D. (2024). Design and validation of the flipped‐learning assessment scale for undergraduate nursing education. Nursing Open, 11(12). https://doi.org/10.1002/nop2.70112
  • Walpole, S.C. (2019). Including papers in languages other than English in systematic reviews: Important, feasible, yet often omitted. Journal of Clinical Epidemiology, 111, 127–134. https://doi.org/10.1016/j.jclinepi.2019.03.004
  • Wang, Z., Brito, J., Tsapas, A., Griebeler, M., Alahdab, F., & Murad, M.H. (2015). Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study. Clinical Epidemiology, 243. https://doi.org/10.2147/clep.s78879
  • Whiting, P., Wolff, R., Mallett, S., Simera, I., & Savović, J. (2017). A proposed framework for developing quality assessment tools. Systematic Reviews, 6, 1 9. https://doi.org/10.1186/s13643-017-0604-6
  • Zeng, X., Zhang, Y., Kwong, J.S.W., Zhang, C., Li, S., Sun, F., Niu, Y., & Du, L. (2015). The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta‐analysis, and clinical practice guideline: A systematic review. Journal of Evidence-Based Medicine, 8(1), 2–10. https://doi.org/10.1111/jebm.12141
  • Zhang, H., Mao, R., Huang, H., Dai, Q., Zhou, X., Shen, H., & Rong, G. (2021). Processes, challenges and recommendations of gray literature review: An experience report. Information and Software Technology, 137, 106607. https://doi.org/10.1016/j.infsof.2021.106607
  • Zhu, L., Lian, Z., & Engström, M. (2020). Use of a flipped classroom in ophthalmology courses for nursing, dental and medical students: A quasi-experimental study using a mixed-methods approach. Nurse Education Today, 85, 104262. https://doi.org/10.1016/j.nedt.2019.104262
There are 96 citations in total.

Details

Primary Language English
Subjects Measurement and Evaluation in Education (Other)
Journal Section Review
Authors

Abdullah Arslan 0000-0002-3979-6371

Submission Date May 5, 2025
Acceptance Date September 29, 2025
Publication Date January 2, 2026
Published in Issue Year 2026 Volume: 13 Issue: 1

Cite

APA Arslan, A. (2026). Methodological gaps and domain imbalances in flipped learning reviews: A critical umbrella review. International Journal of Assessment Tools in Education, 13(1), 21-44. https://doi.org/10.21449/ijate.1692300

23823             23825             23824