A Multidimensional Diagnostic Assessment of Preservice Mathematics Teachers’ Conditional Probability and Bayesian Reasoning

Abstract

This study develops and validates a multidimensional diagnostic test to examine preservice mathematics teachers’ strengths and weaknesses regarding conditional probability and the Bayes rule. We estimated preservice teachers’ mastery of four fundamental skills (i.e., attributes): understanding and applying, i. basic probability concepts, ii. conditional probability, iii. the Bayes rule, and iv. dealing with difficulties and biases. We administered the test to a national sample of 750 preservice mathematics teachers and analysed their responses using the log-linear diagnostic classification model. The analysis revealed that 22.7% of preservice teachers mastered all four attributes, while 21.8% did not master any attributes. The average mastery percentages of the four skills were 74.3, 68.3, 47.8, and 53.2, respectively. We also report the diagnostic quality of the individual test items, mastery classifications, and the relationships among the assessed skills. Implications of the findings and further research suggestions are discussed.

Keywords

• conditional probability reasoning
• diagnostic assessment
• diagnostic classification models
• probability teaching and learning
• The Bayes rule

References

1. Albert, J. (1995). Teaching inference about proportions using Bayes and discrete models. Journal of Statistics Education, 3(3), 1–16. https://doi.org/10.1080/10691898.1995.11910494
2. Arican, M., & Kuzu, O. (2020). Diagnosing preservice teachers’ understanding of statistics and probability: Developing a test for cognitive assessment. International Journal of Science and Mathematics Education, 18(4), 771–790. https://doi.org/10.1007/s10763-019-09985-0
3. Batanero, C., Burrill, G., & Reading, C. (2011). Teaching statistics in school mathematics—challenges for teaching and teacher education: A joint ICMI/IASE study. Dordrecht: Springer.
4. Batanero, C., Contreras, J. M., Díaz, C., & Cañadas, G. R. (2014). Preparing teachers to teach conditional probability: A didactic situation based on the Monty Hall problem. In T. Wassong, D. Frischemeier, P. Fischer, R. Hochmuth, & P. Bender (Eds.), Mit werkzeugen mathematik und stochastik lernen–Using tools for learning mathematics and statistics (pp. 363–376): Springer. https://doi.org/10.1007/978-3-658-03104-6_26
5. Biehler, R., & Pratt, D. (Eds.). (2012). Probability in reasoning about data and risk [Special issue]. ZDM—The International Journal on Mathematics Education, 44(7), 819–952. https://doi.org/10.1007/s11858-012-0468-0
6. Borovcnik, M. & Kapadia, R. (2014). A historical and philosophical perspective on probability. In E. J. Chernoff and B. Sriraman (Eds.), Probabilistic thinking: Presenting plural perspectives (pp. 7–34). Springer, Dordrecht.
7. Borovcnik M. & Kapadia, R. (2018). Reasoning with risk: Teaching probability and risk as twin concepts. In C. Batanero, E. J. Chernoff, J. Engel, H. Lee, and E. Sánchez (Eds.), Research on teaching and learning probability: Advances in probability education research (pp. 3–22). Springer, New York.
8. Böcherer-Linder, K., Eichler, A., Vogel, M. (2018). Visualising conditional probabilities-three perspectives on unit squares and tree diagrams. In C. Batanero & E. Chernoff (Eds.), Teaching and learning stochastics. ICME-13 Monographs. Springer, Cham. https://doi.org/10.1007/978-3-319-72871-1_5

9. Böcherer-Linder, K., & Eichler, A. (2019). How to improve performance in Bayesian inference tasks: a comparison of five visualizations. Frontiers in Psychology, 10, 267. https://doi.org/10.3389/fpsyg.2019.00267
10. Bradshaw, L., Izsak, A., Templin, J., & Jacobson, E. (2014). Diagnosing teachers’ understandings of rational numbers: Building a multidimensional test within the diagnostic classification framework. Educational Measurement: Issues and Practice, 33(1), 2–14. https://doi.org/10.1111/emip.12020
11. Bradshaw, L., & Templin, J. (2014). Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions. Psychometrika, 79(3), 403–425. https://doi.org/10.1007/s11336-013-9350-4
12. Brase, G. L. (2008). Pictorial representations in statistical reasoning. Applied Cognitive Psychology, 23, 369–381.
13. Brase, G. L. (2013). The power of representation and interpretation: Doubling statistical reasoning performance with icons and frequentist interpretations of ambiguous numbers. Journal of Cognitive Psychology, 26, 81–97.
14. Brase, G. L., & Hill, W. T. (2017). Adding up to good Bayesian reasoning: Problem format manipulations and individual skill differences. Journal of Experimental Psychology: General, 146(4), 577–591. https://doi.org/10.1037/xge0000280
15. Brousseau, G. (2006). Theory of didactical situations in mathematics: Didactique des mathématiques, 1970–1990 (Vol. 19). Springer Science Business Media.
16. Budgett, S., & Pfannkuch, M. (2019). Visualizing chance: tackling conditional probability misconceptions. In G. Burrill, & D. Ben-Zvi, D. (Eds.), Topics and trends in current statistics education research, ICME-13 Monographs, (pp. 363–376). Springer, Cham. https://doi.org/10.1007/978-3-030-03472-6_1
17. Carter, T. A., & Capraro. R. M. (2005). Stochastic misconceptions of pre-service teachers. Academic Exchange Quarterly, 9(3), 105–111.
18. Chernoff, E., & Sriraman, B. (2014). Introduction. In E. Chernoff & B. Sriraman (Eds.), Probabilistic thinking: Presenting plural perspectives (pp. xv–xvii). New York, NY: Springer.
19. Chernoff, E. J., & Zazkis, R. (2011). From personal to conventional probabilities: From sample set to sample space. Educational Studies in Mathematics, 77(1), 15–33. doi:10.1007/s10649010-9288-8
20. Chow, A. F., & Van Haneghan, J. P. (2016). Transfer of solutions to conditional-probability problems: Effects of example problem format, solution format, and problem context. Educational Studies in Mathematics, 93(1), 67–85.
21. Costello, F., & Watts, P. (2016). People’s conditional probability judgments follow probability theory (plus noise). Cognitive Psychology, 89, 106–133.
22. Cui, L., Lo, S., & Liu, Z. (2023). The use of visualizations to improve Bayesian reasoning: A literature review. Vision, 7(1), 17–32. doi: 10.3390/vision7010017
23. D’Amelio, A. (2009). Undergraduate student difficulties with independent and mutually exclusive events concepts. The Mathematics Enthusiast, 6, 47–56.
24. de la Torre, J. (2008). An empirically-based method of Q‐matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45(4), 343–362. https://doi.org/10.1111/j.1745-3984.2008.00069.x
25. Díaz, C., & Batanero, C. (2009). University students’ knowledge and biases in conditional probability reasoning. International Electronic Journal of Mathematics Education, 4(3), 131–162.
26. Díaz, C., Contreras, J. M., Arteaga, P., & Batanero, C. (2012, July). Prospective teachers’ difficulties in solving Bayes problems. In the 36th Conference of the International Group for the Psychology of Mathematics Education (pp. 230–239). Taipei, Taiwan.
27. Díaz, C., & de la Fuente, I. (2007). Assessing students’ difficulties with conditional probability and Bayesian reasoning. International Electronic Journal of Mathematics Education, 2(3), 128–148.
28. Falk, R. (1986). Conditional probabilities: insights and difficulties. In R. Davidson & J. Swift (Eds.), Proceedings of the Second International Conference on Teaching Statistics (pp. 292–297). Victoria, Canada: International Statistical Institute.
29. Fischbein, E., & Schnarch, D. (1997). Brief report: The evolution with age of probabilistic, intuitively based misconceptions. Journal for Research in Mathematics Education, 28(1), 96–105. https://doi.org/10.2307/749665
30. Franklin, C., Bargagliotti, A. E., Case, C. A., Kader, G., Scheaffer, R., & Spangler, D. (2015). The statistical education of teachers. Alexandria, VA: American Statistical Association. http://www.amstat.org/education/SET/SET.pdf
31. Gigerenzer, G. (2002). Calculated risks. New York: Simon and Schuster.
32. Gould, R., & Ryan, C. (2014). Introductory statistics: Exploring the world through data. Boston, MA: Pearson.
33. Groth, R. E. (2010). Teachers’ construction of learning environments for conditional probability and independence. International Electronic Journal of Mathematics Education, 5(1), 32–35.
34. Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191–210. https://doi.org/10.1007/s11336-008-9089-5
35. Hoffrage, U., Gigerenzer, G., Krauss, S., & Martignon, L. (2002). Representation facilitates reasoning: What natural frequencies are and what they are not. Cognition, 84, 343–352.
36. Hoffrage, U., Krauss, S., Martignon, L., & Gigerenzer, G. (2015). Natural frequencies improve Bayesian reasoning in simple and complex inference tasks. Frontiers in Psychology, 6, 1473.
37. Hoffrage, U., Lindsey, S., Hertwig, R., & Gigerenzer, G. (2000). Communicating statistical information. Science, 290(5500), 2261–2261.
38. Huerta, M. P. (2009). On conditional probability problem solving research – structures and contexts. International Electronic Journal of Mathematics Education, 4(3), 163–194.
39. Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3(3), 430–454. https://doi.org/10.1016/0010-0285(72)90016-3
40. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. https://doi.org/10.1037/h0034747
41. Konold, C. (1989). Informal conceptions of probability. Cognition and Instruction, 6(1), 59–98.
42. Langrall, C. W., Makar, K., Nilsson, P., & Shaughnessy, J. M. (2017). Teaching and learning probability and statistics: An integrated perspective. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 490–525). Reston, VA: NCTM.
43. Manage, A. B. W., & Scariano, S. M. (2010). A classroom note on student misconceptions regarding probabilistic independence vs. mutual exclusivity. Mathematics and Computer Education, 44(1), 14–21.
44. McDowell, M., & Jacobs, P. (2017). Meta-analysis of the effect of natural frequencies on Bayesian reasoning. Psychological Bulletin, 143(12), 1273–1312. https://doi.org/10.1037/bul0000126
45. Molnar, A. R. (2015). High school mathematics teachers’ knowledge and views of conditional probability (Unpublished doctoral dissertation). University of Georgia, Athens.
46. Molnar, A. R. & Edalgo, S. (2017). Textbook formations of independence. In A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, & S. Brown (Eds.), Proceedings of the 20th Annual Conference on Research in Undergraduate Mathematics Education. San Diego, California. Available online: http://sigmaa.maa.org/rume/RUME20.pdf
47. Muthen, L. K., & Muthen, B. O. (2011). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthen & Muthen.
48. Nabbout-Cheiban, M. (2017). Intuitive thinking and misconceptions of independent events: A case study of the US and French pre-service teachers. International Journal of Research in Undergraduate Mathematics Education, 3, 255–282.
49. Odeja, A. M. (1996). Contestos, Representaciones y la idea de Probabilidad Condicional, Investigaciones en Matematica Educativa, pp. 291-310.(Mexico:Grupo Editorial Iberoamerica).
50. R Core Team. (2022). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. http://www.r-project.org/.
51. Rubel, L. H. (2007). Middle school and high school students’ probabilistic reasoning on coin tasks. Journal for Research in Mathematics Education, 38(5), 531–556.
52. Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219–262.
53. Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford.
54. Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75(2), 243–248. https://doi.org/10.1007/s11336-009-9135-y
55. Shaughnessy, J. M. (1992). Research in probability and statistics: Reflections and directions. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics (pp. 465–494). Macmillan Publishing.
56. Tarr, J. E. & Jones, G. A. (1997). A framework for assessing middle school students’ thinking in conditional probability and independence. Mathematics Education Research Journal, 9, 39–59. https://doi.org/10.1007/BF03217301
57. Tarr, J. E., & Lannin, J. K. (2005). How can teachers build notions of conditional probability and independence? In G. A Jones (Ed.), Exploring probability in school (pp. 215–238). Boston, MA: Springer.
58. Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10(1), 55–73.
59. Tversky, A. & Kahneman, D. (1982a). Judgements of and by representativeness. In D. Kahneman, P. Slovic, and A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 84–98). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511809477.007
60. Tversky, A., & Kahneman, D. (1982b). Evidential impact of base rates. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 153–160). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511809477.011
61. Vargas, F., Benincasa, T., Cian, G., & Martignon, L. (2019). Fostering probabilistic reasoning away from fallacies: Natural information formats and interaction between school levels. International Electronic Journal of Mathematics Education, 14(2), 303–330. https://doi.org/10.29333/iejme/5716
62. Werner, C., & Schermelleh-Engel, K. (2010). Deciding between competing models: Chi-square difference tests. In Introduction to Structural Equation Modeling with LISREL (pp. 1–3). Frankfurt, Germany: Goethe University.
63. Zwanch, K. (2019). A preliminary genetic decomposition of probabilistic independence. The Mathematics Educator, 28(1), 3–26.