A Meta-Analysis of the Effects of Realistic Mathematics Education- based Teaching on Mathematical Achievement of Students in Turkey

Sedat TURGUT * 1 1 Bartın University, Turkey, sedatturgut42@gmail.com * Corresponding Author: sedatturgut42@gmail.com Article Info Abstract The aim of the current study is to determine the effect of realistic mathematics education-based teaching on students' mathematics achievement. For this purpose, a meta-analysis method, which allows combining the results of a series of studies on a subject, was used in the study. A total of 40 scientific publications, 27 thesis and 13 articles, which are suitable for the research problem, were included in the sample of the study. The publications conducted on mathematics achievement in 2020 and earlier in Turkey were used in the study. Process effectiveness method of meta-analysis was employed in the analysis of data and Hedges’s g was used in the calculation of effect size of the study. In determination of the publication bias of the studies included in the meta-analysis, the funnel plot and Rosenthal’s Fail-Safe N-FSN statistics were examined together. In order to determine whether the distribution of the effect sizes is homogenous or not, the results of Q statistic were investigated. As a result, the effect sizes are homogeneously distributed. Therefore, fixed effect model was used. As stated in the fixed effects model, the overall effect size value is 0.760 with a 0.041 level of standard error. As a result of the study, the effect of teaching activities based on realistic mathematics education on mathematical achievement is at a positive medium level. Received: 22 December 2020 Accepted: 25 March 2021


Introduction
Considering the fact that mathematics is a part of many areas including daily and academic lives, and careers of individuals, it can be stated that those who are good at mathematics will be successful at life and get opportunities in life (National Council of Teachers of Mathematics [NCTM], 2000; Organization for Economic Co-operation and Development [OECD], 2013). Nevertheless, it is well known that many individuals show the attitude that mathematics is not their cup of tea (Di Martino & Zan, 2011). One of the reasons why individuals have such an attitude may be related to how they learn mathematics.

302
of the students. Mathematics teaching should be organized as a rediscovering process in a manner that resembles the experience of the process of how mathematicians discovered mathematics (Freudenthal, 1991).

Mathematization
Mathematics is an activity of looking for and solving problems and organizing the solution of a problem. This activity may be a real problem that needs to be organized and solved in accordance with mathematical patterns (Freudenthal, 1971). This organizational activity is called mathematization (Gravemeijer, 1994;Treffers, 1991). Mathematization is a key process in mathematics teaching since dealing with mathematics teaches students to deal with daily life situations with a mathematical approach. When the students deal with mathematical knowledge with a mathematical perspective, they will have a true understanding of concepts and the implementation of these concepts. According to RME, the students need to reach mathematical knowledge by discovering through experiences (Gravemeijer & Doorman, 1999). Treffers (1987) takes on the mathematization in two processes, namely, horizontal process and vertical process. The students use mathematical tools in organizing problems regarding real-life situations in the horizontal mathematization process (Gravemeijer & Doorman, 1999;Van den Heuvel-Panhuizen, 2003; Van den Heuvel-Panhuizen & Drijvers, 2014). The horizontal mathematization process enables students to reach mathematical symbols through their real-life situations (Freudenthal, 1991). Expressing a real-life problem in a mathematical manner is a product of the horizontal mathematization process. On the other hand, the vertical mathematization process is formulizing mathematics in various ways through mathematical rules and reorganizing the mathematical system (Van den Heuvel-Panhuizen, 2003; Van den Heuvel-Panhuizen & Drijvers, 2014). Transforming a reallife problem into a mathematical problem is a product of the vertical mathematization process. Since abstract mathematical symbols are used in this process, it will occur more often in a classroom environment (Gravemeijer & Terwel, 2000). In the vertical mathematization process, the students can make mathematical formulizations of relationships, make explanations with various examples, and reach conclusions.

304
they may acquire ideas that enable them to develop strategies and reach a higher level of comprehension.
Guidance principle: The teachers have a proactive role in the learning process of the students.
Teachers need to consider the aforementioned principles while preparing teaching activities based on RME. In RME, mathematical knowledge must be constructed or reconstructed by the student. Under no circumstance, mathematical knowledge is readily available and transferred in a top-down manner. Even in a perfect lesson, the mathematical knowledge offered to students can only become meaningful through actively reconstructing the knowledge by every student. Students must re-discover mathematics by starting from fundamental experiences under the appropriate guidance (Freudenthal, 1971). Teaching must start with meaningful real-life problems rather than rules and abstract concepts. The role of teaching must not be directly conveying mathematical knowledge; but, guiding the students and expose their theoretical knowledge (Gravemeijer & Doorman, 1999;Van den Heuvel-Panhuizen, 2001, 2003.

The Present Study
When the literature is reviewed, a great deal of research can be found on the effect of activities conducted based on RME on mathematics teaching and learning on an international level (Fauzan, 2002;Gravemeijer & Doorman, 1999;Le, 2006;Sembiring, Hadi, & Dolk, 2008) as well as in Turkey (Demir, 2017;Korkmaz, 2017;Taş, 2018;Yorulmaz, 2018).
In these studies which were conducted independently from each other, the teaching activities implemented based on RME, the effect of different variables such as sample size, level of education, treatment duration, and field of study were examined. These studies also have various limitations and due to this reason, conclusions of these studies may differ or show similarities to each other. Bringing together the research findings and creating a synthesis will lead the way to draw a conclusion and making generalizations of the results. Within this context, meta-analysis studies show great significance. Meta-analysis allows a coherent process of gathering and interpreting the results of individual studies conducted independently from each other (Cohen, Manion, & Morrison, 2007). When the literature is examined, it can be seen that a study by Kaplan, Duran, Doruk, and Öztürk (2015) brings together 12 dissertations that examine the effectiveness of teaching based on RME in Turkey.
In their study, the overall effect size regarding the individual studies conducted between 2007-2014 was calculated. Another study by Özdemir (2020), brings together 23 studies that examine the effectiveness of teaching based on RME in Turkey. In the mentioned study the overall effect size regarding the individual studies conducted between 2007-2019 was calculated. In a meta-analysis study conducted by Çelik (2013) examining the effect of alternative learning methods, the overall effect size of 4 dissertations which examined teaching based on RME 2007-2011 was calculated.
Considering years that these studies were published and the number of studies they took in the analysis, it can be stated that there is a need for a meta-analysis study which takes into consideration more recent studies and summarized the current situation on the matter.
In this respect, this study aims to determine the effects of RME-based teaching on mathematical achievement of the students through meta-analysis. The studies in which the effects of RME-based teaching are measured by standardized achievement tests (knowledge and abilities towards the learning outcomes of mathematics are tested in writing and measured on a standard score) were focused in the study. Moreover, different from the studies of Çelik (2013), Kaplan et al. (2015) and Özdemir (2020), the present study is to investigate if there is a significant difference in the effect sizes of studies included in the meta-analysis in terms of field of study, level of education, size of sampling, and treatment duration regarding RME-based teaching.
In line with this aim, the following research questions were sought to be answered: 1. What is the overall effect of mathematics lessons based on RME on students' mathematics achievement? 2. Does the effect size of mathematics lessons taught based on RME on students' mathematics achievement differ according to the field of study?
3. Does the effect size of mathematics lessons taught based on RME on students' mathematics achievement differ according to the level of education? 4. Does the effect size of mathematics lessons taught based on RME on students' mathematics achievement differ according to the sample size? 5. Does the effect size of mathematics lessons taught based on RME on students' mathematics achievement differ according to the treatment duration?

Research Design
Meta-analysis method was implemented in this study. A meta-analysis provides a general assessment through the analysis of quantitative results obtained from individual studies on a specific topic (Glass, 1976;Lipsey & Wilson, 2001). Through a meta-analysis, the current state of the related subject can be discovered. Effect size is used in the assessment of the findings of the meta-analysis study (Mertens, 2010). The value of the effect size reflects the relationship between two variables (Borenstein, Hedges, Higgins, & Rothstein, 2009;Ellis, 2010). In other words, it represents the size of the relationship between variables. The effect size is a common metric for studies that are included in effect size meta-analysis and it provides the opportunity of interpreting the statistically analyzed studies through the same measurement. There are certain steps to be followed in a meta-analysis study. Firstly, the problem is identified; and then the literature related to the literature is reviewed. The studies obtained as a result of the study are coded in specified criteria. Finally, the statistical analyses of the studies are conducted, and a conclusion is drawn (Pigott, 2012;Sánchez-Meca & Marín-Martínez, 2010). This study made use of the aforementioned steps.

Data Collection
The data of this study were collected within October 2020. The data source is constituted by studies that examined the effect of RME-based teaching on students' mathematical achievement in Turkey. In order to reach the studies, "realistic mathematics education, RME" keywords were searched on indexes such as Web of Science, Education Resources Information Center (ERIC), EBSCOhost, Scopus, Council of Higher Education Thesis Center, TR Index, and Google Scholar. A total of 96 master's thesis, doctoral dissertations and articles were reached as a result of the scanning. It was seen that some of the articles were reproduced from dissertations; instead of thesis, these articles were included in the meta-analysis, and the rest of the studies were picked in accordance with the following criteria: 1. The studies must be conducted in Turkey.

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 17 300-326 307 2. The studies must be conducted in 2020 or earlier.
3. The studies must have an experimental research design (experimental and control group design with pre-test and post-test).
4. There must not be a statistically significant difference between the achievement scores of experimental and control groups as determined by the results of pre-test (groups must be homogeneous in terms of achievement).
5. Experimental groups must be taught based on RME and the control groups must be taught based on the mathematics program determined by the national curriculum for the specific year.
6. Publication language must beTurkish or English. In line with the specified criteria, 40 studies were included in the meta-analysis. Two of these studies used two different achievement tests and one study included one experimental and two control groups. For these reasons, the effect sizes in these studies were calculated and presented in forest table with labels a and b next to the year of the studies. As a result, 43 effect sizes were calculated regarding 40 studies.

Data Coding
A feature to be encoded in meta-analysis studies may have a structure that will affect the effect sizes of the research (Ellis, 2010). For this reason, a coding has been made that can transform the data in these studies into categorical variables by using the studies that meet the inclusion criteria of the research (Lipsey & Wilson, 2001). Thus, the characteristics of the study were determined. The coding form of the study has a structure that is general enough to include all studies related to the effect of Realistic Mathematics Education-based teaching on mathematics achievement, but enough to determine research differences. A coding form was prepared by the researcher by taking into consideration the specified criteria of inclusion. The information included in the forms are: title of the study, year, author, type, sample size (experimental-control), level of education, field of study of the implemented teaching activities, treatment duration, reliability and validity statements regarding the data collection tools (present-absent), and arithmetic mean and standard deviation of the measurements at the end of the teaching activity. The information which was to be included in the study was coded on the form by the researcher. Three weeks after the initial coding, the forms were recoded by the researcher using the same forms. The forms were compared after the two processes and no difference was observed between the two forms. Through this procedure, an error-free statistical analysis of the data gathered from the studies was targeted. In Table 1, the descriptive statistics regarding the studies included in the metaanalysis which investigated the effects of RME-based on mathematical achievement in Turkey are presented.   (80) of the studies were related to mathematics, and 8 (20%) of the studies were related to geometry. While 6-10 hours of implementation (10 studies, 25%) and 16-20 hours of implementation (9 studies, 22.5%) were most prevalent, 7 (17.5%) of the studies did not specify the hour of implementation.

Data Analysis
While calculating the effect size of studies through meta-analysis, the fixed-effects model and random-effects model were used (Borenstein et al., 2009). In the fixed-effects model, the effect sizes of the studies to be included in the meta-analysis are assumed to be fixed; therefore, the effect sizes and standard deviations are taken as zero. In the randomeffects model, the effect sizes of the studies to be included in the meta-analysis are assumed to differ in every study, and the effect sizes and standard deviations are assumed to be different from zero (Ellis, 2010). The distribution of the effect size determines which one of these two models are to be used in a meta-analysis study. For this purpose, meta-analysis studies make use of Q value. Q value in statistics is used with the purpose of testing the null hypothesis that the meta-analysis studies which were analyzed through chi-square distribution share a common effect size. If the Q value is smaller than the equivalent value from the table of chi-square (χ 2 ) in terms of the degree of freedom (df) and level of significance (p-value), the homogeneity is established (Borenstein et al., 2009). If the distribution is homogeneous, the fixed-effects model is used; and, if it is heterogeneous, the random-effects model is used (Ellis, 2010). However, studies with extremely small or large effects, in other words, individual studies that differ significantly from the overall effect, that are so far out of the distribution that they are clearly outliers so they could be thrown out (Hunter and Schmidt, 2004). In order to detect such outliers in the data set, all studies were examined according to the following conditions: For which the upper bound of the 95% confidence interval is lower than the lower bound of the pooled effect confidence interval (i.e., extremely small effects), and for which the lower bound of the 95% confidence interval is higher than the upper bound of the pooled effect confidence interval (i.e., extremely large effects) (Harrer et al., 2019, Searching for extreme effect sizes (outliers) section, para. 2).
As a result, individual studies with extremely small or large effects were excluded from the analysis process (9 studies were excluded).
While calculating the effect sizes, Hedges'g, which determines the intergroup pooled and standard means were used, and the confidence level was accepted as 95% in the calculations. In interpreting the effect size, "0-0.20 level was accepted as weak, 0.21-0.50 was accepted as small, 0.51-1.00 level was accepted as medium, and a level greater than 1 was accepted as large" (Cohen et al., 2007, p. 521).
In  (Rosenthal, 1991). Therefore, it can be stated that as FSN value rises, the reliability of the results increases (Ellis, 2010). This study also made use of N/(5k+10) (k referring to the number of studies included in the meta-analysis) formula which was suggested by Mullen, Muellerleile, and Bryant (2001)

311
This study made use of Comprehensive Meta-Analysis (CMA) software in obtaining the effect sizes, moderator analyses, publication bias analyses, funnel plot, and forest plot.
And MetaWin statistics program was used to examine the normal distribution of effect sizes.
By making use of the interface that CMA offers, the format which enables values such as the sample size (N), mean ( ), standard deviation (SD), and p and t values were used. In this study, the field of study, education level, sample size, and treatment duration were determined as the moderators.

Findings
In order to determine whether it is convenient to combine the effect sizes of the studies with meta-analysis, the normal distribution chart was examined. Normal distribution chart is given in Figure 1. When Figure 1 is examined, it is seen that the effect sizes of the studies are distributed around the normal distribution line and within the confidence interval shown with dashed lines. In this regard, it can be stated that the effect sizes show normal distribution and can be combined statistically with meta-analysis.
A funnel plot was examined in order to determine the publication bias of the studies.
The funnel scatter plot is given in Figure 2.

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 17 300-326 Considering that more than one finding from the same study is used in a small number, it may not be said that this situation has a negative effect on publication bias. However, interpretation of a funnel scatter plot is subjective (Rothstein, Sutton, & Borenstein, 2005). Therefore, Rosenthal's N-FSN value was also examined in determining the publication bias. Statistics regarding this value are given in Table 2.   (Mullen et al., 2001;Rosenthal, 1991).
On the other hand, in this study, both heterogeneity test was performed and graphics were used to determine whether the effect sizes were suitable for normal distribution.
According to this, the results regarding the fixed effects model and random effects model of the studies included in the meta-analysis are presented in Table 3. 314 stated that RME-based teaching has a positive moderate effect on students' mathematical achievement.
Forest plot demonstrating the distribution of the effect size of the studies included in the meta-analysis according to the fixed effects model is shown in Figure 3. Results regarding the significant difference between the effect sizes of the studies in terms of field of study (mathematics and geometry) of RME-based teaching in are shown in Table 4. 0.779 0.625 0.933 0.079 * Mathematics field includes topics such as numbers and operations, fractions, sets, probability and algebraic expressions. The geometry field includes topics such as transformation geometry, polygons, geometric shapes, length, area and volume.
When Table 4 is examined, it can be seen that intergroup homogeneity value (QB) in terms of the field of study is 0.081. In the chi-square table, the critical value of 95% confidence interval with 1 degree of freedom is 3.841. It is also observed that the intergroup homogeneity value is smaller than the critical value in the chi-square table (QB=0.081, p=.777>.05). In this regard, it can be stated that the RME-based teaching does not show a significant difference in terms of the field of study.
The results regarding the significant difference between the effect sizes of the studies in terms of the level of education (primary school, middle school, and high school) of RMEbased teaching are shown in Table 5.

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 17 300-326 316 can be stated that the teaching RME-based teaching does not show a significant difference in terms of the level of education.
The results regarding the significant difference between the effect sizes of the studies in terms of sample size (16-30, 31-45, and 46-60 participants) of RME-based teaching are shown in Table 6. Three study was not included in the analysis as two of them had 1-15 participants and one of them had 76-90 participants as the sample size. Table 6 shows that intergroup homogeneity value (QB) in terms of sample size is 0.722. In the chi-square table, the critical value of 95% confidence interval with 2 degree of freedom is 5.991. It is also observed that the intergroup homogeneity value is smaller than the critical value in the chi-square table (QB=0.722,p=.697>.05). In this regard, it can be stated that RME-based teaching does not show a significant difference in terms of sample size.

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 17 300-326

317
The results regarding the significant difference between the effect sizes of the studies in terms of treatment duration (1-5, 6-10, 11-15, and 16-20 hours) of RME-based teaching are shown in Table 7. In this regard, it can be stated that RME-based teaching shows a significant difference in terms of treatment duration. The calculated effect sizes of the groups are medium. However, it can be stated that the effect size of the 6-10 lesson hours is at the large limit, while the effect size of the 11-15 lesson hours is at the weak limit.

Discussion and Conclusion
This study examining the effects of RME-based teaching on the mathematical achievement of students in Turkey, and a total of 43 effect sizes from 40 studies were examined. It was observed that all the studies had positive values, meaning that RME-based teaching was effective, in favor of the experimental groups. The overall effect size as calculated in accordance with fixed effects model is 0.760. This value is considered medium according to Cohen et al. (2007). In this regard, it can be stated that the RME-based teaching has a positive effect on the mathematical achievement of students. This finding is in agreement with the findings of Kaplan et al. (2015) Özdemir (2020). Moreover, this finding is also in alignment with the findings of some studies claiming that RME-based teaching has positive effects on the mathematical achievement (Demir, 2017;Fauzan, 2002;Gravemeijer & Doorman, 1999;Korkmaz, 2017;Le, 2006;Sembiring et al., 2008;Taş, 2018;Yorulmaz, 2018).
In this study, the field of study, level of education, sample size, and treatment duration were specified as the moderators. The purpose of this study is to examine if there was a statistically significant difference in the effect size of RME-based teaching in terms of these moderators. As a result of the analysis of the moderators;  The effect size values which were calculated in terms of mathematics (ES=0.753) and geometry (ES=0.779) fields had a medium level effect (Cohen et al., 2007), and there was no statistically significant difference.