Meta-Analysis of the Effect of Technologies on Primary School

: The main purpose of this study is to show how much learning technologies increase the success in primary school with meta-analysis method. Advanced technologies such as virtual-augmented reality, animation-simulation, inverted classroom application, web-internet course applications, course and game teaching software, which are subject to experimental studies in computerized learning environments, have been included in this meta-analysis study, since they have features that can improve academic achievement. It is taken into account that having features such as; publication in a peer reviewed open-access magazines, the execution of the application by selecting the sample from the primary school in Turkey, listing the mean and standard deviation values and the number of sample group students for the final test, for the articles to be included in this research. It was concluded that advanced technology supported lesson applications in primary school have the ability to increase academic achievement at a very high level. The effect size found (md: 4.17239) is larger than the highest level value statements defined in the literature.


Introduction
Having young and dynamic population ratios is the most important strength of societies. The existence of a young population is important for countries, however it is much more important that young people receive a good and quality education. The most important contribution of school processes to young people is to equip their learning environments with improved technological software for easy, fast and lasting teaching. A welfare society can be realized by the contribution of young people who are prepared for the unknowns of the future to scientific technological developments.
The economic development of societies, the allocation of necessary and sufficient budget to education, the development of countries, educational institutions, individuals and learning are interrelated. Quality equipment of learning environments ensures that what is learned is permanent. It triggers individuals raised with an improved education system to contribute to production and development.
There are several factors that trigger the academic success of individuals in learning environments. Constructivist or behavioral teaching programs, the dedication and being equipped of teachers, the technological materials of learning environments are some of the important factors that trigger the academic success of students. In order to the students to succeed academically, in the classroom and other settings, teachers have the opportunity to produce and use technological studies with control-group that examines the impact of using technological material are listed. In this study, Easy Meta Analysis software (mix 2.0 version) was selected to find the effect size of the data using technological materials consisting of independent studies. Mix 2.0 is an important and useful meta-analysis software that calculates the effect size based on the average values of the experimental and control groups.

Effect Size Classification
The effect size values based on the effect size classification calculated over the mean values of the groups in experimental studies are as listed below (Cohen, 1988, p. 198;Alegre, Moliner, Maroto, & Lorenzo-Valentin, 2019;January & Klingbeil, 2020); • Has a small effect when md (effect size)=0.20.
• Has a moderate effect when md (effect size)=0.50.
• Has a wide effect when md (effect size)=0.80.
Alegre et al, concluded that the magnitude of the effect was very similar in general for qualitative and quantitative studies, in their literature study, and they used Cohen's d classification as a measure of magnitude for all the studies they worked on (Alegre, Moliner, Maroto, & Lorenzo-Valentin, 2019). January and Klingbeil, carried out a systematic review of measurements regarding the early reading curriculum with computer adaptive testing, and curriculum-based measurement (CMB) tools and in these studies, they showed that the early reading effect size findings and the reading success effect size findings obtained from meta-analysis have an effect size between medium and large classification according to Cohen's d classification (January & Klingbeil, 2020). Vincent & Torres used the effect size interpreted by Thalheimer & Cook for Cohen's d for group comparisons in their study (Vincent & Torres, 2015). Yüzlü & Atay conducted a study in which they applied the strategy of using mother tongue in foreign language teaching, and in these articles, they defined the effect size between the groups with Cohen d effect size calculator created by Thalheimer and Cook (Yüzlü & Atay, 2020).

Data Collection
In the literature survey, keywords covering all technological material applications were used separately or together (Sterpetti, Costi, & D'Ermo, 2020). Keywords such as animation, simulation, technology, technological materials, computers, information, software, artificial intelligence, intelligent lecture system, virtual reality, augmented reality, web, internet, Virtual Lab, Computer-Aided, computer-based, technology-supported, simulation-aided, animated instruction, lab simulations, webbased, internet-based, web-assisted lab, multi-media, multi-learning environment, inside out classroom, inside out classroom environment, course software, IT supported learning environment, computerized instruction, educational technology, instructional technology, success, academic success, effect of success, Primary School, nursery class have been used With the help of the listed keywords, 52 articles published between 2010-2020 in open-access peer-reviewed journals whose experimental samples were selected from primary school or nursery class in Turkey have been accessed. In these collected studies, the contribution of technological material applications to the academic success of the students was examined. 52 articles accessed have been reviewed, 33 data of the 16 articles which are controlgroup and have comparison data and provide other inclusion criteria have been recorded for metaanalysis. These 33 data are derived from articles published in 2010-2011and between 2014 in the meta-analysis of the articles on cancer, determined that there were many missing data in the published articles in order to reach meaningful results, and drew attention to the importance of data collection in meta-analysis studies (Sterpetti, Costi, & D'Ermo, 2020).
The data collected from the articles conducted for primary school nursery class, first grade, second grade, third grade and fourth grade (according to the Turkish education system) students are listed according to grade levels. Unlike other countries, Primary School classroom teachers only educate students at the first, second, third, and fourth grade levels for four years (Official Gazette, 2012;Odabaşı, 2014). Non-compulsory nursery class education in primary school is carried out by pre-school teachers. The fifth grade classes in primary school are in the second tier of compulsory primary education in the Turkish education system. In the Turkish education system, fifth, sixth, seventh and eighth grades are provided at the secondary school level (Official Gazette, 2012;Odabaşı, 2014).

Coding Process
The number of students participating in the application, the average of the experiment group, the standard deviation of the experiment group, the average of the control group, the standard deviation of the control group, publication dates and author names were recorded for the 33 data that constitute the basis for the evaluation. In these studies, preliminary data were also examined in terms of validity and reliability.

Analysis of the Data
The calculation of the differences between the averages of the control group studies in the recorded data was made by means of Mean Difference Effect Meta-analysis. MD common effect size (effect size) is converted from the data of the different studies included with meta-analysis. The conversion formulas for MD effect size are shown in Table 1. Combining the data obtained from the accessible articles and saved with Meta analysis is provided from the MIX -Version 2.0 (made easy meta analysis) package program.

Findings
A meta-analysis of 33 data of 16 control-group articles, which meet the criteria determined by the researcher, and which measure the impact of technological material-supported applications in 4 years of Primary School in Turkey system was carried out. The Meta-analysis process is completed in two stages. The first phase process completed with a measure showing the effect of the learning method with 95% confidence intervals (CI) for each independent study. The second phase is completed by calculating an overall learning effect as the weighted average of summary statistics of independent articles. It should be noted that data from independent articles is not combined with a simple summation for meta-analysis. The data of articles that provide more information are more concentrated on in Meta-analytic combining. Numerical and graphic findings from the meta-analysis of 33 data sets were recorded. Note: In Table 2, the articles included in the meta-analysis are cited.
When articles with more than one data are listed again in Table 2, they are listed together with their respective page numbers. Table 3 lists the class level frequencies of the 33 datasets. According to this, meta analytic consolidation and impact size operations were performed for 19 data at first grade level, two each at nursery class and third grade level, 3 at second grade level, 7 at fourth grade level, and a total of 33 data.  Table 4 lists which technological materials are used in primary school. According to this, computer-aided teaching, animation, digital material and lecturing video, multimedia, multimedia material, media-supported learning, augmented reality, computer-based teaching and similar technological materials were used. As is seen from this table, academic success is achieved by using technological materials in every subject and teaching every achievement in primary school.  Table 4, the articles included in the meta-analysis are cited.
Any articles showing that artificial intelligence technologies such as 'Intelligent lecture system' helps to improve learning in elementary school, not yet published in Turkey.
Easy Meta Analysis Mix 2.0 software program uses article number, author names, publication date, arithmetic mean and standard deviation of test and control groups, number of students in test and control groups for effect size calculation. In Table 5, the information of 33 data compiled from 16 articles, required for the calculation of impact magnitude, are listed by ordering as is mentioned. Note: In Table 5, the articles included in the meta-analysis are cited.
Required calculations have been made for the 33 data uploaded to the Easy Meta Analysis Mix 2.0 software program system. At the end of related calculations, a value of 251,92811 for Q, a value of 2,80584 for H, a value of 87,298% for I2, a value of 5,02568 for t2 are calculated and shown in Table  6. These values indicate that the relevant data is heterogeneous. The fact that the value of I2 is 75% and greater is considered an important indicator for heterogeneity (Owen, Cooper Table 6 in this meta-analysis study remain unchanged according to the calculation for the constant effect and random effect. The selectivity funnel diagram is shown in Figure 1 by drawing a funnel chart. This funnel chart drawing points to the potential for publication bias due to the formation of asymmetry. The potential for publication bias can be eliminated by papers not published in peer-reviewed journals, such as those presented in academic forums. These papers are unlikely to be included in the analysis as they are not always achievable, and publication bias for similar reasons is a limitation in the nature of meta-analysis (Mavridis & Salanti, 2014;van Aert, Wicherts, & van Assen, 2019).

Note: All figures must be printed colored
In this meta-analysis, only the articles that their samples were selected from Turkey and published in Turkey were reported. Therefore, the formation of asymmetry in Figure 1 (Debray, Moons, & Riley, 2018) refers to the reporting of articles whose publication language is Turkish and published in Turkey.
The synthesis estimate is shown in Orange in Figure 1, and a significant portion of the values representing 33 data are excluded from the drawing representing the limit values of the funnel diagram. Thus, it is understood from the funnel chart drawing in Figure 1 that the data being analyzed is heterogeneous. Meta-analysis of this heterogeneous data was performed using random effect model.
The effect size calculated using the random effect model via Easy Meta Analysis Mix 2.0 software program has a value of 4,17239 for MD and is listed in Table 7 along with other detailed calculation values. It is shown in Table 7 that the effect size can take values between 3,1567 and 5,18808. * The bias analysis results should be treated as exploratory The id numbers, subject numbers, contributions to meta analytic effect size of the independent articles and the smallest-largest value of these contributions in the 95% confidence interval, the Z value, the p significance value and the weight values are listed in column positions and other calculated total synthesis details Table 8    The synthesis diagram chart (synthesis forest plot), calculated by Meta-analysis and generated from the information listed in Table 8, is shown in Figure 2 (Rücker & Schwarzer, 2020). In the synthesis diagram chart, created from the findings of 33 data in the synthesis details table, the squares represent the probability ratios of independent articles, the horizontal lines that cut these squares show 95% CI confidence intervals. The field of the squares reflects the weight that each article contributes to the meta-analysis.
(The diamond below the red vertical line represents the MD effect size)

Figure 2. Synthesis forest plot
Note: All figures must be printed colored In Figure 2, the diamond below the red vertical line represents the MD effect size. In this chart, center of the diamond that shows MD the effect size represents the effect of technological material on learning (md: 4,17239). There is one square in the graph for each of the works listed in Table 8. In Figure 2, The Vertical order of the squares and the rows in Table 8 overlap. The lengths of horizontal lines that cross frames represent 95% of the confidence interval between ci -and ci+ for each data, as listed in Table 8.

What is technological material?
In this meta-analytic literature survey, all possible materials with technological content that can be used to improve academic achievement in learning environments were evaluated as technological course materials (Wilson, Ritzhaupt, & Cheng, 2020;Saini & Salim Al-Mamri, 2019). Wilson et al, examined technological course materials that affect learning environments and pre-service teacher training, which are the product of existing technologies and emerging technologies (Wilson, Ritzhaupt, & Cheng, 2020). The technological materials used in the learning environment are listed in Table 4 as augmented reality (Oranç & Küntay, 2019;Garzón, Kinshuk, Baldiris, Gutiérrez, & Pavón, 2020), computer-based education, multimedia-supported material, computer-aided instruction, multimedia material, computeraided realistic math material, animation, digital material and lecturing video. As is seen here, emerging technologies such as animation and augmented reality are also being used as materials in learning environments and contribute to academic success. Garzón et al, conducted a meta-analysis study on course material with augmented reality content (Garzón, Kinshuk, Baldiris, Gutiérrez, & Pavón, 2020). In addition, there are course material studies with virtual reality simulation content in the literature (O'Connor, et al., 2020).
How does using technology in primary school contribute to students' learning?
How do the learning environments using technological materials affect students' knowledge and skill achievements?
According to the findings of each articles included in the Meta-analysis (Şahin & Çakır, 2018;Akaydın & Kaya, 2018), the relevant experimental research with technological material were found contributing to academic success on significant levels (Shen & Ho, 2020;Chen, Xie, Zou, & Hwang, 2020). Shen & Ho, analyzed articles on technological material and noted that the results of teaching and learning were enriched at the higher education level with course material developed with technology (Shen & Ho, 2020). Chen et al, examined articles applying artificial intelligence technologies to education and listed the most commonly used artificial intelligence technologies as machine learning, intelligent tutoring system and artificial neural networks (Chen, Xie, Zou, & Hwang, 2020).
What effect size would be created for experimental research with technological material when combined with Meta-analysis?
The MD: 4,17239 value for the calculated meta analytic effect size for a total of 33 datasets of 16 articles that meet the inclusion criteria in the literature review shows that technological materialsupported teaching contributes far more to academic success than the methods and practices carried out in control groups (Rakes, et al., 2020;Gegenfurtner & Ebner, 2019;Marion-Martins & Pinho, 2020;Scherer & Teo, 2019). The common view of the 16 papers reviewed is that the use of technological materials at primary school level facilitates learning and increases academic achievement. Marion-Martins & Pinho conducted a meta-analysis study that showed that simulation technology was effective in training nursing students (Marion-Martins & Pinho, 2020). Gegenfurtner & Ebner have defined that webinars are effective for higher education and vocational education students through their meta-analysis work (Gegenfurtner & Ebner, 2019). Although it is statistically significant, Rakes et al, concluded that their evidence was insufficient to make claims about the overall effectiveness of technology in a student's mathematics achievement, due to their very small effect sizes in their metaanalysis (Rakes, et al., 2020).
What would the effect size founded by the meta-analysis method for experimental studies carried out at the primary school level be among those size levels defined in the literature?
In the literature, the highest effect size classification in meta-analysis studies has a value of md> 1.45 and the perfect level definition for this effect size value has been made (Dinçer, 2015, p. 102;Vincent & Torres, 2015, p. 68;Cavalcanti, Campos, & Araujo, 2012, p. 1269. In this study, md: 4,17239 value was reached for meta analytic effect size, well above the level defined in the literature, in order to find out the extent of the impact of computerized technological material on academic success in primary school. However, a high-level value for effect size may not only be the usual result of metaanalysis. The asymmetry that appears to be the publication bias in Figure 1, which actually indicates the reporting of articles published in Turkey, created a larger impact value than expected in this metaanalysis study (Debray, Moons, & Riley, 2018;van Aert, Wicherts, & van Assen, 2019). Thus, all the questions of the study were answered and the purpose of the study was realized.

Recommendations
Our children who are tomorrow's elders and will achieve social development and experience a welfare society, should have the opportunity and possibility to recognize technology to grow academic achievement in primary school. For this purpose, classroom teachers should be trained in educational faculties and their learning environments should be equipped with emerging technologies. Teaching technologies and material design courses in classroom teaching programs must be conducted in the technological equipped learning environments. In a word, classroom teachers should be familiar with technology. Classroom teacher candidates should be encouraged to learn and use technology. Thus, the competence of the classroom teacher candidates to produce and use technological materials should be improved. The final recommendation of this study is that how technological material learning environments affect attitudes and motivation in primary school should also be investigated.