The Effect of Technology Integration in Education on Prospective Teachers' Critical and Creative Thinking, Multidimensional 21st Century Skills and Academic Achievements

Accepted: 07.12.2020 In this study, it was aimed to examine the critical and creative thinking, multidimensional 21st century skills and the change in academic achievements as a result of technology integration of prospective teachers who have science education in pedagogy fields. Research was carried out in Turkey's western Black Sea region in a state university. 144 prospective teachers, who were educated in the faculty of education and who were in science, classroom and pre-school education departments, participated in the research. The research was carried out in 3 stages. In the first stage, technology integration is not provided. In the second stage, basic and medium level technology integration is provided. In the third stage, advanced technology integration is provided. Quantitative and qualitative approaches were used together in the research. Academic success test, critical and creative thinking test developed by the researcher as a means of quantitative data collection, and three different scales with validity and reliability were used previously. In addition, project, exam, homework, presentation and group work scores are included in the process. Semi-structured interview, observation and field notes, document review, were used as qualitative data collection tools. The quantitative data obtained were subjected to descriptive and inferential statistics. While doing these operations, SPSS 23.0 and LISREL 9.2 package programs were used. Qualitative data were subjected to descriptive analysis and content analysis. The results of the research show that gradual integration of technology into the education process provides a positive change in prospective teachers' critical and creative thinking, multi-dimensional 21st century skills and academic achievements.

In this study, it was aimed to examine the critical and creative thinking, multidimensional 21 st century skills and the change in academic achievements as a result of technology integration of prospective teachers who have science education in pedagogy fields. Research was carried out in Turkey's western Black Sea region in a state university. 144 prospective teachers, who were educated in the faculty of education and who were in science, classroom and pre-school education departments, participated in the research. The research was carried out in 3 stages. In the first stage, technology integration is not provided. In the second stage, basic and medium level technology integration is provided. In the third stage, advanced technology integration is provided. Quantitative and qualitative approaches were used together in the research. Academic success test, critical and creative thinking test developed by the researcher as a means of quantitative data collection, and three different scales with validity and reliability were used previously. In addition, project, exam, homework, presentation and group work scores are included in the process. Semi-structured interview, observation and field notes, document review, were used as qualitative data collection tools. The quantitative data obtained were subjected to descriptive and inferential statistics. While doing these operations, SPSS 23.0 and LISREL 9.2 package programs were used. Qualitative data were subjected to descriptive analysis and content analysis. The results of the research show that gradual integration of technology into the education process provides a positive change in prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievements.

Introduction
People are continually developing and changing. Accordingly, science and technology are progressing at an unbelievable speed. Especially in the 21 st century, when we are at the beginning of technology development, we can see this. Many things that we could not even dream of have happened now, and they are happening rapidly. Changes in technology can cause people to experience both hope (digital convenience, access to all kinds of information, solution-oriented technological applications, medical developments) and worrying situations (technology addiction, internet abuse, virtual fraud) (Gunuc, 2017). This situation requires technology to be managed systematically and to be included in individuals' life processes in a planned and programmed way (Thomas & Brown, 2016). States made the most investment to people in all periods they existed. It made this investment through education systems (Durnali & Ayyildiz, 2019). The education system may differ from country to country. However, their goals are shared: "Qualified staff and well-educated individuals". At this point, technological changes play a significant role (Palak & Walls, 2009;Yilmaz & Aydin, 2019).
Today, the use of technology has become a necessity, not a privilege. Because technology is included in every area of our life, mobile phones, cars, apps, computers, smart homes, and many things we cannot here count constitute the abundance of examples. According to the "We Are Social -Digital 2020 April Global Statshot" report, 59% (4.54 billion) of the world population is internet users, 49% (3.80 billion) are social media users, and 67% (5.19 billion) are mobile users (Kemp, 2020). This shows how vital technology is in human life. Another area in which technology takes place is the education system. The education system is open to all kinds of changes in the society. Because the task of the education system is to prepare the individual for society and real-life (Ozan, 2013;Robin, 2008). Technology makes many direct and indirect contributions to the education system. Online learning, simulation environments, virtual laboratories, access to scientific information, instant access to technological developments, online learning applications and many other situations are solely some of these (Brito, Dias & Oliveira, 2018). The inclusion of science and technological developments in the educational process causes the emergence of several new skills and concepts. "Technology literacy, computer literacy, 21 st century learners, internet generation, technological native, digital native" can be given as examples (Gunuc, 2017, p.2). In addition, these behaviours, expressed as 21 st century skills, are expressed by NEA please provide the full form (2008) as follows (Tuzel-Iseri, 2018): • Learning and innovation skills (creativity and innovation, critical thinking, critical thinking and problem solving, communication and collaboration) • Information, media and technology skills (information literacy, technology literacy) • Life and career skills (flexibility and compatibility, entrepreneurship, leadership and responsibility).
As can be seen, technological developments also change the expectations of educators. In addition to being academically successful, students are also expected to acquire many alternative skills (Trilling & Fadel, 2009). Because teacher-centred traditional education is replaced by student-centred education. Students are now as close to technology as a smartphone. They can instantly access the information they want with their mobile phones. Software, coding and digital applications have become an indispensable component of our daily life and education process (Area & Ribeiro, 2012;Yilmaz, Gulgun, Cetinkaya & Doganay, 2018). In the educational process, many branches of science come into play while preparing students for life. Mathematics education, social studies education, Turkish education and science education are some of them. However, the place of science education in science branches is slightly different. Because science plays a significant role in students' getting to know and make sense of the environment they live in (Jorde & Dillon, 2012). Science can be divided into sub-branches such as physics, chemistry and biology.
This branch of science is significant for students to acquire scientific process skills, gain systematic working habits, find solutions to problems encountered in daily life, analytical, critical, reflective and creative thinking, and especially gain the so-called 21 st century skills (Lombardo, 2010). In addition, it is another essential feature to provide easy integration of technology and to have alternatives for adapting course content to technology. In the 21 st century, knowledge is learned in a pile. Now, only information learned in schools is not enough for students. Therefore, continuous research, analysis, learning and teaching situations are essential parts of development (Lai & Viering, 2012). In this context, education types should be revised, and education should adopt various uses of technology. In our country, education types are divided into two as formal education and non-formal education. Formal education is education planned and programmed in schools. Non-formal education is the type of education carried out to meet the educational needs of individuals who cannot continue their formal education for any reason (Demirel;Sahin, 2015). However, education is not always carried out in schools. Natural disasters, global crises, wars, international conflicts and epidemics can prevent education from being fulfilled (Burgess & Sievertsen, 2020). One of the prominent factors within the scope of this research is the recent Covid-19 pandemic. Currently, there is a global virus epidemic known as Covid-19 in our world. This epidemic, which first emerged in China in December 2019, has spread to a large part of the world (Ozer, 2020). As of May 1, 2020, can be updated it is estimated that it infected approximately 4 million people and caused the death of 280 thousand people. All countries take precautions in the fight against this virus epidemic (OECD, 2020). Alternative education types, called "Emergency Remote Education", which enable the education process to continue using the technology infrastructure, play an essential role in this process. In the Emergency Remote Education process, technological infrastructures are strengthened and regulated by the education system (UNESCO, 2020).
These studies conducted during the pandemic process show that each individual's technology infrastructure and equal opportunity must be the same in the emergency remote education process. However, this is not the case in many countries around the world. Some students do not have technological infrastructure (computer, tablet). Some of them cannot even access the internet. Sometimes faculty members cannot adequately include technology in their courses. These situations led to a different perspective to the event. There appears a question in this regard: If technology integration is included in the teaching process step by step, what will be the result? Studies in which technology integration is included in the teaching process step by step have been examined in light of the literature. However, there are not many studies where technology integration is gradually included in the education system. From this point of view, it is thought that it would be appropriate to conduct a study in which technology integration is gradually included in the education process and thus it will contribute to the relevant field of science.

Theoretical Framework and Technology Integration Models
"Engagement and Technology Integration Theory" developed by Gunuc (2017, p.22) was used within the scope of the research. In this theory, technology integration is discussed at the micro level. In-class and out-of-class teaching and learning activities have been designed. The basis of this theory is not only the teacher. Both the teacher and the student are at the center. The basic idea of the theory is to explain that student engagement and technology integration are related to student success and effective learning. Gunuc (2017, p.22) expresses student engagement as follows: "Student engagement is the quality and quantity of the student's psychological, cognitive, affective, behavioural responses and energies to participate in the learning process, academic and social activities inside/outside the classroom to achieve successful learning outcomes." Figure 1 shows the Engagement and Technology Integration Theory. (Gunuc, 2017, p.23) When Figure 1 is examined, first of all, it is necessary to emphasize the feelings of value and belonging of students. After these steps are fulfilled, activities should be done in order to create cognitive, affective and behavioral commitment. These should be accomplished by providing practical technology integration. As a result, feelings of commitment will be combined with technology integration, and effective learning outcomes will be created. This process can be used continuously in educational environments as a cycle.

Figure 1. Engagement and Technology Integration Theory
During the research process, two different models were used. "Technology Integration Planning Model" which was developed by Robyler (2006) and consists of six stages was used first. The purpose of this model is to provide teachers with a general planning approach in the process of integrating technology into their lessons. In this model, which has six different stages, all stages are followed, and teachers are guided step by step like a guide. In other words, teachers are presented with an extensive planning map. The second model is "Pedagogy, Social Interaction and Technology Generic Model" developed by Wang (2008). The purpose of this model is to guide teachers again and to provide the skills to plan and use pedagogy, social interaction and technology components together.

Purpose of the Research
In this study, it was aimed to investigate the effect of "Technology Integration in Education on Prospective Teachers (with science education in the field of Pedagogy) on Critical and Creative Thinking, Multidimensional 21 st Century Skills and Academic Achievements".

Problem Statement and Subproblems
Within the scope of the research, the main problem seeks the answer to the question: "What is the Effect of Technology Integration in instruction (gradually) on Critical and Creative Thinking, Multidimensional 21 st Century Skills and Academic Achievement of Prospective Teachers (who have science education in the field of Pedagogy)?" Within the framework of the fundamental problem situation, answers were sought for the following subproblem situations: (1) How does the instruction without technology integration affect the prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievements? (2) How does the instruction provided by providing basic and intermediate level technology integration affect the prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievements? (3) What are the effects of the instructions provided by advanced technology integration on prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievements? (4) How do the different approaches applied at each stage affect the prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievements? (5) What are the opinions of the prospective teachers regarding the application scales and sub-dimensions?

The Research Model
In the research process, quantitative and qualitative research approaches were used together. The research model was created by using descriptive sequential pattern from mixedmethod research. In descriptive sequential pattern applications, the process begins with quantitative applications first. Then quantitative applications are analyzed, and the results are reported (Creswell, 2014;Sozbilir, 2017). However, quantitative application results provide limited information about the overall results of the study by providing statistical significance, confidence interval and effect dimensions. This situation is not sufficient for how the results are formed and for establishing cause-effect relationships. In the second stage, qualitative practices come into play. With qualitative application results, problem situations are subjected to a more in-depth examination, and the results are interpreted (Goktas, 2017).
In the quantitative stage of the application, pretest -posttest semi-experimental pattern and survey method were used together. The study was designed in a semi-experimental design in general, and sub-applications (use of scale) were used from time to time. Technology integration has been implemented in 3 different stages. In the first stage, applications were made only by considering science education without technology integration. In the second stage, technology integration is included in the basic and intermediate level processes, and applications are realized. In the third stage, technology integration is included in the advanced process and applications are completed using fully emergency remote education procedures. In the qualitative phase of the application, a case study was used. In this context, the descriptive case study was preferred. Because descriptive case study is a frequently preferred method in cases where complex and cause-effect relationships need to be established. During the research, both superficial and in-depth information can be collected (Guclu, 2019).

The Study Group
In his research 144 teachers studying at a state university located in Turkey's western Black Sea region, participated. While determining the study group, criterion sampling was chosen from non-probabilistic sampling methods in order to increase the effect factor and reflective level of the application (Buyukozturk, Kilic-Cakmak, Akgun, Karadeniz & Demirel, 2016). As a determination criterion, departments with science education were chosen within the fields of pedagogy. In this context, prospective teachers who are studying in early childhood, science and classroom teaching departments are preferred. Both experimental and control groups were formed at all stages of the application.
Experimental and control groups are divided into three subgroups. Each subgroup consists of 8 prospective teachers. Prospective teachers who are early childhood, science and classroom teachers are both in the experimental group and the control group. During the research, prospective teachers were coded as Experiment 1 and Control 1 for science knowledge, Experiment 2 and Control 2 for classroom teaching, Experiment 3 and Control 3 for early childhood. Also, after all, quantitative applications, semi-structured interviews were made with two people from each group. Regardless of these sample groups, sampling was done using the appropriate sampling method in the validity and reliability analysis of the data collection tools to be used. However, the characteristics of prospective teachers participating in pilot applications are not included. The demographic characteristics of prospective teachers participating in the application are shown in Table 1.

Data Collection Tools
Qualitative and quantitative data collection tools were used together in the research process. Therefore, the data collection tools used are specified separately. In the quantitative data collection phase, five different tools were used. Two of these data collection tools were developed by the researcher. The first data collection tool is "Academic Achievement Test-AAT" with 25 items, and the second data collection tool is "Critical and Creative Thinking Test-CCTT" with 25 items. Other data collection tools are, "Critical Thinking Standards Scale for the Teacher Candidates-CTSCTC", which was developed by Aybek, Aslan, Dincer & Coskun-Arisoy (2015), consisting of 3 factors and 41 items, "Multidimensional 21 th Century Skills Scale-MSS" consisting of 5 factors and 41 items developed by Cevik & Senturk (2019) and "Student's Perception Scale About Instructors Technology Integration Competence-SPSITIC", consisting of 2 factors and 25 items developed by Artun & Gunuc (2016). For all data collection tools, necessary permissions were obtained from the relevant authors via email. This situation is clearly stated in the ethical statement section. In addition, validity and reliability analyses were carried out at all stages by making pilot applications. The original of all scales was developed in the Turkish language. For this reason, it is recommended to use Turkish forms in applications.
At the qualitative data collection stage, firstly, two prospective teachers from each group were determined (the average score was the lowest and the average score was the highest). A semistructured interview with three pre-determined prospective teachers and developed by the researcher was held at each stage. Then, during the applications, observations were made by the researcher, and field notes were taken. In addition to these practices, the assignments and projects prepared by prospective teachers as a result of the applications and exam grades were examined through document analysis. In the evaluation process of qualitative data, both teacher-oriented evaluation and peer evaluation were made. Validity and reliability information of data collection tools are presented in detail later in the article.

Application Process and Data Collection
In the research process, qualitative and quantitative applications were discussed separately. In Figure 2, the application process and the procedures performed at each stage are shown in detail.

Figure 2. Application process and procedures
The research process consists of 3 stages in total. Quantitative and qualitative applications were carried out at each stage separately. However, some of the qualitative applications (observation, field notes and document review) were carried out in coordination with the quantitative applications. Quantitative applications have always been applied first, and then qualitative applications have been made. Each stage lasted 12 weeks. During the study, applications were made without technology integration. In the second stage, technology integration has been provided at basic and intermediate levels.
In the third stage, advanced technology integration was provided.
All applications are structured considering the science course. While making quantitative applications, data collection tools were applied as a pre-test in the first week and as post-test in the 12 th week. The data for the 1 st stage were collected in the spring semester of the 2018-2019 academic year. Data for the 2 nd stage were collected in the fall semester of the 2019-2020 academic year. Data for the 3 rd stage were collected in the spring semester of 2019-2020 academic year and during the emergency remote education (pandemic). The faculty member factor, another component of technology integration, was also taken into account in the research. For this purpose, a data collection tool that measures the instructor's usage skills is also included in the process.
Different procedures were carried out for the experimental and control groups in all applications performed during the research. Traditional teaching methods were used for prospective teachers in the control group at all stages, and technology integration was carried out only at stage 3 (mandatory). However, the technology integration realized at this stage is presented in a similar way to traditional teaching. Technology integration was presented to the prospective teachers in the experimental group gradually, and the changes in this group were examined in detail. Detailed information regarding the applications made in Table 2 is given. Control 1, Control 2, Control 3 -Conductedwith two people from each group.
-It was carried out on the 12th week.
-All groups are observed every week.
-Field notes were taken regularly.
Academic achievement test Academic achievement test Critical and creative thinking test Critical and creative thinking test Scale 1 -CTSCTC Scale 1 -CTSCTC Scale 2 -MSS Scale 2 -MSS

Applications
Course notes of the researcher are given.

Course notes of the researcher are given. Evaluation Procedures
For academic readings, articles etc. documents have been distributed.
The course process was conducted in the form of interactive dialogue and question-answer.
-Examining and scoring of research assignments.
The course process was conducted in the form of interactive dialogue and question-answer.
Prospective teachers gave presentations without using digital materials.
-Examining and scoring of the projects produced.
-Examining and scoring midterm exams. They were asked to do their research assignments without digital resources.
They were asked to do their research assignments without digital resources.
-Examining and scoring presentations.
They were asked to produce a project that could be used in science education. Midterm exams were held.
-Examining of observation and field notes, transcript and analysis of interviews.

Application Groups
Interview Observation and field notes
-It was carried out on the 12th week.
-All groups are observed every week.
-Field notes were taken regularly.
Academic achievement test Academic achievement test Critical and creative thinking test Critical and creative thinking test Scale 1 -CTSCTC Scale 1 -CTSCTC Scale 2 -MSS Scale 2 -MSS Scale 3 -SPSITIC Scale 3 -SPSITIC

Applications
Google Classroom activities Course notes of the researcher are given. Evaluation Procedures

Data collection and compilation in the digital environment
The course process was conducted in the form of interactive dialogue and question-answer.
-Examining and scoring of research assignments.

Group work and project production
Prospective teachers gave presentations without using digital materials.
-Examining and scoring of the projects produced.
-Examining and scoring midterm exams.
-Examining and scoring presentations.
Preparing interactive presentations They were asked to do their research assignments without digital resources.
-Examining of observation and field notes, transcript and analysis of interviews. E-portfolio application Midterm exams were held.
-Examining and scoring of e-portfolio files

Application Groups
Interview Observation and field notes Experiment 1, Experiment 2, Experiment 3 Control 1, Control 2, Control 3 -Conducted with two people from each group.
-It was carried out on the 12th week.
-All groups are observed every week.
-Field notes were taken regularly. -Course notes of the researcher are given.
-The course process was conducted in the form of interactive dialogue and question-answer.
-Prospective teachers were asked to prepare digital course material.
-They were asked to do their research assignments in the digital environment.
-Online midterm exams were held.
-Examining and scoring of research assignments.
-Examining and scoring of the projects produced.
-Examining and scoring midterm exams.
-Examining and scoring presentations.
-Examining and scoring digital course materials.
-Examining and scoring of infographics.
-Examining of observation and field notes, transcript and analysis of interviews.
-Peer assessment (Experimental group) -Analysis of necessary activities.
-Analysis of mandatory activities.
-Analysis of conversations in Whatsapp groups.
-Determining rates of participation in emergency remote education. Assignment, presentation preparation and online midterm exam applications in the digital environment Group work and online project production Google Classroom activities Experiment 2 -Classroom Education Online emergency remote education (Zoom) Assignment, presentation preparation and online midterm exam applications in the digital environment Group work and online project production Google Classroom activities Necessary activities (problem-solving applications, Quizizz, Testmoz, Kubbu, word matching, Learningapps, Mentimeter.

Experiment 3 -Science Education
Online emergency remote education (Zoom) Assignment, presentation preparation and online midterm exam applications in the digital environment Group work and online project production Google Classroom activities Mandatory activities (problem-solving practices, Quizizz, Testmoz, Kubbu, word matching, Learningapps, Mentimeter) Preparing infographic Establishment of Whatsapp groups and constant communication in problem-solving (with researcher participation)

Data Analysis, Validity and Reliability Applications
While analyzing the data, as in all stages, quantitative and qualitative findings were analyzed separately. Quantitative data has been subjected to descriptive and inferential statistics. While doing these operations, SPSS 23.0 and LISREL 9.2 package programs were used. Qualitative data were subjected to descriptive analysis and content analysis. The results obtained are presented in the findings section with the help of tables and figures. In this study, which aims to determine the effects of technology integration in education, three successive and supportive practices were implemented. These applications were mainly carried out using quantitative approaches. Although there are differences between the applications, it includes similar processes in terms of analysis, validity and reliability (Yilmaz & Yanarates, 2020). Reliability and validity applications are discussed separately.

Reliability Applications
Within the scope of the application, reliability measures were taken considering the quantitative and qualitative data. During quantitative applications, five different data collection tools were used together. Of these, the academic achievement test and the critical and creative thinking test were developed by the researcher. At this stage, firstly, the literature review was done, and a draft item pool was created by using the indicator table. Then, expert opinion was taken from 10 different academicians who worked as science specialists in science education by using Lawshe (1975) technique. As a result of expert opinions, the draft pool of articles was revised, and a pilot application was made (McMillan & Schumacher, 2009).
Item difficulty, item discrimination, 27% subgroup and upper group mean values and Cronbach Alpha coefficients were determined after the pilot and final application. Finally, both data collection tools were given their latest form, and final applications were made. The other three scales used in the quantitative data collection phase are the previously valid and reliable scales. However, in order to be compatible with the application sample, all scales were piloted, and internal consistency Cronbach Alpha values and 27% subgroup and upper group averages were calculated (Fraenkel, Wallen & Hyun, 2011).
In the qualitative data collection phase, semi-structured interview and observation were used. Opinions of the field experts were received during the preparation of the interview questions. As a result of the pilot implementation, some of the interview questions were rearranged (in terms of language and content) and finalized. Then, participant selection criteria were created, and a systematic selection was made. Transcripts recorded as a result of the interviews were subjected to content analysis. At this stage, transactions were carried out by adhering to the criteria of content analysis (Yilmaz & Yanarates, 2020).
Before starting the coding and sorting process, several preliminary preparations were made in order to perform high-quality coding. These preparations can be expressed as creating coding guide, giving detailed training to coders, pretesting and improving management procedures in order to check the applicability of the coding system and whether it is working (Krippendorff, 2004;Ozkan, 2019). Analyzer triangulation was used for coding and extracting the obtained interviews. With this application, it is aimed to prevent the occurrence of similar and controversial situations while coding. In addition, the consensus and divergence levels of three different encoders were calculated with the help of the formula determined by Miles & Huberman (1994), and this rate was determined as 92%. Because of the consensus levels of the coders are in ideal ranges, the Cohen Kappa Coefficient was finally determined, and the coordination and interoperability ratio was determined as .84. All statistical results calculated in this context are presented in Table 3. In addition to these studies within the scope of reliability measures, other measures mentioned in the relevant literature and included in this application are as follows (Batdi, 2019;Flick, 2009;Patton, 2014;Yilmaz & Yanarates, 2020): (1) First of all, clear, simple and detailed information was provided at each stage.
(2) As it is mainly a quantitative study, triangulation has been made with qualitative applications. The subject has been deeply studied with multiple applications and data collection tools. (3) In the context of credibility and transferability, direct quotations were made from time to time. By providing examples over raw data, the reliability of the study was increased. (4) Due to the use of content analysis in the analysis of qualitative data analysis units, codes to be used (preparation of coding guide, pilot application, training of coders), categories, data processing and interpretation steps are also included in the process. (5) Other measures included in the process involve the choice of well-known research methods, continuous observation, long-term and systematic reviews, detailed presentation of information, implementation of audit trail (detailed description of the data collection and data analysis process), and comparison with findings in the literature.

Validity Applications
Validity measures were taken in the research considering the quantitative and qualitative data. Validity applications, as in reliability applications, require standard processes for some applications and different processes for some applications. Firstly, the content and appearance validity of the academic achievement test, critical and creative thinking test and interview questions were examined (Mor-Dirlik, 2020). Lawshe (1975) technique, which is a statistical application, was used for content and appearance validity. Content validity rates and content validity indices were calculated for each question individually. Confirmatory factor analyzes (CFA) were then performed to ensure structure validity.
CFA analyses were performed for all quantitative data collection tools. LISREL software was used while performing these operations. Despite its validity in the literature, confirmatory factor analyses were carried out both before and after the final implementation in order to determine the status of the scales serving the purpose and to support the construct validity (Ozdamar, 2002;Mor-Dirlik, 2014). In addition, convergent validity and combined reliability values were calculated as a result of these analyzes. Results for the analyzes are presented in Table 4. In addition to these studies carried out within the scope of validity measures, the other measures mentioned in the relevant literature and included in this application are as follows (Batdi, 2019;Ozkan, 2019): (6) In order to ensure descriptive and interpretive validity, the data in the research process are presented randomly, objectively and without exaggeration. (7) In order to provide theoretical/internal validity, necessary care has been taken to eschew overlap of the concepts and categories created by the researcher with the results achieved and to support the different practices used. (8) Comparison of research data on generalizable/external validity and findings in the related literature has been made, results obtained are generalizable, and they are expressed consistently.
Attention has been paid to ensure criterion validity in the process of determining the prospective teachers to be interviewed, cross-referencing during the examination of qualitative data and creating the coding guide during the coding phase and attention has been paid to each stage in this regard.

Findings
Research findings are handled separately for each problem case. Firstly, the applications made in the first stage were examined. At this stage, technology integration is not included in any process. There are quantitative analyzes made in Table 5.  Table 6 and Figure 3. Technology affects success to some extent. C2 1 Success is possible with the effort of the individual. C6 1

Observation and Field Notes
Prospective teachers produce more systematic solutions when they do academic readings. 3 Scientific process skills education works very effectively in project production. 4 Scientific projects affect prospective teachers' sense of taking responsibility.
2 Traditional teaching approach affects the diversity of thoughts of prospective teachers.
3 Conducting research assignments without using technology challenges prospective teachers. 3 21 st century skill education is very effective in generating alternative thoughts and ideas. 3 Researching in groups and providing the division of labor are included in the process as a source of motivation. 5 Traditional applications cannot adequately meet the needs of students. 2 The quality of projects and assignments cannot go too far without technology. After the first stage was completed, the second stage was started. At this stage, technology integration is included in the process at a basic and intermediate level. There are quantitative analyzes made in Table 7.  Table 8 and Figure 4.  After the second stage was completed, the third stage was started. At this stage, technology integration is included in the process in an advanced manner. There are quantitative analyzes made in Table 9. .05] in favor of prospective teachers studying in science education, there were significant differences between the groups. To summarize, as the level of technology increases, both between-group and within-group significant differences increase positively. Qualitative findings for the third stage are presented in Table 10 and Figure 5.

Observation and Field Notes
Advanced technology integration allows students to focus longer. 3 As the use of technology increases, the rates of communication increase.
3 Digital messaging environments and constant online status affect group success.
3 The use of technology ensures that the educational process continues actively outside of school.
6 Some skills, such as working with a group and problem solving, can be acquired directly without the need for additional effort as communication increases with the use of technology. 3 As the level of technology integration increases, prospective teachers can work deeper. 4 The fourth problem situation of the research was: "How do the different approaches applied at each stage affect the prospective teachers' critical and creative thinking, multi-dimensional 21 st century skills and academic achievement?" In this context, in order to determine the effects of the applications performed at each stage, multi-dimensional regression analyses were conducted on the post-test results of the prospective teachers in the experimental group. Regression analysis for the first stage is presented in Table 11.  [F(4-19)=56.74, p<.05] results were reached and significant differences were found. Here, each dependent variable showed a positive change depending on the subparameters. In addition, dependent variables are determined by sub-parameters; AAT test (R=.74, R 2 =.55), CCTT test (R=.62, R 2 =.38), CTSCTC scale (R=.79, R 2 =.62) and for the MSS scale (R=.74, R 2 =.55) affect levels. Regression analysis for the second stage is presented in Table 12. Technology integration has affected the opinions of prospective teachers towards the first scale and at least the views towards the third scale. Regression analysis for the third stage is presented in Table 13. =.81) and for the SPSITIC scale (R=.76, R 2 =.57) affect levels. Technology integration has mostly affected the opinions of prospective teachers for CCTT test, and at least the views for the third scale. For he fifth problem of the research; "What are the opinions of the prospective teachers regarding their application scales and sub-dimensions?". the average results are presented in Table 14.

Conclusion, Discussion and Suggestions
This study, in which technology integration in instruction was examined within the scope of science education, was carried out in 3 different stages. Technology integration has never been achieved in the first stage, at the basic and intermediate levels in the second stage and the advanced level in the third stage. Within the scope of the study, critical and creative thinking, multidimensional 21 st century skills and changes in academic achievements of prospective teachers were examined. Experiment and control groups are divided into three subgroups at each stage. In all stages and sub-steps of the research, measurement, evaluation and research methods training has been kept constant. Firstly, as there was no technology integration in the first stage, the experimental group was given scientific process skills education and 21 st century skills education, unlike traditional education. Traditional education was given to the control group. Besides, academic readings and project activities were made to the experimental group. While all activities were being held, prospective teachers were informed about not including technology in the process and explicitly stating the resources they use. In this context, when table 5 was examined, it was determined that there was no significant difference between the pre-test results of the experimental and control groups. This is a situation that should be in experimental studies. Because groups with similar characteristics should be included in the process. When the post-test results of the experimental and control groups are examined, it is seen that there is a significant difference, and this difference is in favour of the experimental group. Besides, the pre-test and post-test results of both groups were compared, and significant differences were found.
Since the prospective teachers in the experimental group had a higher significance level, the analyses of this group were deepened. As a result of the one factor ANOVA test, it was determined that the scores of prospective science teachers differed significantly from prospective early childhood and classroom teachers. This difference occurred in AAT and CCTT tests, and there was no significant difference in scale applications. When the literature is analyzed, it is seen that prospective science teachers show scientific process skills and the so-called 21 st century skills more frequently than the prospective teachers studying in other departments and they have a high tendency towards these skills (Beaumont-Walters & Soyibo, 2001;Cetin & Solmaz, 2020;Downing & Filer, 1999;Duran & Ozdemir, 2010;Farsakoglu, Sahin, Karsli, Akpinar & Ultay, 2008). Reasons for this include the fact that critical and analytical thinking is the basis of science education, frequent use of scientific processes (experiment, application, etc.) and areas require sub-branches of science such as physics, chemistry and biology to actively use 21 st century skills (Demir, 2007;Tifi, Natale & Lombardi, 2006). It can be interpreted that there is no significant difference in prospective teachers' thoughts about application scales, when technology integration is not provided, these processes are not actively used, and technology should be considered as a factor that supports these behaviours (ChanLin, 2005;Hussain & Safdar, 2008). Hsu & Kuan (2013) stated that there are many factors affecting technology integration in their study. Among these factors, it has been stated that individuals need to interact with technology for a long time in order to develop technology integration. These results support the current research results.
In Table 6, it is seen that prospective teachers have different views about technology integration. It is especially emphasized that some skills are directly related to technology integration. It is a natural result that prospective teachers think in this way. Because the individuals who participated in this study were born in the 21 st century and started their education life after 2000. From the first stage of their educational life to the present, they have incorporated technology into their educational processes at various levels and have benefited from it. These opinions also support the results of the scale study conducted within the scope of quantitative applications (Gunuc, Odabasi & Kuzu, 2012;Kolikant, 2010). Because when there is no technology integration, all prospective teachers advocate similar thoughts. When Figure 3 is examined, it has been determined that project development, research assignments, presentation grades, exam grades and observation results for prospective teachers in the experimental group differ significantly from the prospective teachers in the control group. This supports the interview results, AAT test and CCTT test results. In the second stage of the research, technology integration is included in the basic and intermediate level process. In the first stage, only scientific process skills education and 21 st century skills education given to the experimental group were given to all groups equally after this stage. However, technology integration in the control group was not achieved. Technology integration was provided only to the experimental group. At this stage, a new one was added to the quantitative applications, and the technology usage of the instructors was also questioned. Within the scope of the applications, technology-based applications such as Google Classroom activities in the experimental group, data collection in the digital environment, preparation of interactive presentations, e-portfolio application were included in the process. When Table 7 is examined, it is seen that there is no significant difference between the pre-test results of the experimental and control groups as in the first stage. When the post-test results of the experimental and control groups are examined, it is seen that there is a significant difference in favour of the experimental group. The pre-test and post-test results of the experimental and control groups also differ significantly among themselves. While there was a significant difference in the AAT test and SPSITIC scale application in the control group, it was seen that there was a significant difference in all applications in the experimental group. Since the level of significance was higher in the experimental group, the analyses were deepened. As a result of the one factor ANOVA test, it was determined that there was no difference for AAT and MSS scale, and that there was a significant difference for CCTT, CTSCTC and MSS scale. Here, prospective teachers studying in the science department achieved higher results than prospective teachers in both the classroom and early childhood department. The reason for this can be said to arise from the content and technology suitability of science education as in the first stage (Bybee, 2010;Gibson, 2012).
Also, as a result of the interviews and observations done with prospective teachers, they think that the technology level used in the instruction process is sufficient and that technological applications will contribute to a certain point. When the reasons of this view are examined, it can be shown that the use of advanced technology in schools and higher education has not been established yet, the technological infrastructure is not equal in every university, and prospective teachers do not use technology for specific purposes (Bittman, Rutherford, Brown & Unsworth, 2011;Cetin, 2021). At this stage, where technology integration is applied more than the first stage; it was determined that the project development, research assignments, presentation grades, exam grades and observation results differed significantly and increased positively for prospective teachers in the experimental group compared to the prospective teachers in the control group. Also, it was stated in the interviews with prospective teachers that after the technology was included in the process, they needed to ask more questions, had to communicate more with the instructor, and creative ideas arose as the level of access to information increased (Gray, 2008;Naish, 2008).
Asadi, Abdekhoda & Nadrian (2019) stated in their study that the willingness levels of the teachers in the process of adapting to technology integration positively affects their success levels. It has been observed that as the relationship with technology increases, the behaviors of using technology and including it more actively in lessons improve. Technology integration from the third and final stage of the research was mandatorily applied to both groups. Because during this period, owing to the global epidemic (Covid-19) and the announcement of the pandemic period, all education processes had to be carried out as emergency remote education. However, technology integration was implemented at the advanced level in the experimental group, and the basic and intermediate level in the control group. At this stage, traditional education in the control group was tried to be given as emergency remote education. In the experimental group, technology applications were gradually increased. These applications are; Google Classroom activities that include online project preparation, necessary activities, mandatory activities, infographic preparation and the establishment of WhatsApp groups in solving problem situations. These processes are presented in detail in Table 2. When the quantitative results for the third stage are examined, no significant difference was found between the pretest results of the experimental and control groups as in the other stages. When the posttest applications of the experimental and control groups are examined, it is seen that there is a significant difference in favour of the experimental group. Also, all groups create differences in terms of pre-test and post-test results. The control group created significant differences in itself in all applications except for the CCTT test. The experimental group showed a significant difference in all subapplications. Since the results in the experimental group yield higher results than the control group, the analyses for this group have been deepened. As a result of the one factor ANOVA test, prospective teachers in science teaching department made a positive difference in all applications compared to other branches. Providing advanced technology integration has also changed prospective teachers' thought structures and approaches to technology (Dewitt & Siraj, 2010). When the technology integration is made at lower levels, the prospective teachers who find the process sufficient show more interest in this process and as the technology level increases and express that new needs and skills emerge (Cakiroglu, 2016).
Advanced technology integration encourages prospective teachers to focus longer, increases technology usage times and enables them to continue their educational work outside of school (Jung & Ottenbreit-Leftwich, 2020;Turac, Caliskan & Gulnar, 2017). Besides, it has been determined that project development, research assignments, presentation grades, exam grades and observation results differ significantly and increase positively for prospective teachers compared to all other stages. Technology integration was realized in a group with the necessary activities before starting the lesson. In this process, most of the prospective teachers did not participate because the activities were not compulsory. In another group, some activities were mandatory before starting the lesson. Later, lessons were taught. When the results of the study were examined, it was determined that the group's mandatory activities had higher grade averages and differed in all applications compared to other groups (Anderson & Putman, 2020;Conejar & Kim, 2014;Kirschner & Karpinski, 2010). In another experimental group, in addition to these activities, an infographic was prepared, and a WhatsApp group was established in which the researcher participated in this process. The researcher acted as a guide and a team member in this group and meticulously followed up correspondence for the whole process.
As a result of the application, it was determined that the establishment of WhatsApp group and continuous discussion about problem situations, sharing information and increasing the interaction caused the students to increase their academic success depends on many 21 st century skills, critical and creative thinking skills and the result of these. There are many studies supporting this situation in the related literature (Ebersole, 2019;Jones, Blackey, Fitzgibbon & Chew, 2010;Korkmaz & Ozturk, 2020;Sebetci, Topal, Hanayli & Gurel-Donuk, 2018). As a result of the third stage, it was determined that the prospective teachers' opinions, project development, research assignments, presentation grades, exam grades and observation results increased in all groups. Regression analysis results that are carried out for the level of the applications applied at each stage can be examined. As the technology integration and use of technology increases in all processes, the skills desired to be acquired increase systematically. The important point here is; increasing the interaction is the necessity for the technology to be well-structured in the education process and to be integrated in a harmonious way. Prospective teachers showed a regular upward trend in their thoughts regarding their application scales. As the level of technology integration increased, their attitudes towards the application scales and the mean of thought increased. Within the scope of this research, the following suggestions can be made; • Technology integration should be made gradually to the education process and presented with well-prepared educational contents. In the use of technology, emphasis should be placed on practices that can involve participants. This situation increases participants' cognitive, affective and behavioural loyalty and affects their success levels positively. • Technology integration is a labour-intensive process that requires advanced computer and technology literacy. In this context, it has become a necessity rather than a need for educators to improve themselves and keep up with the needs of the age. While providing technology integration, participants should be taken to the centre of the applications, and the process should be structured together so that they can both enjoy it and contribute fully to the very process.

Ethical Declaration
Ethical rules conducted this study. During the research, all details were kept under control and carried out within the framework of scientific ethics.

References
Aguliera, E., Nightengale-Lee, B. (2020). Emergency remote teaching across urban and rural contexts: perspectives on educational equity. Information and Learning Sciences, 121