Kahoot! as a Gamification Tool in Vocational Education: More Positive Attitude, Motivation and Less Anxiety in EFL

This study aims to reveal the effect of the gamified response system, Kahoot!, on attitude towards the EFL course, motivation, and exam anxiety. For this purpose, an embedded mixed design, in which quantitative and qualitative methods are used together, was preferred. The study group consists of 88 ninth-grade vocational high school students. Before the experimental process, attitude, motivation, and exam anxiety scales were applied to the experimental and control groups as pre-tests. At the end of the experiment, attitude, motivation, and exam anxiety scales were conducted as post-tests to the experimental and control groups. In addition, the views of randomly selected students from the experimental group on Kahoot! application was examined. The results revealed that Kahoot! significantly increased the attitude towards the EFL course with a 0.22 effect size. Kahoot! increased EFL learning motivation and decreased exam anxiety, but this was not significant. Finally, it was revealed that the students thought Kahoot! as funny. Received: 21 April 2021 Accepted: 27 July 2021


Introduction
The rapid change in instructional technologies enables the emergence of new teaching techniques in language teaching. Especially, the development of Web-based systems and the reduction of internet access barriers to a minimum level required the implementation of these techniques. Since the 1980s, much research has been conducted using technology in foreign language teaching (Thorne, Black, & Sykes, 2009). Figueroa Flores (2015 argues that foreign language teachers need new strategies to increase students' motivation and interest in foreign language teaching. Many researchers like Flores now support the use of technology in education to make learning meaningful. Chun, Kern, and Smith (2016) state that technology increases creativity and intellectual capacity. Having a progressive view on the meaningful use of technology in education, Prensky (2001) states that children have the digital language of the internet, computer, and video games. They spend most of their time with games and argue that students' learning should be supported. Kessler (2018) also suggests that technology should be included in education for effective learning because it has become difficult to motivate students with traditional methods (Premarathne, 2017).
Digital game-based learning (DGBL), one of the increasingly popular teaching methods in recent years, is beneficial in education in many respects. DGBL motivates students in the teaching process and is thought to increase engagement and interest. DGBL is an instructional approach in which computer use is supported for educational purposes (Mavridis & Tsiatsos, 2017). Many software designers and commercial companies hope to make academic learning more fun by developing games for learning (Kafai, 2006). Because game-based learning includes the discipline of the game and aims to involve the student in the learning process and increase learning performance (Graham, 2015). According to the report of the world-famous The New Media Consortium (NMC), which provides reports on the development of technology every year, educational games develop critical thinking, problem-solving, and group work, as well as the ability to solve the complexity of social and environmental contrasts (Johnson, Adams Becker, Estrada, & Freeman, 2014). Gee (2003) suggests that motivation can be provided through games, and its continuity can be learned.
Game-based learning performs three tasks; encouraging learning, developing knowledge, and increasing skills (McFarlane, Sparrowhawk, & Heald, 2002). The digital games used in education show their effects in all areas of education. Indeed, Kebritchi, Hirumi, and Bai (2010) argued that digital games increase mathematics achievement; Zin, Jaafar, and Seng Yue (2009) suggest that students who found history courses boring can become more fun with digital games. Papastergiou (2009) suggested that digital games positively affect physics and health education in terms of engagement. Many researchers claim that digitalbased games, which are more promising in foreign language education, have an important effect. Thomas (2012) argues that language learning through digital-based games provides success-oriented learning. Therefore, the use of digital games in the classroom can support cognitive development. It has been suggested that a carefully chosen game allows people to measure their status in terms of cognitive and behavioral development and motivation in the process of learning a foreign language (Cornillie, Thorne, & Desmet, 2012). Hubbard (2009) argues that digital games provide new information and skills to students in modern foreign language teaching methods and reinforce this information even without teachers. has been suggested that sometimes Kahoot!'s causing extreme competition may lead to negative consequences (Licorish, Owen, Daniel, & George, 2018). Bolat, Şimşek, and Ülker (2017) in their study with university students concluded that formative assessment activities were carried out with Kahoot! contributed to affective and cognitive skills. Orhan-Göksün and Gürsoy (2019) have found Kahoot! to be effective in academic success and engagement.
When the literature is examined, it is seen that many studies are examining the effect of Kahoot!. In a new review study, the researches examining Kahoot! were investigated, and it was concluded that Kahoot! mostly increased motivation and engagement (Wang & Tahir, 2020). Another result of the related study was the effectiveness of Kahoot! in learning a foreign language. Kahoot! also offers an effective strategy in terms of providing a fun environment in English education (Putri, 2019;Yürük, 2020), improves Arabic language grammar knowledge and motivation (Eltahir, Alsalhi, Al-Qatawneh, AlQudah, & Jaradat, 2021), increases engagement (Budiati, 2017), and especially helps students in concept teaching (Medina &Hurtado, 2017;Plump & LaRosa, 2017). It benefits educators as well as students (Zengin, Bars, & Şimşek, 2017). Kahoot!, used as a measurement tool by teachers, can positively affect students' exam anxiety by gamifying the evaluation process in education and moving it to a digital platform. According to a study, including gamification in the evaluation process allowed students to reduce their exam anxiety (Isbister, Karlesky, Frye, & Rao, 2012). In this study, the effect of Kahoot! on motivation, attitude, and exam anxiety in English as a foreign language (EFL) teaching was investigated in the sample of vocational high schools. In this context, the research questions are as follows:

Research Design
This study aims to examine the effect of Kahoot!, a gamified online response system, on students' exam anxiety, motivation, and attitudes. For this purpose, an embedded mixed design, in which quantitative and qualitative methods were used together, was used. In the embedded mixed research method, the quantitative or qualitative approach is more dominant, and the research is largely qualitative or quantitative (Yıldırım & Şimşek, 2013).

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 18 682-701

686
This research method can be used in cases where quantitative data collected while answering the research questions are desired to be enriched with qualitative data (Creswell, 2013). In this study, quantitative data obtained with quasi-experimental design were interpreted by supporting them with student views. The research process is presented in Figure 1. As shown in Figure 1, the Motivation Scale in English Language Learning, the Attitude Scale Towards the English Course, and the Exam Anxiety Scale were applied as pretests. Then, a 10-week application period was carried out. After the experiment, all three scales were used as post-test. After the experimental process, the views of the four randomly selected students from the experimental group regarding the activities performed by the Kahoot! application were taken.

Study Group
The study group of the research consists of ninth grade (the first grade of high school) students in four different classes studying at a vocational high school in Mardin in 2019 (the second semester). We used the convenience sampling method while specifying the study group. The classes are similar in terms of academic characteristics, and their English course teacher is the same person (the first author). Two of the classes were randomly allocated as the experimental group (N = 44) and two as the control group (N = 44). All of the participants were male because it was a boys' high school.

Experimental Process
Both the experimental and control groups took English courses by the same teacher for five hours a week for ten weeks. The experimental process was carried out by the first author of the study, a teacher in the relevant high school. To summarize the concepts learned in the course and reinforce the learning during the week, the students in the experimental group were applied gamified online quizzes using the Kahoot! application at the fifth hour.
The quizzes prepared by the teacher were reflected from the smartboard, and students accessed the exam with their quiz passwords. Each quiz consists of 20 questions in total.
Throughout the experimental process, the Ministry of National Education ninth grade second semester English course curriculum was followed. Within the scope of that curriculum, the topics of "6. Bridging culture", "7. World heritage", "8. Emergency and health problems", "9. Invitations and celebrations", and "10. Television and social media" were covered. Two quizzes were applied for each subject, and a total of 10 quiz activities were carried out during ten weeks. It has been reported that the students who are in the top five will gain five points for their verbal grades to ensure active engagement before these activities are implemented. In Figure 2, a one-week course process carried out in the experimental group is presented.

Data Collection Tools
To collect the quantitative data of the research, the Attitude Scale Towards the English Course (Takkaç-Tulgar, 2018), Motivation Scale in English Language Learning (Mehdiyev et al., 2017) and The Revised Exam Anxiety Scale developed by Benson and El-Zahhar (1994) and adapted to Turkish by Akın, Demirci, and Arslan (2012), were used. The Motivation Scale in English Language Learning consists of self-confidence, attitude, and personal use dimensions and 16 items. The scale was developed as a five-point likert type.
Answers to the items were scored as (1) for "Never agree", (2) for "Partially agree", (3) for "Moderately agree", (4) for "Mostly agree" and (5) for "Totally agree". There are four reverse items in self-confidence and two in attitude dimensions. Therefore, a total of six items were reverse coded. The validity and reliability analysis of the scale were made, and the internal consistency reliability test result was found as α = .83. The Revised Exam Anxiety Scale consists of four dimensions: tension, physical symptoms, anxiety and thoughts unrelated to the exams, and 20 items. The scale is scored in four levels as (1) "Never", (2) "Sometimes", (3) "Mostly", and (4) "Always". Scale items were developed as negative expressions. Therefore, 688 the exam anxiety level was interpreted high as the score obtained from the scale increased, and exam anxiety was interpreted low as the score obtained from the scale decreased. The validity and reliability analysis of the scale were made, and the internal consistency reliability test result was found as α = .88. The Attitude Scale Towards the English Course consists of four dimensions: interest, importance, contribution, and informationentertainment, and 26 items. The scale is a five-point likert type, and the scores range from "1: I never agree" to "5: I totally agree". All items were positively expressed on the scale.
Validity and reliability analysis of the scale were made, and the internal consistency reliability test result was found as α = .96.

Data Analysis
Two-factor analysis of variance is a method that analyzes pre-test and post-tests simultaneously with group variables. In this study, two-factor analysis of variance (Mixed ANOVA) for mixed measurements was used since the scores of experimental and control groups will be interpreted according to the pre-test and post-test scores (Can, 2014). The assumptions of the two-way analysis of variance were tested, and the data were found to meet these assumptions. When deciding on the analysis of the quantitative data of the research, the assumptions of the two-way analysis of variance were taken into account. The normal distributions of the data were examined with Kurtosis-Skewness values, Kolmogorov-Smirnov test, and Histogram graphics. Kolmogorov-Smirnov test results are presented in Table 1.

689
When Table 1 is examined, it is seen that the data of the pre-test and post-tests of the experimental and control groups showed normal distribution. The content analysis method was used in the analysis of the qualitative data of the research. Content analysis is an inductive analysis method based on coding revealing the facts underlying the data (Patton, 1990). In content analysis, the data are coded and organized within the framework of certain themes and concepts and interpreted in a way that the reader can understand (Yıldırım & Şimşek, 2013). Draw.io software was used during the visualization of qualitative findings.
The qualitative data were analyzed by the researchers and interpreted with direct citations.
The intercoder reliability method was used to ensure the reliability of the content analysis. In addition, to increase the validity and reliability of the research, the study spread over ten weeks in total, so the innovation effect was prevented.

The Effect of Kahoot! on Students' EFL Learning Motivation
Two-way analysis of variance was used to analyze the pre-test and post-tests of the experimental and control groups to examine the effects of using classroom assessment activities with Kahoot! in the ninth-grade English course on students' EFL learning motivation. The descriptive statistics obtained from the analysis are presented in Table 2, and the significance results are presented in Table 3. In addition, the change in the pre-test and post-test scores of the groups is given in Figure 3. According to this result, similar changes occurred in both groups. To show whether this situation has a significant effect between the two groups, two-way analysis of variance results is given in Table 3. As seen in Table 3, there is no significant difference between the experimental and control groups (F = 1.51, p = .22) at the significance level of p = .05. The fact that the partial effect size (η 2 = 0.02) of the group variable is very close to zero confirms this situation. In Figure 3, the graphic of the change in both groups' pre-test and post-test motivation scores is presented. As seen in Figure 3, the experimental group's motivation increased more than the control group. Based on this result, it can be claimed that the activities carried out with Kahoot! have no significant effect on the students' EFL learning motivation. However, the change in motivation score means does not have a significant impact.

The Effect of Kahoot! on Students' Exam Anxiety
Two-way analysis of variance was used to analyze the pre-test and post-tests of the experimental and control groups together to examine the effects of using classroom assessment activities with Kahoot! in the ninth grade EFL course on students' exam anxiety.

691
Descriptive statistics and significance results obtained from analyzes are given in Table 4 and   Table 5. In addition, the distribution of pre-test and post-test scores is presented in Figure 4.  Table 5 to show whether this situation has a significant difference between the two groups. As seen in Table 5, there is no significant difference between the experimental and control groups (F = 0.68, p = 0.41) at the significance level of p = .05. The fact that the partial effect size (η 2 = 0.01) of the group variable is also very close to zero confirms this situation. In support of this result, the exam anxiety change in both groups' pre-test and post-test is shown in Figure 4.

The Effect of Kahoot! on Students' Attitudes Towards EFL Course
Two-way analysis of variance was used to analyze the pre-test and post-tests of the experimental and control groups to examine the effects of using classroom assessment activities with Kahoot! in the ninth grade EFL course on students' attitudes towards the EFL course. Descriptive statistics and significance results obtained from analyzes are given in Table 6 and Table 7. In addition, the distribution of pre-test and post-test scores is presented in Figure 5. increased, but the control group decreased slightly. To show whether this situation has a significant effect between the two groups, the results of two-way analysis of variance are given in Table 7. As seen in Table 7, a significant difference was found between the experimental and control groups (F = 9.61, p = 0.003) at the significance level of p = .05. The partial effect size of the group variable was found to be η 2 = 0.10. When the impact of Kahoot! on attitude was calculated with the pre-test and post-test measures, the effect size was 0.22. This finding shows that the group variable causes 22% of the variance in students' attitudes. In support of this finding, Figure 5 shows the distribution of the pre-test and post-test measurements of both groups' attitudes toward the EFL course. As seen in Figure 5, while the attitude of the experimental group students towards the course increased, the control group decreased slightly. Based on this finding, it can be

Students' Views about Kahoot! Application
The views of the students about the Kahoot! application used in the 10-week EFL learning process were consulted, and the obtained codes are presented in Figure 6.
Accordingly, the following codes are obtained; funny, providing effective learning, offering equal opportunities, remembering in the exam, usefulness, providing permanent learning, and creating a competitive environment. When the codes in Figure 6 are examined, it can be said that students are satisfied with the use of Kahoot! in EFL courses and the views are positive. In these views, the students emphasized Kahoot! app's features of permanence, effective learning, and remembering at the time of exam. In addition, motivational elements such as being fun and creating a competitive environment in the classroom have been focused. It can be interpreted that this situation can support the motivation of students towards EFL courses. Kahoot! 's being useful and offering equal opportunities among students may also be effective in students' developing ideas favoring using this application more in EFL courses. The direct expressions of the students regarding these codes are presented in the following section.
The participant students expressed their views related to the code of Kahoot!'s effective learning as follows: F. Y.: "It is an application that we can raise the level of English both by having fun and learning.", Y. Ü.: "I have positive, fun, and instructive thoughts about Kahoot! application" and O. E.: "It is a very educational and guiding application". Related to the code of permanence, Ş. A. expressed his view as: "Because sometimes it was permanent in our minds when a word was repeated more than once. But if it was shown only once, we could only remember these words instantly. We started to keep the words better in our minds". Related to the code of giving an advantage in the exam, Ş. A. stated: "We could find these words in the exams questions. So, we were a bit more advantageous". Related to the code of fun, the students emphasized this aspect of Kahoot! as follows; F. Y.: "It is an application that we can raise our level of English by having fun and learning", O. E.: "I don't think anyone will get bored. There is nothing funny like learning through play", Y. "We have observed that even the unsuccessful students can learn something. Because it can teach even those, who don't know well. There were a lot of students in our class who didn't know English, but even they learned". By these statements, they emphasized it enabling the opportunity of learning for relatively unsuccessful students.

Discussion and Conclusion
In this study, it is aimed to examine the effect of the online response system Kahoot! application, which was applied in the ninth grade EFL course in vocational high school, on students' motivation, attitudes and exam anxiety towards the course, as well as student views on these activities they experienced. The research data were obtained from a study group of 88 students at a vocational high school in Mardin. The results of the research are limited to the answers given by the students to the scales and interview questions, and the activities carried out within the scope of five topics in 2019 (the second semester) ninth-grade textbook as; (6. Bridging culture, 7. World heritage, 8. Emergency and health problems, 9. Invitations and celebrations and 10. Television and social media). Besides, Kahoot! application was used to summarize and reinforce the learnings in the last hour of the week.
The results of the research should be evaluated within these limitations. According to the According to the third research question of the study, the effect of Kahoot! on attitude towards EFL courses was examined. As a result of the analysis made for this purpose, it was revealed that the evaluation activities were carried out with Kahoot! significantly increased the attitudes of the experimental group students towards the EFL course compared to the control group. Studies investigating the effect of Kahoot! on students have revealed that Kahoot! has a positive effect on teachers' and students' attitudes and increases engagement (Wang & Tahir, 2020). Yürük (2020) found that Kahoot! positively affects students' EFL pronunciation skills. As a matter of fact, students with positive attitudes are expected to participate in the lesson. According to Md. Yunus and Nur Rashidah Khairunnisa (2011), a positive attitude and higher motivation may occur in learning English when supporting environmental factors are provided to students. In our study, the fact that the teacher gave additional points to the top five students in the 10-week experimental process may have increased the students' attitudes. Gamification-based teaching practices positively affect students' attitudes (Yildirim, 2017). In addition, the proven effectiveness of Kahoot! in English courses (Wang & Tahir, 2020) supports these results.
The quantitative results of the study have shown that Kahoot! increases both the attitude towards EFL course and EFL learning motivation and reduces exam anxiety.
However, there was a significant difference only in terms of attitude towards the EFL course.
Students' views also support these results. As a matter of fact, students pointed out especially the fun features of Kahoot!. These findings are in parallel with the results in the literature. Kahoot! offers an effective strategy to provide a funny environment in English teaching (Putri, 2019;Yürük, 2020) and increases engagement (Budiati, 2017). Also, in our study, it was understood that students thought Kahoot! contributed to permanent learning, provided an advantage in exams, and created a competitive environment. Views expressing that Kahoot! has a user-friendly interface were also put forward. It can be claimed that Kahoot! is motivating because it provides a competitive environment (Wang, 2015).
However, it should not be forgotten that sometimes Kahoot!'s extreme competition may lead to negative consequences (Licorish et al., 2018). Therefore, it may be beneficial for teachers to consider this situation while using Kahoot!. It may be useful for teachers to use it only in a formative manner so that competition does not cause concern. All these results indicate that if Kahoot! is used in EFL courses in vocational high schools, it will increase the interest in the course. In addition, these results are expected to be reflected in academic achievement.

Journal of Computer and Education Research
Year 2021 Volume 9 Issue 18 682-701 698 Eltahir et al. (2021) used Kahoot! as a game-based online assessment tool and determined that Kahoot! improved the Arabic language grammar knowledge and students' motivation.
Because students' high motivation or positive attitudes may affect academic achievement (Good & Brophy, 2000). Accordingly, Bury (2017), who researched online assessment tools, revealed that these tools improve students' grammar knowledge.
As a result, with this experimental study, we tried to reveal the effect of Kahoot! on EFL learning in vocational high schools. As a result of this study, we conducted it within the framework of the EFL course; it has been proved that enriching the courses with Kahoot! activities significantly affects the attitude towards the EFL course. According to this result, in addition to the attitude towards EFL courses, Kahoot! can positively affect the attitude towards the school. Thus, it can be claimed that the attitudes of vocational high school students, who have a negative attitude towards the school in general (Atalay-Mazlum & Balcı, 2018), can be changed positively with gamified digital-based activities. Finally, it is a limitation that the study participants were only males. The fact that our sample consisted of only males may have manipulated our results. Therefore, similar new research can be conducted with a mixed sample to provide more conclusive results evidence.