The effects of socio-scientific STEM activities on 21st century skills of middle school students

Accepted: 03.08.2021 In this study, the effects of the STEM activities integrated with socioscientific issues on students' 21st century skills were investigated. In this research, nested mixed design was used. The study group of this research consisted of 16 middle school students who studied in the seventh grade of a public school in an eastern province of Turkey. Socio-scientific STEM activities were applied to the study group for 24 weeks in the first and second semesters. Students followed the engineering design stages for STEM activities. In the research, 21st Century Skills Scale, Interview Form, Perception of 21st Century Skills Form, Field Notes, and Informal Interview Notes were used as data collection tools. SPSS was used to analyze the quantitative data of the research, and content and descriptive analysis methods were used to analyze the qualitative data. The study results showed that STEM activities integrated with socio-scientific issues had a positive effect on students' 21st century skills. Also, as a result of the examination of the opinions of the students, it was determined that the application developed the “creativity and innovation” skills of the students the most. Furthermore, students’ social and intercultural entrepreneurship, self-management, leadership and responsibility, critical thinking, problem solving, communication and cooperation, creativity and innovation skills were enhanced/developed at the end of the study.


Introduction
Individuals who have innovation, entrepreneurship, and production skills should be trained to provide qualified workforce to the industry. Qualified individuals with these skills can be trained through comprehensive and coordinated education. The USA needed a comprehensive and coordinated education approach and developed STEM education to raise STEM-literate individuals who have technical knowledge and skills (Bybee, 2010). To this end, STEM schools were opened in many states (Akgunduz et al., 2015). by people's moral, belief, ideological, thought structure, emotional, and cultural structure, we can say that it is a socio-scientific issue (Eastwood et al., 2012).
It is important that socio-scientific issues are included in the curriculum and hence taught. In recent years, many studies have indicated that socio-scientific issues are important to achieve scientific literacy in school education (Walker & Zeidler, 2007). As socio-scientific issues are an inseparable part of daily life, countries have started to include these issues in their curricula. It is seen that socio-scientific issues are included in the science education programs of many countries (Albe, 2008), especially in the USA and the UK. Turkey did not fall behind this development, and the term "socio-scientific issues" was used for the first time in the 2013 Science Curriculum prepared by the Ministry of National Education (MoNE). In this program, it was aimed to develop students' scientific thinking habits, scientific and moral reasoning skills for solving socio-scientific problems related to science and technology by using socio-scientific issues (MEB, 2013). Science Curriculum was renewed in 2018, and it was aimed to develop students' reasoning ability, scientific thinking habits and decision-making skills by using socioscientific issues (MEB, 2018).

The importance of the research
Students who follow developments in science and technology, take part in the production, think innovatively, and have more knowledge and skills will have important roles in the business world in the future. Because of this importance, both our country and other countries have placed importance on STEM education and recently included it in their curricula. STEM education has recently become one of the most important research topics in Turkey and around the world (Ciftci, 2018;English, King & Smeed, 2016;Hare, 2017;Harris, 2018;Jolly, 2017;Judson, 2014;Meadows, 2018). It is observed that the studies have been carried out at almost all levels of education, from kindergarten to university. It is seen that studies in Turkey are mostly studies conducted with prospective teachers. It has been determined that the studies conducted on STEM have mostly concentrated on its aspects related to scale development, student/prospective teacher/teacher opinions on STEM, attitude towards science, motivation, academic success, the permanence of knowledge, creativity, transfer of knowledge, critical thinking skill, problem-solving skill, attitude towards STEM, integration of STEM, 21st century skills, success in mathematics, and STEM career (Akar, 2019;Akın, 2019;Atasoy, Tekbıyık & Yüca, 2019;Aydın and Karslı Baydere, 2019;Ciftci, 2018;Hebebci, 2019;Higde, 2018;Kurtulus, 2019;Yerlikaya, 2019).
In the field-specific skills section of the Middle School Science Curriculum of the Ministry of National Education (MoNE), the skills of analytical thinking, decision-making, creative thinking, entrepreneurship, and communication and teamwork were presented as "life skills", and innovative thinking as "engineering design skills" (MoNE, 2018). In this study, it was aimed to analyze the 21st century skills of students by integrating STEM education with socioscientific issues. As a result of the literature review, a limited number of studies that integrate STEM with socio-scientific issues and conducted with middle school students was found. Our study will contribute to the science education literature in terms of determining the effect of STEM activities integrated with socio-scientific issues on seventh grade students' 21st century skills.

The purpose of the study and the sub-problems
STEM education develops students' skills such as scientific thinking, research and inquiry, problem solving, critical thinking, creativity, and reflectivity (Sahin, Ayar & Adiguzel, 2014). Besides, with this education, it is aimed to provide students with 21st century skills (Akgunduz et al., 2015;Moore et al., 2016). The main purpose of this study was to investigate whether middle school students' developing STEM designs/activities in socio-scientific issues were effective in their gaining, developing, and maintaining 21st century skills. To this end, answers to the following questions and sub-questions were sought: (1) How do students' developing STEM activities in socio-scientific issues affect their 21st century skills?
• Is there a statistically significant difference between students' 21st century skills scale pre-test and post-test results? • Is there a statistically significant difference between the post-test and permanence test results of the students regarding the 21st century skills scale?
(2) Are socio-scientific STEM activities effective in the development of students' 21st century skills? (3) What are the students' perceptions of their gaining of 21st century skills?

Research model
In this study, "Nested Mixed Design" was used (Creswell & Plano-Clark, 2011). This is because this study was designed as a pretest-posttest single-group experimental study, qualitative data were collected during and after the application, and the collected data were combined and analyzed in the interpreting stage. In this study, in line with the nature of the case study method, interviews, field notes, and informal interviews were used as qualitative data collection tools. Research diagram of the study (Figure 1) is given below:

Study group
There were two seventh grade classes in the school where the study was conducted. The group with which the study would be conducted was determined by the simple random sampling method. The study was carried out with 16 seventh grade students. The study group included sixteen students (11 females and five males) (Table 1). It was determined that of the students, 68.7% (11 students) did not have a computer, 43.7% (7 students) did not have Internet connection at home, and 87.5% (14 students) did not receive any additional training (training center, a private lesson, and alike) during their education.
The fathers of three students were unemployed, four were building/building complex employees, four were artisans, one was a barber, two were workers, one was an accountant, and one was a civil servant, and the mothers of all students were housewives. Monthly incomes of the families of the students in the study group were found to be between a minimum of $138 and a maximum of $970 (In September 2017, one US Dollar was equal to 3.61 Turkish Liras). Based on this, it was observed that the socio-economic level of students in the study group was low. Besides, in terms of the academic success status, when the end-of-the-year achievement scores of the students for the 2016-2017 academic year were examined, it was determined that the scores of the students ranged between 47.95 and 94.09 points, and it was concluded that the class in which the study was conducted was heterogeneous in terms of academic achievement scores.

Context of the research
The study was conducted with middle school students who received education in a school located in the city of Van in the Eastern Anatolian region of Turkey. According to the "Ranking of the Socio-Economic Level of Development of Provinces and Regions Survey", this region ranks fifth and sixth among six categories of development level, and in terms of the index values, the city ranks 77th among 81 provinces of Turkey in terms of socio-economic development level (SEGE, 2019). The school where the study was conducted is located in a socio-culturally disadvantaged neighborhood in one of the worst-ranking provinces in terms of socio-economic development that is located in a region of Turkey where the development level is the lowest. The school has a prefabricated structure with a single floor. The school does not have a science lab, information technologies, and software classrooms, a visual arts workshop, a gymnasium, a music class, or a dining hall.

Implementation of the research
This study examining the effect of socio-scientific STEM activities on 21st century skills of students was conducted in the fall and spring semester (48 hours). The application process followed in the study is given Table 2. The socio-scientific subjects included in the study were selected from the subjects specified in the Middle School Science Education Program (MoNE, 2018). In the first week of the study, the students in the study group were informed about the study to be carried out for two class periods (40 min. + 40min.). The next week, study groups were formed. The opinions of the students were taken into consideration while the groups were being formed. In the same week, the 21st Century Skills Scale was applied to students as a pretest for two class periods.
Applications started the following week. The studies were applied on the Science Applications elective course, which was taught two classes a week. During the study, the students designed six STEM projects in total with three projects each semester. While the groups were making their designs, they followed the "engineering design process" proposed by Hynes et al. (2011). The design process for each socio-scientific issue was determined as four weeks. One of the designs that the groups made throughout the application by following the engineering design process (Wind Energy from wind-to-light) is presented below in Table 3. Week 4 2

First week
The teacher told the students that a study on "Wind Energy", which is a socio-scientific subject, would be conducted. All groups tried to get an idea about the design to be made for a class period with the tablets, mobile phones, and computers provided to them by the researcher.
They discussed how their design would help people in their daily lives, whether the design would be useful and whether it would be helpful. Later, they wrote down on a piece of paper the ideas of the design that they had decided on and gave it to their teacher. The teacher took design ideas from all groups and assessed the ideas he received with all the students. As a result, various alternatives were discussed in a design that could be used in daily life by making use of wind energy.

Second week
The groups had come up with ideas on how to create a design on the issue of wind energy in the previous week. All members of each group made their first drawings about what kind of design they would make. The drawings and ideas in all groups were evaluated and then when the teacher and students reached a common design idea on the issue "wind energy", and "wind-to-light" designs were decided to be made.
All groups listed the materials they would use in their designs and these materials were stated by the group members. The materials required for the "wind-to-light "designs that were planned to be made are given Table 4.

Third week
All groups put out the materials they brought for prototyping, and the necessary controls were made first by the group members and then by the teacher. For the missing materials, the group members consulted to the teacher and obtained lacking materials (Table 3). Later, all groups formed the prototypes of the design they would make. The process of making "Windto-Light" designs is given in detail in Table 5. Table 5. Stages of "wind-to-light" design 1-The groups started to create their designs. They firstly cut the wooden blocks (a) and glued the pieces together in an appropriate manner to create the base structure (b) of the design. a b 2-After creating the structure of the design, they made angle and length measurements and created propeller pieces from canisters and plastic bottles accordingly. Later, they assembled the propeller by running these parts through the plastic bottle caps at certain angles.
a b 3-After creating the propeller, they glued the led lamp through the hole opened through the center of the plastic bottle cap and attached the led lamp cap to the wooden block (a). Then they attached the DC motor to the upper part of the wooden block and attached the propeller they built earlier on the end of the DC motor using silicone (b). They established the electrical connection between the DC motor and led lamp and connected the circuit switch to control the current (c). a b c 4-Later, they rotated the propeller with the artificial wind created by the blow dryer, the DC motor moved thanks to the rotation of the propeller, electric current was created as a result of the motion of the DC motor, and thanks to the current, the led lamp emitted light. They also measured the amount of current coming from wind energy with a measuring device.
All groups created the "wind-to-light" designs following the steps above. After creating their designs, they tested them, and then each group came up to the board, in turn, operated the created design and presented it to the teacher and other groups.

Fourth week
In the last week of each design process, the groups reconsidered their draft and improved them, compensated for the deficiencies, and decided to revise their design. After all the necessary improvements, they finalized their "wind-to-light" designs ( Figure 2).

Figure 2. The final form of "wind-to-light" design
After all groups tested their designs, they preserved their designs in a safe place until the next week. In all these processes, the teacher provided the necessary consultancy to the groups. The groups benefited from science, mathematics, engineering, and technology in the process of creating their designs. In this activity, it was aimed to help the students comprehend the effectiveness of renewable energy sources, question recycling in terms of effective use of resources, realize the mutual interaction between the individual, the environment and society, raise an awareness of sustainable development in society, economy, and natural resources, develop reasoning ability, scientific thinking habits and decision-making skills using socioscientific issues, and improve innovative thinking skills. The STEM fields utilized by the groups and the work they carried out regarding these fields are given in Table 6. Table 6. STEM fields that groups utilized and their activities in these fields

Wind-to-Light
Science -Working on energy transformations.
-Creating light from wind energy.
-Using renewable energy sources.

Mathematics
-When creating a propeller from the canister, making length-width measurements of the parts that will form the propeller, and trying to place these parts inside the cover at a certain angle.
-Measuring length and width measurements using a ruler during cutting, gluing, and assembling of wooden blocks.

Engineering
-Making design drawings.

Data Collection Tools
21st Century Skills Scale The 21st Century Skills Scale developed by Kang, Kim, Kim, and You (2012) and adapted to Turkish by Karakas (2015) was used as the data collection tool. The scale has a 5point Likert-type rating system and 32 items under three sub-dimensions including Cognitive, Affective, and Socio-cultural dimensions. Internal consistency of the sub-dimensions were found to be .77, .70, and. .67, respectively (Karakas, 2015).

21st Century Skills Interview Form
This form was developed by the researchers, and with this form, the development of the 21st century skills of students was investigated. The 21st century skills proposed by the USbased organization called "Partnership for 21st Century Skills" were taken into account while deciding on the interview questions in this form. While creating the interview form, firstly, the literature on 21st century skills was reviewed P21, 2016; Battelle for Kids, 2019; European Commission, 2018; European Council, 2006;World Economic Forum, 2016), and as a result, a draft form containing 20 open-ended questions was created. The created draft form was sent for expert review (4 academicians, one science teacher and two graduate students in science education) and based on expert's evaluation, the final version of the interview form included 11 questions (Appendix 1) and three main categories, which are Learning and Innovation Skills, Information, Media, and Technology Skills, and Life and Career Skills.

Students' Perception of Gaining 21st Century Skills Form
This form was developed by the researchers. After the literature search on 21st century skills (P21, 2016; Battelle for Kids, 2019; European Commission, 2018; European Council, 2006;World Economic Forum, 2016), the skills in P21 were decided to be included in the form. These skills are Critical Thinking and Problem Solving, Creativity and Innovation, Communication and Cooperation, Information Literacy, Media Literacy, Technology Literacy, Flexibility and Harmony, Entrepreneurship and Self-management, Social and Intercultural Skills, Productivity and Responsibility, and Leadership and Responsibility. After the evaluation of two field experts who are academicians in science education department, the form was finalized. With this form, it was aimed to reveal the students' opinions about to what extent this application improved their 21st century skills. The students were asked to rate these skills from 1 to 5. Then, the arithmetic mean value of each skill was calculated and interpreted within the determined evaluation ranges.

Field Notes
While consulting the groups during the application process, the researcher tried to determine the difficulties the students faced during the application process by taking note of whether the groups were working in harmony, the level of communication and interaction between group members, how students handled difficulties, whether they were able to come up with creative ideas and designs and original/different/contrary solutions. The data obtained from the field notes were intended to reinforce the quantitative and qualitative data. Therefore, direct quotations from students assisted the researcher in the process of finding a solution to the research problem.

Informal Interview Notes
Since the researcher also worked as a teacher at the school where the study was conducted, he had the chance to spend time with the students in the study group on all weekdays and one day at the weekend. The researcher spent time with the students at the school entrance and departure times and during recesses, as well as in areas such as corridors and schoolyards, and had conversations with them about the practices. The researcher took notes when the opinions of the students included comments regarding 21st century skills.

Analysis of the data
The 21st Century Skills Scale was analyzed by using SPSS 18.0 package software. The qualitative data were analyzed through content analysis. The codes obtained were brought together within the context of pre-determined themes, and sub-themes were created. While the students' responses were being transferred to the computer environment, the students were given codes as S1, S2, and so on. Then, the second researcher independently re-examined the suitability of student opinions to the codes and themes created by the first researcher, and the inter-rater reliability between the coding of the two researchers were calculated according to the reliability formula proposed by Miles and Huberman (1994), and the reliability coefficient was found to be .88.
During the analysis of the "Students' Perception of Gaining 21st Century Skills", evaluation ranges were determined. The mean values of each skill were calculated and interpreted according to the evaluation ranges. Scoring ranges were used in the evaluation of the students' answers. When determining score ranges, the formula of Score Range = (Highest value-Lowest value) / 5 was used (Kucukali & Sercemeli, 2019), and the score ranges were calculated to be .80. Therefore, while interpreting the arithmetic means of skills, the mean value between 1.00 and 1.80 was determined to be "Very weak", between 1.81 and 2.60 to be "Weak", 2.61 and 3.40 to be "Medium", between 3.41 and 4.20 to be "Good", and between 4.21 and 5.00 to be "Very good."

Reliability, Validity, and Ethics Studies
Internal validity is defined as the extent of explanation of the changes observed in the dependent variable with the independent variable, and external validity as the extent of generalizability of the results to the large groups and population, where the subjects are selected (Buyukozturk et al., 2014). To increase internal validity, many data collection tools were used to demonstrate that the data supported or did not contradict each other (interviews, Perception of Gaining 21st century skills form, field notes, informal interview notes), and an independent expert other than the researcher was included in the analysis of the data. To increase the transferability of the study, the characteristics of the students were presented in detail and direct quotations from the opinions expressed by the students were provided. Besides, within the framework of the ethical aspect of the research, all necessary permissions were obtained from Istanbul University Social and Humanities Research Ethics Committee, and all legal permissions were obtained from Van National Education Directorate for the actual implementation. Before the application, the necessary permissions were obtained from the students and their legal guardians through the informed consent form.

Findings regarding the first sub-problem of the research
This section examines the changes in the students' 21st century skills during the implementation process. As the data collection tool, the 21st Century Skills Scale was used, and the findings were explained. In order to check whether the test scores showed a normal distribution, Skewness-Kurtosis, Kolmogorov-Smirnov Test, Shapiro-Wilk Test and histogram graphics methods were used. When the data showed normal distribution, related and unrelated t-test was used, when the data did not show normal distribution, Wilcoxon signed ranks were used for related measurements from non-parametric tests, and Mann Whitney-U test was used for unrelated measurements.
In the study, the 21st Century Skills Scale was applied to the students before and after the application as pretest and posttest. To determine whether there was a meaningful difference between the pretest and posttest scores of the participating students, the data were analyzed using the t-test. The findings are given in Table 7. When Table 6 was analyzed, it was seen that there was a statistically significant difference between the students' pretest and posttest scores in favor of the posttest (p<.05). Accordingly, it was concluded that socio-scientific STEM practice significantly improved the students' 21st century skills. To determine whether there was a significant difference between the 21st century skills scale pretest and posttest scores of the participating students in terms of the subdimensions of the scale, the related samples were analyzed using the t-test. The findings are given in Table 8. When Table 8 is analyzed, it can be seen that the difference between the level of significance between the averages of the students' pretest and posttest results in terms of the Cognitive, Affective and Sociocultural sub-dimensions was determined to be in favor of the posttests (p<.05). Based on these findings, it was concluded that socio-scientific STEM practice significantly improved all of the students' sub-dimension skills of the 21st Century Scale.
In the study, it was seen that there was a positive and significant change in the students' 21st century skills. However, it was necessary to determine whether the positive change was permanent. For this purpose, four (4) months after the study, the 21st Century Skills Scale was reapplied to the same students as a permanence test. The related samples were analyzed using the t-test. The findings are given in Table 9. When Table 9 is analyzed, it can be seen that there was no statistically significant difference between the students' posttest and permanence test scores (p<.05). From this point of view, it was concluded that socio-scientific STEM practice permanently improved the students' 21st century skills.
It was also examined whether there was a significant difference between the posttest and permanence test scores of the students' 21st Century Skills scale in terms of the sub-dimensions. The t-test scores at the significance level of .05 showed that in terms of Cognitive, Affective and Sociocultural sub-dimensions of the scale, there was no statistically significant difference between the students' posttest and permanence test scores (p<.05). Based on these findings, it was concluded that socio-scientific STEM practice permanently improved the students' skills that are within the scope of Cognitive, Affective and Sociocultural sub-dimensions of the 21st century skills scale.

Findings regarding the second sub-problem of the research
In this section, the opinions of the students regarding 21st century skills after the socioscientific STEM application and the findings obtained from these opinions are explained. After the analysis of the opinions of the students, various codes were created, and these codes were presented under pre-determined themes and sub-themes. The frequency and percentage values of the sub-themes and codes included under the theme of "Learning and Innovation Skills" are given in the table below: In Table 10, it was determined that all of the students used Creativity and Innovation Skills; and 11 students used Critical Thinking and Problem Solving, Communication, and Cooperation Skills.
As a result of the examination of the students' opinions within the context of the sub-theme of Critical Thinking and Problem Solving, the codes of "having difficulties", "overcoming through cooperation", "getting help from the teacher", "gradual adaptation", and "reasoning" were created. Six of the students stated that they had difficulties in the design process. Five of the students thought that they coped with these difficulties by cooperating with their group mates. Some students' opinions that contributed to the creation of these codes are given below: "Yes, there was a lack of material. The design did not work. So, we worked in cooperation with our group members and overcame the problems." (S2) "Of course, there were difficulties. For example, we had some difficulties in the design process and of course, we solved these problems by talking to each other." (S7) "We had difficulties only at the beginning. We had a difficult time since we designed a product for the first time. But then we got used to it in a gradual fashion and we were very successful.

We came up with a very good invention." (S9)
Also, the field notes of the researcher on 01.03.2018, which read "It was a little difficult to disassemble the wireless doorbell without breaking it, to open the remote controller without breaking it, and to place the devices. The groups shared tasks within themselves, and it was observed that they acted in cooperation and created their design better" support the findings above.
As a result of the examination of the students' opinions within the sub-theme of Creativity and Innovation, all of the students expressing their opinions thought that the application developed their creativity. Some students' opinions that contributed to the creation of these codes are given below: "Yes, because before I created a design, I found inventions, tools, and product-making boring; but, after creating my design, I have become more interested in them." (S2) "Yes, because one gets used to designing and tries to develop the design. Then, as the design is developed more, so does creativity and a very nice product occur." (S9) "Of course, I do. For example, I did not know how to make a paper plane. But the designs and products we made helped a lot with my improvement. Now I know how to make a paper plane." (S16) As a result of the examination of students' opinions within the sub-theme of Communication, the code of "exchange of ideas" was created. Almost all of the students (11 of them) stated that they were in contact with their friends during the application process and exchanged ideas when necessary. Some students' opinions that contributed to the creation of this code are given below: "Yes. My friends and I came up with very good ideas. We came up with good ideas and created beautiful activities and designs." (S1) "Yes, we exchanged ideas. It had a great effect on producing better products. For example, they thought of what I couldn't think of, I thought of what they couldn't think of. Thus, we made up for each other's shortcomings." (S7) As a result of the examination of the students' opinions that are within the context of the subtheme of "cooperation", the codes of "working together" and "complementing each other" were created. Six of the students thought that they created products by working together with their group mates, and five thought that they made up for each other's shortcomings in the process. Some students' opinions that contributed to the creation of this code are given below: "Yes. We were a group and we tried to create a better product together. We showed success with this product we created." (S3)

"Of course. Because I was very happy with my group. If I had an idea about something, I would share it and if it was well-liked, we would implement that idea. If it was not liked, we would discuss why it was not. Then we would choose and apply the best of other ideas. Thus, higher quality ideas and products were created." (S14)
"Yes, working together produced very good results. We created different good inventions together." (S15) Also, the field notes the researcher took on 03.05.2018, which read "It was observed that while creating the space shuttle, there was a serious task-sharing within the group, that their communication and interactions were good, that they constantly exchanged ideas about what to do and how to act until the design was completed" support the findings on the "communication" and "cooperation" sub-themes.
The skills put forward within the scope of "Information, Media and Technology Skills" are "Information literacy", "Media literacy", and "Technology literacy". This category and subcategories were considered as themes and sub-themes. After the examination of the opinions of the students, various codes were created, and these codes were presented under predetermined themes and sub-themes. The frequency and percentage values of the sub-themes and codes included in the theme of "Information, Media, and Technology Skills" are given in the table below: In Table 11, the sub-themes of "information literacy", "media literacy", and "technology literacy" are presented under the theme of "Information, Media, and Technology Skills". It was determined that all of the students participating in the interview used skills related to information literacy, seven of them related to media literacy, and six of them related technology literacy.
As a result of the examination of the students' opinions that are within the context of the subtheme of "information literacy", the codes of "New information", "Global warming", "Technology", "Domestic waste", "Science", "Solar energy", "Mathematics", "Recycling", "Space", "Engineering", "Reflection", and "Air pressure" were created. All of the students (12) stated that they gained new information thanks to the application. Eight of the students stated that they gained new knowledge on global warming, namely its effect on our climate change and our role in how to eliminate it, seven on technology and domestic wastes, six on science, five on solar energy, four on mathematics, recycling and space, and one on air pressure. Besides, one student expressed his/her opinion about transferring what s/he learned to his/her real life and using it (reflection). Some students' opinions that contributed to the creation of these codes are given below: The informal interview notes the researcher took on 30.03.2018, which read "In the conversations I had separately with S6, S7, S9, and S16, they said that they learned a lot about global warming," and on 05.01.2018, which read "During the short, small talk I had with S3 and S12, they both said they learned a lot thanks to the activities we did in the science practices lesson" support these findings.
As a result of the examination of the students' opinions that are within the context of the subtheme of "media literacy", the codes of "Internet", "YouTube", and "Short film" were created. Five of the students stated that they used the Internet during their research. Besides, one of the students stated that s/he tried to shoot a short film, and another student stated that s/he started to prepare to open a channel on YouTube, a popular video uploading platform. Some students' opinions that contributed to the creation of these codes are given below: "Yes, I did. I searched for global warming on the Internet. I was able to get the information I never had via the Internet. This research was very helpful to me." (S16) "I think it was a very nice activity. After this activity, I tried to make short films with the computer, but I was not exactly able to. Then I tried again and succeeded." (S9) "I think this activity was very nice. I think it taught valuable lessons to both our teacher and my friends. I have been using the application I used in this activity for my YouTube channel." (S12) The informal interview notes the researcher took on 29.03.2017, which read, "S12 told me during the break that s/he had made videos similar to one we made in our activity and uploaded them on his/her YouTube channel," support this finding.
As a result of the examination of the students' opinions that are within the context of the subtheme of "technology literacy", the codes of "Digital storytelling", "Global warming", "Technology", and "Computer" were created. Five of the students thought that thanks to the activity of making a digital story, they developed an interest in technology, four became sensitive to global warming thanks to this activity and gained information about it, two of them used technology, and one of them created various film works using a computer. Some students' opinions that contributed to the creation of these codes are given below: "About digital storytelling, I think we did it as a warning to people against bad habits and environmental damage that results in the emission of greenhouse gasses, and consequently, the emergence of global warming." ( Under the category of "Life and Career Skills" are "Flexibility and Harmony", "Entrepreneurship and Self-management", "Social and Intercultural skills", "Productivity and Responsibility", and "Leadership and Responsibility" skills. This category and subcategories were considered as themes and sub-themes. After the examination of the opinions of the students, various codes were created, and these codes were presented under pre-determined themes and sub-themes. The frequency and percentage values of the sub-themes and codes included in the theme of "Life and Career Skills" are given in the table below: In Table 12, the sub-themes of "flexibility and harmony", "entrepreneurship and selfmanagement", "social and intercultural skills", "productivity and responsibility", and "leadership and responsibility" are presented under the theme of "Life and Career Skills". It was determined that eleven of the students participating in the interview used skills related to flexibility and harmony, social and intercultural skills, and productivity and responsibility, four of them used entrepreneurship and self-management skills, and seven used leadership and responsibility skills. As a result of the examination of the students' opinions that are within the context of the subtheme of "Flexibility and Harmony", the codes of "Group work", "Harmony", "Teacher", and "Reflection" were created. Seven of the students who participated in the interview stated that they made projects together with their group mates, three worked in harmony with their group mates, two received support from their group mates and teachers when they faced difficulties, and one transferred (reflected) what s/he learned to the next project. Some students' opinions that contributed to the creation of these codes are given below: "I got along well with my group mates. And I did not have difficulty in making new designs after the designs we made. Because one gets used to it." (S9) "Of course, I was struggling a bit because I was making that design for the first time. But I have been making designs for a long time, so I grasped it immediately because my dexterity was good" (S12) As a result of the examination of the students' opinions that are within the context of the subtheme of "entrepreneurship and self-management", the codes of "introduction of the project", and "voice recording" were created. Two of the students who participated in the interview stated that they introduced the designs they made to the other students at the school and one of the students aimed to convey the product s/he made to more people through the digital story (with a voice recording) s/he created. Some students' opinions that contributed to the creation of these codes are given below: "To build and operate the product we designed outside caught the attention of all students." "Yes. We brought the necessary materials. We took our project and showed it to everyone. It was very well-liked." (S15) "Yes, I have. For example, we recorded voice during the digital story creating activity to announce to people." (S16) As a result of the examination of the students' opinions that are within the context of the subtheme of "social and intercultural skills", the codes of "Working with the group", "Getting along", "Communication", and "Exchange of ideas" were created. Six of the students participating in the interview stated that they made their designs by working with the group and acting as a group, two of them had good relations with their group mates and they had good intra-group communication, and one of them exchanged ideas with his/her friends during the product creation process. Some students' opinions that contributed to the creation of these codes are given below: "We had some problems with our group mates at first. Because they did not make much effort. But after I warned them, their performance got better. I think it is better to work as a group and it suits me better." (S14) As a result of the examination of the students' opinions that are within the context of the subtheme of "productivity and responsibility", the codes of "Effort", "For human benefit", "Remote control with a bell" were created. Five of the students participating in the interview expressed opinions that they spent their labor (effort) while making a product, four reported that they made a remote control with a bell and it was for the benefit of people, three said that they took responsibility during the application, and one expressed his/her opinions about solar energy and digital story creation. Some students' opinions that contributed to the creation of these codes are given below: "Yes, for example, the remote control with a bell. Some people lose their remote controls, so we designed a remote controller with a bell to address this problem." (S3) "We used solar energy. The biggest problem in our country is the price of electricity. So, let's use solar energy." (S8) "Every product we tried to make in technology design classes was for the benefit of humanity, and I took great responsibilities." (S14) Besides, the field notes the researcher took on 14.12.2017, which read "It was observed that the problem situations that the students put forth about solar energy and the designs they wanted to create were beneficial for humanity and had the features that would facilitate the daily life," and on 19.04.2018, which read "The groups shared materials and distributed tasks in the process to create their designs. In the examinations made today, it was seen that all the group members fulfilled their duties and performed with great responsibility while making their designs," support these findings.
As a result of examining the opinions of the students in the sub-theme of "Leadership and Responsibility", codes such as "Leadership", "Problem-solving" and "Responsibility" were created. Five of the students who participated in the interview stated that they led their groups, four of them tried to solve the problems in the group, and three of them took various responsibilities in the group. Some students' opinions that contributed to the creation of these codes are given below: "I led my group. I tried to solve the problems in the group as much as I can. Because I took responsibility for it. I believed I could fulfill this responsibility." (S6) "No, I did not lead. But I took responsibility and tried to solve the problems in the group." (S9)

Findings related to the third sub-problem of the research
In this section, the data obtained using "Students' Perception of Gaining 21st Century Skills Form" have been explained. This form contains eleven skills proposed by P21. In the form, the students were asked, "Which of your skills do you think the application improved and how much? Can you give a score from 1 to 5?" The students' answers were analyzed using descriptive analysis methods. The analysis of the obtained data is given below: When Table 13 is analyzed, it can be seen that the arithmetic mean of 21st century skills varies between 3.0 and 4.25. It was determined that the students thought that the application developed their "creativity and innovation" skills the most with an arithmetic mean of 4.25. Besides, the field notes the researcher took on 03.01.2018, which read "In my conversation with a few students in the schoolyard, I realized that they produced different, original, and unique ideas about the work to be done in tomorrow's practice. For example, I found that a student developed different perspectives on how to make the handle of the fruit saw they would make, which materials would be more useful," and on 22.02.2018, which read "Their idea of making remote controls with bells is different and original and so are the prototypes they have created for their designs," support this finding.
When the rating in the form is evaluated, it can be seen that the social and intercultural skills (avg. 3.08), "entrepreneurship and self-management "(avg. 3), and "leadership and responsibility" (avg. 3.25) skills are at the "medium" level; "critical thinking and problemsolving" (avg. 3.83), "communication and collaboration" (avg. 3.83), "information literacy" (avg.3.66), "media literacy" (avg. 3.58), "technology literacy" (avg. 3.83), "flexibility and harmony" (avg. 3.75), and "productivity and responsibility" (avg. 3.91) skills are at the "good" level, and the "creativity and innovation" (avg. 4.25) skill is at the "very good" level. Based on this, it was concluded that the socio-scientific STEM application had a positive effect on the students' 21st century skills.

Discussion and Conclusion
In this study, it was attempted to show that the STEM education integrated to scientific issues that we believe to have social effects such as recycling/domestic wastes, wind energy/motion energy, solar energy, the benefits/harms of technology and its use in daily life, global warming, space studies/life in space/space pollution effectively helped to gain the 21st century skills that the people of our era should have and permanently improved these skills that people have. According to the results of the study, it was concluded that the socio-scientific STEM application performed for 24 weeks positively affected the 21st century skills of the students.
In the research, the 21st Century Skills Scale was applied to the study group before and after the application, and there was a statistically significant difference in favor of the posttest. Also, there was no statistically significant difference between post-test and the permanence test. Besides, it was observed that there was a statistically significant difference in terms of all subdimensions of the 21st-Century Skills Scale in favor of the posttest. Based on this, it was concluded that socio-scientific STEM practice positively affected all of the skills that are within the scope of the cognitive, affective, and sociocultural sub-dimensions of the 21st century skills scale. In his study he conducted with middle school students, Karakas (2015) found that the level of the students' skills within the context of the cognitive, affective, and sociocultural dimensions of 21st century skills was high. In their study, Sahin, Ayar, and Adiguzel (2014) showed that after-school STEM activities could contribute to the development of the 21st century skills of the students. In their study with seventh-grade students, Aydin and Karsli Baydere (2019) stated that STEM activities contributed to the students' 21st century skills such as collaboration, critical thinking, problem-solving, creativity, and self-confidence. The findings obtained in these studies are similar to the findings we obtained in our research.
As a result of the analyses, it was observed that there was a positive change in the students' 21st century skills, but it was necessary to determine whether the positive change that occurred was temporary. Accordingly, four months after the study, the 21st Century Skills Scale was readministered to the students in the study group as a permanence test. As a result of the analyses, no statistically significant difference was determined between the scores of posttest and permanence test of the students in the study group. From this point of view, it could be concluded that the socio-scientific STEM application permanently improved the students' 21st century skills.
At the end of socio-scientific STEM application, the students' thoughts and opinions were examined in the context of 21st century skills proposed by P21. In this context, in terms of the theme Learning and Innovation Skills, all students stated that they used their creativity and innovation skills. As for the opinions of the students on the Critical Thinking and Problem Solving sub-theme, the students stated that they had difficulties from time to time during the activities, but they overcame these difficulties by working with their friends, asked for help from the teacher and other friends when they had difficulties, used reasoning while solving problems, and coping with these difficulties also had a positive effect on their adaptation to the activities. Also, the results showed that the application improved the creativity of the students and contributed positively to their individual development, quick thinking, desire to create products, problem-solving skills, handcrafting experience, communication skills, cooperation skills and their socialization skills. From this point of view, it was concluded that socioscientific STEM activities had a positive effect on the development of the students' learning and innovation skills, critical thinking and problem solving, creativity and innovation, and communication and collaboration skills.
After the examination of the students' opinions on the theme of "information, media, and technology skills", the results showed that students started to use some reliable websites, made videos and short films, and uploaded them to the Internet. Also, students learned new information during the application process, especially on global warming, the use of technology in daily life, domestic waste, science, solar energy, mathematics, recycling, space, engineering, and air pressure. From this point of view, it was concluded that the socio-scientific STEM application had a positive effect on the development of information literacy, media literacy, and technology literacy skills of the students.
For the theme of "life and career skills", the students stated that they utilized flexibility and harmony, social and intercultural skills, productivity, entrepreneurship and self-management, and responsibility skills. From this perspective, it was concluded that the socio-scientific STEM application contributed positively to the development of the students' flexibility and harmony, entrepreneurship and self-management, social and intercultural skills, productivity and responsibility, and leadership and responsibility skills. In a study with prospective teachers, Murat (2018) concluded that in terms of their competence, the prospective teachers "mostly" agreed with the learning and innovation skills, life and career skills, and information, media, and technology skills dimensions of 21st century skills. Tunkham, Donpudsa and Dornbundit (2016) stated in their study with high school students that STEM activities significantly affected the students' learning and innovation skills, their knowledge, media and technology skills, and their life and career skills in a very positive manner.
After examining the opinions of the students, it was determined that the application improved the students' creativity, innovation, self-management, responsibility, critical thinking problem solving, communication and cooperation skills. Based on this, it was concluded that the socioscientific STEM application had a positive effect on the students' 21st century skills. Chapoo (2019) stated that 21st century skills such as digital-age literacy, creative thinking, effective communication, and productivity of the students participating in integrated STEM activities were positively affected. Also, Anjarsari, Prasetyo and Susanti (2020) reported that STEMbased science learning had a positive effect on the students' technology literacy skills. In this study, we revealed that the STEM activities integrated with socio-scientific issues helped the students gain 21st century skills permanently.
This study was carried out with seventh-grade middle school students. In new studies to be conducted, how socio-scientific STEM applications affect the 21st century skills of students in all grade levels from preschool to university can be investigated. Since socio-scientific STEM activities are effective in getting students to gain 21st century skills permanently, it is thought that it would be beneficial to extend this method to educational institutions.