Content validity and confirmatory factor analysis of Cooperative Learning Attitude Scale (CLAS) for the EFL students

This study reports on the development of a cooperative learning attitude scale for the EFL students based on expert reviews and examines its construct validity. Three rounds of expert reviews were conducted. In each round, five content experts reviewed the items and rated their relevance to the content domain and their comprehensibility. The expert panel also assigned the items to the suggested sub-scales. After item elimination and major revisions, 45 items representing five factors with perfect reviewer consensus were retained. Although the scale was developed via expert review, the authors checked the construct validity of the instrument through CFA by collecting data from 136 pre-service EFL teachers. The results suggested elimination of six more items owing to their insignificant t values or low factor loadings, which were eliminated after two reviewers recommended to. The resultant fit statistics supported a good-fitting model with t statistics and item-total correlation significant across all variables. The coefficient alpha confirmed internal consistency of the scale and that of sub-scales.


Introduction
'Language acquisition' is vastly being highlighted as a significant alternative to the 'language learning' in foreign/second language education. There is a distinction between the two terms. That is, learning a language is seen as a deliberate and intentional process, while its acquisition is considered to happen in the opposite way. It is argued that the language acquisition takes place unintentionally by means of frequent social interactions (Mitchell et al., 2013). If a language is acquired it comes automatically when needed, but if it is learned consciously then there will not be this automaticity and the learners ought to verify what they want to utter in their mind first to check for accuracy (Harmer, 2007). Children, for example, exhibit an astounding level of aptitude in acquiring more languages in addition to their mother tongue without actually being taught about the forms or vocabulary but acquire them through communication in a particular social environment. However, Harmer argues that the language acquisition process weakens by age growth. Nevertheless, getting the right input in stress-free learning atmosphere individuals can acquire a language even in their adolescence. This can be made possible by the amount of "...exposure to it, motivation to communicate with it and opportunities to use it" (Harmer, 2007, p. 47).
According to Ur (2012), the second languages could be acquired intuitively in the same way as the native languages are acquired through repetitive exposure to social interaction and communication with others. Truly, "[to] learn a second language is to learn a skill" and it requires extensive practice (McLaughlin, 1987;cited in Pienemann, 2003, p. 681). In some cases, if the teachers only focus on acquisition and always give it a priority, students will remain uninformed about the structures and functionality of the target language, especially in adult language education. Students require explicit teaching of the structures and forms of the language by the teacher. Here it is that they have to learn consciously about the language structures (Mitchell et al., 2013). However, the focus of the language acquisition theorists is on the language learners to acquire their target language through social interactions; even the grammar of that language alongside its respective language skills. In fact, "language is social -a social practice, a social accomplishment, a social tool" and thereby must be acquired through social interactions (Atkinson, 2002, p. 526).
From a cognitive theory perspective, language acquisition is seen as an internal and mental process. Cognitive theorists believe that human beings come with innate language acquisition abilities. To them, language acquisition occurs by means of social interactions, which, in turn, will facilitate the internalization and discernment of the linguistic features even in L2. Moreover, the cognitive theory is considered as an underlying theory for the other most important emerging theories in second language acquisition research area (Atkinson, 2011). Here, sociocultural theory to language acquisition seems to meet the purpose of the L2 acquisition in college education.
Sociocultural theory focuses on the students' exposure in in-class social interactions recommending their active engagement in certain activities. No doubt, it has a profound impact on their L2 development thereafter. A distinguishing trait of this theory from the other alternatives is that "...it does not assume that acquisition is a universal process" (Lantolf, 2011, p 43). It sees the L2 acquisition process differently according to the various social and cultural settings. The sociocultural theory also encourages direct explanation of various 'linguistic features' of the target language, so that the students can apply them in their communicative activities. Language learners' mediating characteristics in language acquisition process might be different. For instance, students may later apply the explicitly learned grammar implicitly for their communicative needs in their own ways based on their capabilities in mediating different linguistic features. Gillies and Boyle (2008) report that "by encouraging students to share their ideas with their peers, clarify their misconceptions, and work together to construct new understandings, the teacher…" provides them "mediated-learning" opportunities when they are working in cooperation with each other (p. 1346). Pawlak (2011) calls this process "instructed foreign language acquisition". He contends that the foreign language acquisition is limited to the classroom teaching and the students get little exposure to the language practice in their surroundings outside the classroom context. As noted elsewhere, the foreign language students need plentiful exposure to the natural target language practice in cooperation with their peers within the class through the guidance of the language instructor. The use of cooperative learning in language acquisition is elaborated in more details beneath in the literature review.

Cooperative learning and language acquisition
Cooperative learning (CL) comes with a variety of methods into effect as an effective approach in foreign/second language acquisition seemingly serving the purpose of sociocultural learning theory, where the students both receive input from the instructors as well as interact within cooperative groups to enhance one another's learning. According to Slavin (1991), CL "supplements the teacher's instruction by giving students an opportunity to discuss information or practice skills originally presented by the teacher", while sometimes they are also required to locate the sources of information by their own efforts to have contributed to the accomplishment of their shared cooperative tasks (p. 73). This is exactly in congruence with the discussion made above as regards the implication of sociocultural theory in an L2 acquisition context.
Keeping the contextual difference of foreign language education in mind from that of native languages, the CL methods could provide a stimulatory simulative learning environment where the learners get encouraged in actual social interaction to promote their language skills in cooperation with each other by building small communities of learners. So doing, they can benefit from the comprehensive inputs they receive through teacher mediated opportunities to communicate with and use the target language during the collaborative activities with their peers. Learners' interaction within a group of L2 learners is, in fact, a sort of simulation in the target language community that they will interact someday and it plays as a motive for them to be persuaded to have full participation in communicative activities in the class (Norton & McKinney, 2011). Kagan and Kagan (2009) maintain that besides developing social and higher-order thinking skills, CL serves the purpose of language acquisition. They further argue that in a well-tailored CL environment all the students with no exception get enough exposure to the target language practice by using it in self-disclosure, exchanging ideas and negotiating. This way, students in all teams in the class will be improving their listening and communicative language skills. As a matter of fact, "students learning a new language are more willing to participate and preserve in a cooperative versus competitive setting" especially in college education (Kagan & Kagan, 2009, p. 3.6). Besides, CL plays a crucial role in increasing self-esteem, and motivation, while also decreasing student language anxiety.
In addition, CL puts an incremental positive impact over the academic achievement of the students compared to the competitive or individualistic approaches to learning irrespective to the grade level or subject area (Johnson, Johnson, and Stanne, 2000;Slavin, 1996). The achievement of the students in CL groups is interrelated with the level of motivation students get while working on accomplishing a shared goal with their peers in small groups. In CL activities, students are required to be responsible for their learning and that of others in their group (Slavin, 1991(Slavin, , 1996. As a motivational attribute, Slavin (1996) emphasizes the presence of group rewards or stimuli like scores on student individual performance in the quizzes. This will also play an important role in student cohesiveness in cooperative group works. Contrarily, Kagan and Kagan (2009) are against the group or individual rewards of such kinds and believe that the motivation will be temporary in such cases. When the rewards diminish, the motivation will also diminish. Instead, they suggest immediate praising and constructive feedback to acknowledge students efforts towards their own learning and that of their teammates while the cooperative group task is running. Likewise, Baer (2003) maintains that rewards may work in school level, but will not have any impact on college students in tertiary level CL activities.

2.2.Cooperative group design
Cooperative group works are different from normal groups where students do not feel connected to each other and are unmotivated. In cooperative learning, students are required to work in small groups to accomplish a common group goal. They have to help each other learn an intended content knowledge through discussions ensuring the active contribution and hard work of every single member of the group in the process (Johnson & Johnson, 1999). CL tasks must be completed by exerting efforts both in an individual and group level (Schul, 2011).
The cooperative groups are formed on heterogeneity basis, where students with differences "in skill level, interest, motivation, experience, and family backgrounds" work together (Apple, 2006, p. 279). They invest in each other's learning and feel proud of having a positive role in the success of their group members while also believe in their own potentials in carrying out the tasks they are assigned to (Johnson & Johnson, 2003). Meanwhile, the students' equal participation in teamwork irrespective of differences in terms of social rank and potential aptitudes is of primary importance in CL (Apple, 2006). So doing, they will learn to see things from different viewpoints, build s strong rapport with one another, learn to work with others, and start being optimistic as regards their learning process (Favor, 2012).
One more thing that is worth to mention is that the students are better aware of one another's problems than the teacher is with the new learning material they are dealing with and can better help each other understand thereafter (Gillies & Boyle, 2008). Likewise, their contribution will increase, when they know their standpoints in group discussions will be treasured by their peers.
The elements of successful cooperative group work are five as suggested by Johnson (1999, 2002): Positive Interdependence: students must believe that their success is interrelated to the success of the entire group. The task they are required to accomplish cooperatively is based on a common goal equally and mutually benefits everyone in the group.
Individual Accountability: to ensure that everyone in the groups has improved during the cooperative learning, it necessitates individual assessment that, in turn, will cultivate a sense of individual responsibility in students to contribute to a cooperative shared task; especially when the result of the assessment is shared with the group or the individual students.
Face-to-face promotive interaction: is referred to the contribution of the students to the promotion of one another's achievement "by helping, assisting, supporting, encouraging, and praising each other's efforts" (Johnson & Johnson, 1999, p. 71). When working together, they can improve the way of thinking and interpreting skills of one another alongside their social or personal relations.
Social skills: CL activities necessitate students to exhibit a number of social and interpersonal skills to deal with cooperative tasks. If they lack such skills, they have to be taught. Just placing them in groups to work in cooperation with each other does not help. Being more social leads to more interaction and better relationships among the students. This could be improved by assigning them different roles during the CL activities.
Group processing: it refers to the students' reflections over the successes and failures they experience during the CL activities and identifying behaviors which need reinforcement to im-prove or need to be changed by discussing within their groups to increase the efficacy of their contributions to accomplish a shared goal.
To sum up, in spite of differences all of the cooperative learning methods and structures share one trait in common, that is, "having students work in small groups or teams to learn academic material" (Slavin, 1991, p.71) while taking the five CL elements into consideration in the process as briefed above. As discussed elsewhere the CL methods provide the students enough opportunities to engage in extensive social interactions, which will contribute a lot in their learning especially in language acquisition. Although there are controversies about the contextual problem of foreign language education, which might limit the language acquisition process, again it is believed that when students get the right input and exposure to the target language will benefit them a lot.

The present study and its significance
The main purpose of this study was to develop a "Cooperative Learning Attitude Scale (CLAS) for the EFL Students" through a content validity study based on the expert review. However, the authors tended to examine its construct validity through confirmatory factor analysis (CFA). The researchers did not come across any measurement tool on EFL students' attitudes towards the use of cooperative learning in their language learning process especially with a focus on EFL acquisition. Attitudes are a series of beliefs and ways of judgment towards something, which, in turn, will shape the behaviors of an individual. It has a direct relation with the students' motivation and if they hold a 'negative attitude', it will affect their achievement level. Discovering the attitudes of the students about learning strategies will allow discovering which instructional approaches are appropriate and helpful to them during the teaching and learning process (Mentz and Van Zyle, 2016).
Besides, the scale development method used in this study might be the first of its kind in language education with the specifications elaborated above. Content validity studies based on expert review for keeping or discarding items in a measurement tool is widely used in nursing education (Pilot & Beck, p. 2006). The researchers thought that this would be a good approach to develop measurement tools for other fields of education, too; EFL education in the current study.

3.1.Research design
This study used a content validity approach to measurement tool development in accordance to the reviews of a group of content expert panel and their consensus over the representativeness of the variables and their significance to the content domain (Rubio, Berg-Weger, Tebb, Lee, & Rauch, 2003;Ayre & Scally, 2014;Zamanzadeh et al., 2015). Pilot, Beck, and Owen (2007) argue that "Content validity concerns the degree to which a scale has an appropriate sample of items to represent the construct of interest…whether the domain of content for the construct is adequately represented by the items" (p. 459).
There is a variety of ways to the quantification of content validity and/or computing the agreement level of the content experts with regard to the relevance of items in a scale (Pilot and Beck, 2006;Lynn, 1986). However, the content validity index (CVI), which is extensively used in nursing education, was adopted in this particular study seemingly being its first example in language education. As a supplementary method to calculating the consensus level of the raters, a modified kappa statistical test was also conducted which is elaborated later on (Pilot and Beck, 2006).

3.2.Procedure
Content validity study is carried out in two stages of development and judgment in general. The development stage is qualitative-based with three sub-steps, where the content domain is identified, the scale items are generated, and put into expert review form. In the judgment stage, the relevance and the clarity of the items are judged by a group of content experts to identify valid items. Then their CVI will be computed to quantify the judgment of the expert panel (Lynn, 1986). A similar procedure was followed in this study as elaborated below:

3.3.Developing the items
First of all, an extensive literature review was carried out on cooperative learning and language acquisition in order to identify the representative variables as relevant to the respective domains. At the initial stages of the study, 113 items were generated with a direct focus on English as a foreign language (EFL) and language acquisition. Then all the items were screened for identifying the similar as well as unrelated items by the researchers. As a result, 32 items of such kinds were found and excluded from the study. After doing major revisions and rewording, the remaining 81 items were put into an expert review form (Lynn, 1986). This form required the content experts to rate the relevancy and clarity level of each item on a four-point rating scale from 1-4, with 4 reflecting perfect relevance of the items in the relevancy box and very clear in the clarity box (e.g. see : Lynn, 1985;Pilot and Beck, 2006;Pilot et al., 2007, Zamanzadeh et al., 2015. Besides providing the required written instructions on how to rate the items, the experts were requested to check the items for their accuracy and make corrections where necessary. In addition, they were asked to write down the possible summarizing sub-scales of several variables; each item considered as a distinct variable.

Selecting the expert panel and the sample
Nine different content experts participated in the study. Each of them had substantial skills in using CL in higher education and had sufficient EFL skills. Four of them had expertise in EFL teaching while the rest taught other fields. Eight of them had Ph.D. in curriculum development and instruction and one in assessment and evaluation. Of these experts, two of took participation in all three rounds of the expert-review. In the first round, nine experts were selected in total. Six expert review forms were handed over in person, while three copies were sent by email. Of nine forms, only five were returned which is an acceptable number for a content validity study based on the experts review. According to Lynn (1985), the number of content experts should not be less than three. However, preferring a panel of minimum five experts prevents the occurrence of chance agreement problem. In the second round of expert review, again a panel of five content experts participated (two of them being new members) to rate the new set of revised items after the removal of the irrelevant ones. The third round of review required the content experts to assign the items to the given sub-factors suggested in the first two rounds of the expert reviews to form the final form of the scale. As usual, five reviewers, including two new members, participated in the final stage of the expert review, too.
After the completion of expert-review procedure, the last version of the scale was responded by 136 pre-service EFL teachers in Turkey, including sophomores, juniors, and seniors. The freshmen were not included in the study since they did not have much experience in CL activities in EFL education. The collected data from these students was used for the CFA.

Data analysis
The collected data in the first two rounds of the study were entered into two content validity index (CVI) calculation tables in each round of expert review; one for calculating the CVI for the relevancy of the items and another for their clarity. First, item-level CVI (I-CVI) for the relevancy of all the items were calculated by counting the number of the experts who give a rating of 3 or 4 divided by the total number of the content experts. The items having an I-CVI of '1' were retained while the ones under that floor were excluded. According to Lynn (1986), if the number of the content experts is five or less than that the CVI should be '1', that is, everyone in the panel should agree for an item to be retained. However, Pilot et al. (2007) maintain, "an I-CVI of 0.78 or higher for three or more experts could be considered evidence of good content validity" (p. 459). Moreover, the scale level CVI (S-CVI) of the items were calculated using two approaches for both the relevancy and the clarity of the items, that is, by averaging the S-CVI (S-CVI/AV) by dividing the sum of I-CVI to the total number of the items and the universal agreement S-CVI (S-CVI/UA) was calculated by dividing the total of items with an I-CVI of '1' to the total number of the items. A minimum 0.80 for both S-CVI/AV and S-CVI/UA is considered acceptable, while a value of 0.90 or over would be excellent (Pilot et al., 2007).
To control the chance agreement issue, which CVI does not, modified kappa tests were run for each item. To do this, first "the probability of chance agreement was" calculated using the formula presented in Pilot et al. (2007, p. 466) and given in a fitting format here in Zamanzadeh et al. (2015, p. 69) as "Pc = [N! / A! (N -A)!]*.5 N ". N indicates the number of content experts, A the number of experts confirming the relevance of items to the content area by rating them as 3 or 4. Then the modified kappa was calculated using "K = (I-CVI -Pc) / (1 -Pc)" formula (Zamanzadeh et al., 2015, p. 69;Pilot et al., 2007). Full kappa agreement value equals to '1' while the chance agreement equals to zero (Viera and Garrett, 2005). In this study, the perfect modified kappa (k * =1) was preferred as an acceptable level as was in I-CVI.
To examine whether the scale developed based on expert reviews support the construct validity, a correlated traits model of CFA was performed via LISREL 8.71. Prior to CFA, the normality and linearity assumptions were ensured. The VIF (< 5) and Tolerance (> 0.20) values indicated the absence of singularity and multicollinearity problems. The missing values of less than 5% were substituted with mean scores and the negative items were reverse-coded. Afterward, the normed chi-square, Root Mean Square Error of Approximation (RMSEA), Normed Fit Index (NFI), Non-Normed Fit Index (NNFI), Comparative Fit Index (CFI), and Standardized Root Mean Square Residual (SRMR) fit indices. According to Pituch & Stevens (2016, p. 654), NFI, NNFI, and CFI of 0.90, as well as SRMR of ≤ 0.10 and RMSEA of ≤ 0.08, are indicative of adequate fit. Besides, the normed chi-square is expected to be smaller than 3 (Kline, 2016). By the end, the internal consistency of the measurement tool was evaluated through Cronbach's alpha with a minimum acceptance level of 0.60 to 0.70 (Hair, Black, Babin, & Anderson, 2014, p. 90).

4.1.The first round of expert review
In the first round of expert review, the content experts (N = 5) provided substantial constructive written feedback on improving the comprehensibility of the items while rating the relevancy and clarity of each item. This helped the researchers in identifying and discarding several unrelated items. They also suggested some possible factors each representing several variables (i.e. items). As reported elsewhere, the collected data were put in CVI computing tables and the items having an I-CVI of '1' were kept while the others were discarded. As a result, 29 items were found to be less than the acceptable measure (I-CVI<1) set for the current study and therefore eliminated. So, 52 out of 81 items were retained.
Moreover, the S-CVI/AV and S-CVI/UA were found to be 0.916 and 0.642 respectively. Here, the S-CVI/AV showed an excellent congruity level of the content experts over the relevancy of the items to the content domain, whereas the S-CVI/UA is inadequate. This issue was considered to be solved in the second round of expert review after the revision and elimination of the unrelated items. Further, a modified kappa test was run for each item to control for the chance agreement problems. The items which had a perfect I-CVI showing perfect agreement level of the content experts over the relevancy of individual items, showed the highest agreement level in this test, too (k * =1). This indicates that the problem of relevancy chance agreement was not met in any of the items.
The same procedure was applied in calculating the consensus of the content experts over the clarity or the comprehensibility of the items using CVI tests. The results showed that the majority of the items (61) were rated as 3 or 4 showing high clarity of the items; each with an I-CVI of '1'. Moreover, the S-CVI/AV and S-CVI/UA were found to be 0.946 and 0.753 showing high expert congruency over the clarity of the items in averaging, but not in the universal agreement level.
Although some of the items, which were considered as highly relevant and comprehensible, some of the experts recommended some revisions. Besides, one of the items, i.e. working with other students, informs me of my strengths and weaknesses, was proposed to be divided into two separate items. Therefore, after performing the recommended revision process, a second version of the expert review form was devised by the inclusion of 52 items for the second round of expert consultation. In this form, the content experts were requested to assign the items to seven different representative sub-scales of cooperative learning they had provided in the first round of their review along with the ones suggested by the researchers. Further, they were asked to rate the relevance and the clarity of the items once again to enable the researchers to create the final form of the instrument.

The second round of expert review
The second version of the form was structured after major revisions adding seven of the subscales suggested by five experts (i.e. Self-Efficacy in EFL, Motivation in EFL, Personal Growth in EFL, Language Proficiency, Language Anxiety, EFL Teacher Role, Peer Influence on EFL Acquisition). Besides rating the relevancy and clarity of the items, the content experts assigned each item to the given sub-scales. The CVI and modified kappa calculation were run for a second time for both the relevancy and clarity of the items. The results show that all the items, but one, meet the excellent congruency of the reviewers considering the relevancy and clarity of each item (I-CVI=1, k * =1). As shown in Table 1, the S-CVI/AV and S-CVI/UA calculations also showed an excellent level of consensus (0.996 and 0.981 respectively). The only item, which did not meet the perfect agreement of the experts (I-CVI=0.8, k * =0.76) was discarded. Moreover, two items related to the EFL Teacher Role were discarded, as they were less than three to be placed under a separate sub-scale. Therefore, this sub-scale was excluded. As a result, 49 items were included in the final scale form.
However, while assigning the items to the given sub-scales, the expert panel assigned each item to more than one sub-scale. Some of them suggested merging some of the subscales to prevent this confusion. Therefore, the number of the sub-scales was decreased to five; some of them being named differently along with their definitions, to go for the third round of expert review to only the assign the items to the new sub-scales (Self-Efficacy in EFL, Language Proficiency, Self-Regulation in EFL, Motivation in EFL, and Affective Attitudes).

The third round of expert review
In this phase, five reviewers assigned the 49 items to five different subscales noted above. Items assigned to a subscale by at least three reviewers were retained accordingly. From these items, four of them were assigned to different subscales, where the reviewer consensus was not met and therefore were eliminated. Table 1 contains the final list of 45 items that met three to five matches of item-factor assignment with the resultant content validity statistics obtained from the second expert review.

Confirmatory factor analysis
The preliminary CFA results indicated that almost all of the fit statistics were of adequate cutoff, x 2 (N = 136) = 1604.34, df = 934, p < 0.001; x 2 /df = 1.71, NFI = 0.86, NNFI (TLI) = 0.93, CFI = 0.93, SRMR = 0.079, RMSEA = 0.073. Only NFI was under the threshold. Moreover, the t statistics belonging to four items were insignificant and two others had a factor loading of smaller than 0.30. They were later removed as two content experts suggested to. These problems are probably because of the small sample size in the study since CFA requires a larger sample, say, 200 or more. Aside from these three issues, the fit indices suggested an acceptable measurement model without modification. Furthermore, the statistics were examined by discarding four items (item 3, 38, 40, and 41) with insignificant t test values and two others (item 14 and 35) with low factor loadings of < 0.30. Four items, 1 and 2 as well as 13 and 19, were modified by sharing their error variances. The resultant fit indices displayed a considerable improvement within acceptable criteria and all items had significant t values (x 2 (N = 136) = 1252.75, df = 690, p < 0.001; x 2 /df = 1.81, NFI = 0.90, NNFI = 0.95, CFI = 0.95, SRMR = 0.072, RMSEA = 0.071). As seen in Table 2, the factor loadings of the observed variables ranged from 0.33 to 0.77 and the item-total correlations were significant across all variables.
The internal consistency reliability of the subscales of the CLAS tested through Cronbach's alpha varies between 0.68 and 0.87. However, the alpha for the entire scale was excellent (α = 0.95). To conclude, the coefficient alphas of 0.60 to 0.70 show minimum level of adequacy (Hair et al., 2014, p. 90).

Discussion and conclusion
Measurement tool development as a complex process requires substantial time and resources to collect data several times. Unfortunately, this may not be always feasible for the researchers. Thus, the emergence of alternative methods, such as the one used in the first phase of this study, could minimize the hustle and bustle of research tool development. Herewith, instead of employing an exploratory factor analysis, the CLAS was developed through a three-round rigorous expert review. All the analyses related to content validity were carried out according to the prominent researchers in this respect (e.g. Lynn, 1986;Rubio et al., 2003;Pilot and Beck, 2006;Pilot et al., 2007).
In the first round of expert review, the numbers of items were reduced to 52 out of 81 based on the CVI and modified kappa statistics computed for individual items. The remaining items were subjected to major revision based on the written comments provided by the expert panel and the low S-CVI/UA values for both the relevance and the clarity of the items. The second expert review form, containing 52 revised items, was devised along with the seven subscales suggested by the reviewers in the first round.
The second-round expert review provided positive results with regard to the relevance of the items to the content domain and their comprehensibility at an excellent level. In this process, three items were eliminated; one because of low CVI and two others were not enough to form a sub-scale. As a result, 49 items were retained. Further, the confusion arising from the similarity of the suggested factor names caused some of the experts to assign one item to more than one factor. Therefore, the factors were revised and reduced to five as recommended. So doing, the third round of expert review was conducted with 45 items to be assigned to five different factors (i.e. Self-Efficacy in EFL, Language Proficiency, Self-Regulation in EFL, Motivation in EFL, and Affective Attitude). This time the reviewers only assigned the items to the relevant subscales without rating their relevance or clarity. Of 49 items, 45 that were assigned to the subscales by at least three experts were retained, while four of them were eliminated since they did not meet reviewer consensus.
Furthermore, the construct validity of the remaining 45 items, representing five factors, was examined through CFA based on data from a small sample of 136 EFL pre-service teachers. The first run of CFA suggested the elimination of four items with insignificant t values and two with low standardized estimates. Of these eliminated items (i.e. 3, 14, 35, 38, 40, & 41), five of them exclusive of item 14, were reverse-coded before CFA since they were negative. Despite this, the fit statistics were within acceptable ranges except one. However, the second run of CFA, by excluding the stated items, suggested a good-fitting model with adequate factor load-