A Content Analysis Study on the Use of Analytic Hierarchy Process in Educational Studies

In this study, it is aimed to examine the studies based on the AHP (Analytic Hierarchy Process) method in the field of education and to present the researcher's perspective on how to use the AHP method in the field of education. Within the scope of this aim, firstly the AHP method was introduced with a sample application and then the results were interpreted. The other aim of the research; studies which based on the AHP methods in the field of education in the last five years have been examined through content analysis. AHP; is one of the “Multiple Criteria Decision Making (MCDC)” methods that can determine the priority or weights among the criteria and alternatives based on comparative judgments. The content analysis conducted within the scope of the research was carried out in the context of eight criteria determined by the researchers. According to the results of the analysis; the AHP method has shown an increasing tendency compared to the years, and usually is used for determining and prioritizing teaching priorities. Especially in Asia Pacific countries, the AHP method is used much more intensive. Another result is that the AHP method is used to make group decisions rather than individual decisions. It has been seen that the research has been done especially with undergraduate students. In addition to these, there are lots of studies with academicians and experts.


INTRODUCTION
Decision making is one of the indispensable component of human life. Because we need to make a decision at every stage of our lives and in any situation. However, the first condition for decision making involves multiple alternatives, the person tries to determine the most appropriate alternative for him / her based on more than one criterion. If this process is tried to explain this process via a simple example; when the individual wants to choose any university or department for higher education the individual pass through a decision-making process. There are many criteria that influence this decision-making process: location of the university, facilities, education quality etc. As you can see, many criteria are influencing the decision to choose the university. One of the most critical points in making a decision is determining the important criteria that influence decision making (Saaty, 1990). In this context, the decision-making process can be based on the individual's perceptions, predictions and also can explained via a mathematical model. Multiple criteria decision making is an analytical method used to rank, classify, or select alternatives according to the criteria specified when there are multiple criteria. Especially this method, which is widely used in business, politics, engineering, agriculture and economics (and nowadays decision-support systems), unfortunately does not seem to have much use in the educational field. Within the scope of this research, the studies which is in educational field via AHP have been examined.

The Purpose of the Study
In this study, it is aimed to examine the studies made using the AHS method in the field of education and to present the perspective of the researchers about how to use the AHS method in the field of Unlike multi-criteria decision making algorithms (TOPSIS, ELECTRE, UTA, PROMETHE, etc.); The AHP aims to combine qualitative and quantitative factors and arrive at a single judgment (Alsamaray, 2017). Advantages of the AHS method; a) use of hierarchical and ratio scales, b) comparisons of intuitive, qualitative, quantitative and rational factors, c) comparison of both criteria and alternatives according to criteria and d) solving decision problems which have objective and subjective criterias (Bhutta & Huq, 2002).
Another advantage of the AHP is that can be used both individually (to be applied to one person) and in group decisions. In the process of obtaining individual decisions, there are some algorithmic operations on comparison matrices, but in making group decisions there are some differences. Because there are more than one individuals when group decision is made, there are naturally more than one comparison matrix. These comparison matrices are reduced to a single matrix. Geometric averaging is often used when this reduction is done (Saaty, 2008).
Because of inherently AHP based on comparative judgment all alternatives and criteria are compared with each other in pairs. It's decided to according to the eigen values which is the result of decomposing the obtained matrices.
The AHP method also has a conceptual hierarchical structure. This structure was constructed by Saaty (1990); a) establishment of hierarchical structure of problem, b) determination of comparative judgments decisions and c) determination of priorities. Zahedi's structure (1986) is very similar to Saaty as; a) setting up a decision hierarchy, b) collecting data with comparative judgment, c) using eigen values to calculate relative weights and d) obtaining a range of ratings for alternatives. If the AHP method will be used, it must be followed this process. The steps of the AHP are much more detailed by Timor (2011) and Esen (2008); • Identification of the decision problem and determination of the goal, • Determination of appropriate decision criteria, • Determination of the alternatives, • Constructing of the hierarchical structure of the decision problem, • Comparison of criteria for each level of the hierarchy and determination of importance levels, • Comparative judgment of alternatives according to the criteria and calculation of priorities, • Calculation of corresponding index, • Sorting alternatives according to relative priorities, • And as a last step consistence analysis.

378
Within the scope of the research AHP method was used as; a) the identification of the purpose of the decision and the constructing of the hierarchical structure, b) making comparative judgments and c) determination of priorities.
Step 1 At this step, the problem is identified and the hierarchical structure is constructed. The most important and priority step in the AHP is constructing the hierarchical structure (Zahedi, 1986). Information about how to construct the hierarchy is given in Figure 1.  Figure 1, there are several levels in the AHP process. At the top of the hierarchy is the purpose of the decision problem. In the second level, the criteria set for this purpose and in the third level, alternatives which will be determined the priority order according to these criteria. The AHP structure can be manipulated as desired by the investigator or researchers. For example, a hierarchical structure can be created by adding sub-criteria under this criterion.
Step 2 At this step, the data is collected via the data collection tool which is created in accordance with the hierarchical structure. Comparative judgment of criteria and alternatives in the hierarchical structure are made. In order to make these comparative judgments, a scale with 17 bipolar and equally spaced units is used. This scale is a similarity to semantic differential scale. Descriptions of units of this scale are referred to as " Intensity of Importance" and these intensity of importance are given in Table 1 Table 1. Intensity of Importance Table (Saaty, 1990) Intencity of importance Definition Explanation 1 Equal Importance Both factors have the same importance 3 Moderate Importance According to experience and judgment is more one factor important than the other. 5 Strong Importance One factor is strongly more important than the other. 7 Very Strong Importance One factor is strongly preferred at a higher level than the other. 9 Extreme Importance One of the factors is very important at a very high rate.

379
One of the points to be noted in the comparison is that the alternatives are repeatedly compared according to each criterion, not just once. If we give an example through our model; Alternative1, Alternative2, Alternative3, Alternative4 and Alternative5 are repeatedly compared according to the first criterion, the second criterion and the third criterion.
Step 3 The third step is to determination of the priorities. For this firstly, comparison matrices are created. For example, if there are 3 different criteria (fuel consumption, performance, comfort) in deciding which of the 5 different car models (alternatives) will be taken, 3 * 3 comparison matrix for the criteria, and a 5 * 5 comparison matrix for the alternatives have been created. Then priority calculations are made based on these matrices. At the last step, consistency analysis is performed to obtain the validity of the results and then the results are reported. The comparison matrices for the criteria are given in Table 2.
The package programs can be used AHP method analyze, and also can be made manually step by step.

Sample application: AHP used for university selection
In this example, the AHP process and approaches of students in university selections are examined. Hence, the decision problem of the AHP method is the university selection and university selection is placed at the top of the hierarchical structure ( Figure 2). The AHP method can be applied to only one person, event or situation and also it can be applied multiple persons, event or situationWithin the scope of this study, AHP method has been employed in 3 steps as a) determination of decision problem and establishment of hierarchical structure, b) comparative judgment and c) determination of priorities.
Step 1 The hierarchical structure of the research as shown in Figure 2. As shown in Figure 2, the university selection was identified as the main problem and placed at the top of the hierarchical structure. Then the prestige of the university, the social facilities of the university and the location of the university were determined as criteria. Finally, according to these criteria; Universities A, B, C, D and E are determined as alternative. After the determination of the first step of decision-making and the establishment of the hierarchical structure have been completed, the second step, comparative judgment, has been passed.
Step 2 In the second step, the data obtained from the comparative judgment is placed in the comparison matrices. An example is given in Figure 3 to show how comparative judgment are made. Comparative judgments are read from left to right. For example, if the answer of the individual is to the left of the expression "1" which expresses equality, it is "9,8,7,6,5,4,3,2" and f it is on the right side of this expression it is "1 / 2,1 / 4/5, 1/6, 1/7, 1/8, 1/9 ". After the comparative judgment were made, a matrix was created which the criteria were compared with each other. This matrix is given in Table 3. The comparison matrix is a symmetric matrix. The diagonal values are "1". The comparison of the C1 criterion with C2 is "9" and the comparison of the C2 criterion with C1 is "0,111 (1/9)". After the criteria matrix, the alternative matrix is constructed which include comparative judgments of the alternatives based on each criterion. The matrix is shown in Table 4.
Step 3: In the third step, mathematical operations were performed on the comparison matrices and eigen values were determined. Then consistency analysis was performed. As a final step, priorities have been determined.
• First of all, normalized matrix is calculated.
• For this purpose, first the column values are sum and then the normalized matrix is calculated by dividing each element in the column by the column sum. • The vector of priorities is calculated by taking the average of each line in this matrix. At this stage, all the priorities matrices have to be obtained. • The priorities vector is multiplied by the comparison matrix given at the beginning and the matrix of all priorities is calculated.
The normalized matrix for the criteria, the priorities vector and all priorities matrix for are given in Table  5. max refers to maximum eigen value and n refers to number of criteria or alternatives. The random consistency index is given in Table 6. The consistency index, the random consistency index and the consistency ratio which is obtained according to these calculations are given in Table 7. The consistency ratio should be less than 0,1, otherwise an attempt should be made to increase consistency (Saaty, 1990). It can be said that the consistency rate which is obtained (0.0623) is below the desired value and the consistency is acceptable.

Journal of Measurement and Evaluation in Education and
In the analysis phase, the calculation has been made just for the criteria. These calculations are made in the same way for all alternatives based on each criterion. The priorities which is identified for the criteria are presented in Fig.   Figure 4. The Priorities for the Criteria As shown in Figure 4, this person's first priority for university selection was determined theprestige of the university (%70), the second is location of the university (%23), and the last priority is college social facilities (%7). These results are the second level results. The third level results will be evaluated according to three different criteria. The priorities of the student based on the prestige of the university for the five universities are given in Figure 5. As shown in Figure 5, this person's first priority for university selection based on prestige of the university criterion was determined A (%49), the second is B (%22), the third is D (%17), the fourth is C (%9) and the last priority is E university (%3). The priorities of the student based on the social facilities criterion for the five universities are given in Figure 6. As shown in Figure 6, this person's first priority for university selection based on social facilities was determined B (%40), the second is B (%33), the third is A (%16), the fourth is C (%7) and the last priority is E university (%4). Finally, the priorities of the student based on the location criterian for the five universities are given in Figure 7. As shown in Figure 7, this person's first priority for university selection based on location of the university criterian was determined A (%34), the second is B (%33), the third is D (%15), the fourth is C (%13) and the last priority is E university (%3). The priorities of a person or a group can be determined via AHP method. In addition to this, the AHP method can be used to determine the weights of the criteria or to sort them. In order to determine the situation of AHP studies in the field of education, the studies were examined with content analysis METHOD Content analysis was used as a method in this part of the research. Content analysis classifies texts by reducing them into interrelated and manageable data sets (Weber, 1990). Content analysis can be performed in four steps ; a) collecting data, b) data coding, c) finding themes and d) arrangement of codes and findings identification (Yıldırım ve Şimşek, 2004). Within the scope of the research, first of all literature rewiev was made.
• The properties which used for the literature review are as follows. "Analytic Hierarchy Process + Education" in web of science database, • "Analitik Hiyerarşi Süreci + Eğitim" in Google Scholar.Last five years' studies beetwen 2013-2017, • Studies in education fields, • Research that can be accessed from the databases provided by the university, • The publication language is Turkish and English.
As a result of the literature rewiev, 42 articles were included in the content analysis and examined. While content analysis is performed, coding is performed according to previously determined criteria. These criteria; • The purpose of the study • The purpose of the AHP method The results of the content analysis is presented in findings section.

FINDINGS
In this section findings which based on content analysis are presented.

Findings of the Purpose of the Studies
The aims of the studies used AHP method is system development, evaluation, selection and prioritization. Findings for the purpose of the studies are given in Table 8 in detail. As seen in Table 8, the AHP method firstly was used for determination of priority (15), secondly for evaluation (12), thirdly for selection select (9) and finally for the system developing (6).Environment and material evaluation includes product, evalution of distance learning and gamification. Evaluation of the instruction includes evaluation of universities, students, academics instruction performance, method, etc.

Findings of the Purpose of the AHP Method
In some studies, a different multi-criteria decision making method has been used in addition to the AHP method. Therefore, a title for the purposes of use of the AHP method has been included. Findings of the purpose of using the AHP method are presented in Table 9. As shown in Table 9, the AHP method was used in order to rank the criteria, sub-criteria or alternatives (42.86%) in the most studies. Secondly for evaluation (23,81%), thirdly determination of weight

386
(14,29%), and lastly for selection and system development (9,52%). Especially the studies which used AHP method for determination of weights, after this procedure another method is utilized and the study is carried out in this way.

Findings of Group / Individual Decision
AHP method is used for determination of group or individual decision. For this reason, there's a sub title about group or individual decision in this study. Using this method, it is possible to determine the priorities or trends of a person or a group or a university. Findings about this is given in Table 10. As shown in Table 10, most of the studies which is used the AHP method have been used to determine group priorities. This method was used in 14.29% of the studies to determine the individual and 85.71% to determine the group priorities.

Findings About Year of the Study
The studies which are publicated between 2013 and 2017 are examined. The distribution of the studeis by years is given in Figure 8. Kazakhistan and Pakistan. The African region includes; Saudi Arabia, United Arab Emirates, India and Malezia. Detailed information about the studies which carried out in these regions is given in Table 11. As shown in Table 11, it is seen that the most studies in Asia (%41), secondly Europe (%38), thirdly Africa (%14) and lastly America (%7) are the most investigated. %88 of studies in the Asian region (15) were conducted in Asia-Pacific countries (South Korea, China and Taiwan). %63 of the study in Europe region (10) were conducted in Turkey. Because, the literature rewiev was conducted via not only English but also Turkish keywords.

Findings About Level of the Studies
The first step of the AHP method is to determine the decision problem and to create a hierarchical structure. According to the problem situation, the number of levels of the structure can be completely determined by the researcher(s) and manipulated. So this criteria determined by the researchers in this study. Detailed information on the level of studies conducted is given in Table 12. As it is seen in Table 12, it is seen that the most of the studies have 3 levels (52.38%) and there are not many studies which have 5 levels. It has been found that studies consisting of four levels generally consist of criteria, sub-criteria and alternatives. Studies consisting of three levels include studies on criteria and sub-criteria. There are criteria in two level studies and findings about these criteria.

Findings About Sample Size
In the AHP method, it is possible to perform both group and individual calculations and decisions. The study group intervals were determined by the researchers. Detailed information about the sample size is presented in Table 13.

388
As shown in Table 13, the size of the study group varies. The group size were determined by the researchers as four interval. Another group is the ones that do not mention how many people they study with in their research (19,05%). Individual calculations are possible via AHP method, so there is also a single person studies (7,14%). Findings show that the preferred group size is usually 2-100 participants (54.46%) in the research. It is seen that the ratio of the studies' group size between 101-1000 is %16,67. The studies which have 1000 and over participants are very few (%2,38).

Findings About Study Group Level
It is possible to conduct research with all the stakeholders of a problem situation via AHP method. Detailed information about level of the group is presented in Figure 9. As can be seen in Figure 9, studies using the AHP method were frequently carried out with undergraduate students. The academics and experts are also seen as very preferred groups. One of the reasons for this is that the literature rewiev is done with the "education" keyword. There were not so much studies which conducted with administrator and staff in the education field. Some studies were conducted with different and multiple study groups. There is no information about study group level in some studies.

CONCLUSION and DISCUSSION
Within the context of the research, AHP that one of the most multi-criteria decision-making techniques, has been introduced, applied through an authentic sample and presented in the content analysis of research conducted in the field of education between the years 2013-2017. AHP method is one of multicriteria decision making methods and has a hierarchical structure. Information about this method is presented in the introduction section of this study. Then, this method was explained via an authentic sample. The university selection for the authentic sample is considered as a decision problem. For solving this decision problem firstly, a) determined the purpose of the decision and established a hierarchical structure, b) made comparative judgment and c) determined priorities.
When the results of the content analysis are examined, it is found that the studies which using the AHP method for system development, evaluation, selection and determination of priorities. System development studies includes decision support systems and quality evaluation systems. Especially in the development of decision support systems, the AHP method offers researchers opportunities. Because there are multiple criteria and alternatives in decision making and this situation make the process complicated. Decision support systems can be developed, which can help individuals make decisions, via AHP method. The evaluation studies consist of product and material evaluation and evaluation of instruction. The selection studies consist of student selection, course selection and university selection. It has also been observed that these studies are usually done on paper, but not via an electronic system. It's suggested to the researchers firstly development an electronic system based on AHP. It seems that there are lots of studies which used AHP method about sociology, politics, auatomation profit and loss analysis, budget planning etc (Zahedi, 1986). There are also studies used in education field (Wang, 2014;Weng, Zhang & Liu, 2014;Thanassoulis, Dey, Petridis, Goniadis & Georgiou, 2017) but it is seen that these studies are limited. Educational studies refer to the teaching and learning process. It is possible to say that the studies which carried out in faculty of education are much more limited. is suggested studies which is used AHP should be conducted in order to determine the learners' need.
The purpose of using the AHP method in studies may differ from the purpose of the study. For example, while the purpose of the research may be to choose the best method, but AHP can be used with the purpose of determining the weights criteria. In the literatuere, the AHP method is used for the purpose of respectively ranking, evaluation, weight determining, selection and system development. There are lots of studies ranking the criteria or alternatives. In studies which using weight determination purposes, the AHP is the first leg of the study; weights of criteria or alternatives are determined via AHP and in the second step a ranking was obtained by the TOPSIS method (Kecek & Söylemez, 2016;Lokare & Jadhav, 2016).
Both individual and group decisions can be made by the the AHP method studies. Comparison matrix uses for individual decision, but for the group decision the matrices have to reduce just one comparison matrix. Geometric mean is used for reducing the matrices. It is generally seen that the AHP method is used for group decision and even if it is a small number AHP is used for individual decision.
It is seen that the distribution of the studies using the AHP method is increasing year by year. The studies which have done in 2017 is limited to the beginning of December 2017. Because the literature rewiev was done by the beginning of December. The most studies were conducted Asia region. And the frequency of the studies that conducted especially in the Asia-Pacific countries is remarkable. Europen region is the second order. The reason of this is the keywords. For the literature rewiev both English and Turkish keywords were used. In order to reveal the situation in our country Turkish keywords were used.
The AHP is an approach that adds a second order level to the scaling techniques and adds criteria to the model and resolves the resulting hierarchical model. Hence the AHP includes more than one level. In the literature it is seen that the studies consisting of three levels are the majority. These studies include usually the criteria and the sub-criteria. In addition, there are three levels of study in which the criteria and alternatives are included. The level of the hierarchical structure can be determined and manipulated by the researcher(s). Thus providing a very flexible structure to the researcher(s). Levels can be determined appropriately for the purpose of the study. In the literature, it is also seen that qualitative studies have been conducted for determining criteria and alternatives (Ertuğ & Girginer, 2014;Chiu, Kao, Pu, Lo & Huang, 2015). The criteria, sub-criteria or alternatives are situated in a hierarchical structure based on findings of the qualitative studies.
The findings about the sample size, it is concluded that the maximum frequency is within the range of 2-100. Besides this, there are also studies carried out with one person. There is also a study which the size of the study group is 1000 and more. Studies which used the AHP method were usually carried out with small groups. The groups priorities or preferences can be determined with the small groups studies. However, it is thought that it is necessary to work with wider working groups in order to reach a general judgment. Most of AHP studies carried out with undergraduate students. Other than this the study groups were comprised of academics and experts. Students and academics are the most studied group because keywords include "education". Especially for the system development studies, AHP can be used for the need analysis which is the first step of the studies. All stakeholders' priority or preferences, differences and similarities can be determined via the AHP method. According to these findings, designs can be