A Statistical Comparison of Norm-Referenced Assessment Systems Used in Higher Education in Turkey*

The purpose of this study is to identify different norm-referenced assessment systems used in Turkish higher education, and to compare them empirically. Norm-referenced assessment regulations of 70 universities in Turkey was primarily analyzed, and universities were divided into four different groups depending on their norm-referenced assessment systems (only applying T-score conversion, the most commonly used method; applying T-score conversion and quantiles together; applying T-score conversion, quantiles and standard deviation together; applying standard deviation based norm-referenced assessment system). After the algorithms of two universities applying T-score conversion and three universities applying other norm-referenced assessment system were selected, they were used to convert end-of-year grade for each course of 19,574 students in a state university into letter grades and 4-point system. To test the differences of the norm-referenced assessment systems used in these universities, the norm-referenced system of a university were compared with the criterion-referenced system of the same university as well as norm-referenced systems of other universities. The paired t-test was used to identify the difference between norm-referenced and criterion-referenced assessment, while the differences between norm-referenced assessment systems were analyzed through one-way analysis of variance. The findings revealed that the letter grades calculated through the norm-referenced assessment were statistically different than the ones calculated with criterion-referenced; besides, a statistically significant difference was identified between the letter grades obtained using the norm-referenced assessment systems of universities. At the end of the study, the findings were discussed in term of students and instructors.


INTRODUCTION
The aim of education and training is the disclosure of the cognitive, affective and psychomotor skills to the students in a planned and programmed way.To ensure this, the curriculum consisting of four basic components should be primarily determined.These components include (a) determining the behavioral objectives, (b) constructing the content in accordance with these objectives and the readiness of the students, (c) creating learning and teaching activities with the idea that each student learns differently; and (d) performing meaningful assessment and evaluation (Tan, 2015).In particular, the significance of measurement and assessment cannot be underestimated in terms of determining the extent to which the behavioral objectives within the program reflect the readiness of the students and identifying as to what extent learning and teaching activities are appropriate to the objectives and behaviors.
The concepts of measurement and assessment are different and even complementary concepts.Specifically, measurement refers to a variable or an object with numbers or symbols, while assessment provides a meaningful interpretation of the results obtained from the measurement by comparing them through a frame of reference.Previous studies have revealed that a frame of reference will vary across teacher notions, student success distribution in the class, student ability and their achievement scores related to the program (learning difference at the beginning and end of the program) as well as the Although there does not exist a definite line in terms of the use of criterion-referenced and normreferenced assessment, the exams that the criterion-referenced method is applied may be prepared in accordance with the exam preparation guidelines (Thorndike, 2005).Haladyna and Rodriguez (2013) have indicated that susceptibility should be displayed for the construction of exam questions based on the objectives, item-writing guidelines and the cognitive taxonomy levels.Otherwise, the scores obtained as a result of the examination will tend to be less homogeneous; the variation of the measured feature will not be explained at the maximum level, and in this case, ranking or placement of the students according to the scores will not be reasonable in terms of measurement and assessment (Hambleton et al., 1978).
In particular, the question related to what kind of assessment type is applied is the subject of hot debate in higher education today.The current relevant studies have put forward that the use of normreferenced assessment in universities with high competition and success motivation will be much more effective, and that criterion-referenced assessment will be more appropriate in schools with low achievement motivation since norm-referenced assessment will cause grading inflation (Başol, 2013;Selvi, 1998).In one of the most comprehensive studies on this subject, Johnson (2003) has argued that grading inflation is a serious problem in universities and that the method used for grading can vary across universities and faculties.Unfortunately, a limited number of studies have been conducted to compare criterion-referenced and norm-referenced assessment.
Having analyzed the guidelines of two different state universities using criterion-referenced and normreferenced assessment, Nartgün (2007) has compared the grades with different distributions and suggested that the grades calculated with the norm-referenced assessment provide more than those of the students deserve.In other words, when compared with the criterion-referenced assessment, it is evident that norm-referenced assessment leads to grading inflation.However, criterion-referenced assessment method offers more effective results than the norm-referenced assessment in cases when the raw achievement grades are close to one another, that is, the scores are similar (standard deviation is low).Atılgan, Yurdakul, and Öğretmen (2012) have found different results compared to the findings of Nartgün (2007).3,120 grades obtained from the students studying at the faculty of education and calculated through use of norm-referenced assessment have been converted into criterion-referenced assessment, and it has been determined that 42% of the grades are free from any change.Twenty-two percent of these grades increase in favor of the norm-referenced assessment, while the rest increases concerning criterion-referenced assessment.In addition, the results of the interviews conducted with the faculty members in the same study have shown that the grading system calculated through the 14 norm-referenced assessment has a negative impact on the interaction among students.This situation results in negative competition among the students and grouping; moreover, it also decreases the sense of trust towards each other and damages the value education of the students.
Besides, Duman (2011) has emphasized the positive side of the norm-referenced assessment method.In a study conducted with primary school prospective teachers, the norm-referenced assessment has been identified to partially compensate the grading deficiencies emerging due to the faculty members and the exam questions.Similar results have emerged in previous studies.In particular, both classroom assessment questions prepared by teachers and questions in question banks are of low quality (Demir & Atalmış, 2017;Downing, 2005;Masters et al., 2001;Mehrens & Lehmann, 1991;Tarrant et al., 2006).As a result of the analysis on measurement and assessment, both the reliability and validity of the test scores with low-quality questions will decrease and a false assessment mechanism (decision) will be formed depending on the test scores that are calculated incorrectly (Çelen & Aybek, 2013).
Besides, Nartgün (2007) has argued that the norm-referenced assessment would provide an advantage to students after graduation.The use of norm-referenced assessment system in higher education in the countries, especially in Turkey, where competition is common in terms of both university entrance exam and after graduation provides an opportunity to compare the grades of the students graduating from different universities on the same scale.To illustrate, the student who graduated with a score of 60 from a university that requires high score is considered to have near or equal achievement score with the student who has graduated with the score of 80 from a university requiring low achievement score.
In a more explicit way, just as the person who graduated with a lower GPA from University A that admits those with high scores cannot be perceived in the same way as the person who graduated with a higher GPA from University B that accepts those with low scores, the person graduating from University B can be perceived as more successful at first sight.However, the person graduating from University A may have received far more comprehensive, well-equipped and innovative education.Therefore, s/he may have experienced a difficult process and graduated with a low GPA.For this reason, a norm-referenced assessment system may enable to compare two students graduating from these two different universities through using the same scale.

The Assessment Systems Used at Universities in Turkey
Upon reviewing the state universities in Turkey, the norm-referenced assessment system is used as a grading system in the majority of the universities.Considering the universities that use criterionreferenced assessment, they have been identified to hold different pass/fail cutoff score and their letter grades are determined variously.
The letter grade of CC equals to 60 in most of these universities (e.g.Selçuk University, Şırnak University, Uludağ University), one of which is 54 (Amasya University), one is 64 (Abdullah Gül University), while 50 in others (e.g.Recep Tayyip Erdoğan University, Gümüşhane University, and Bayburt University); moreover, the grade of CC corresponds to 65 in some of the universities (e.g.Hacettepe University and Bolu Abant İzzet Baysal University) and 70 in only a small number of them (METU, Boğaziçi University, and Gaziantep University).In particular, the universities that keep CC score range high were observed to have high university entrance grades (e.g.Hacettepe University, Gebze Technical University, Boğaziçi University, and METU) or they have been separated from such universities in the following years.For instance, Gaziantep University may be considered a university separated from METU but still able to protect METU traditions.
Given the instructions of the universities using norm-reference assessment system in Turkey, they use different methods and algorithms depending on the number of students in the class, classroom grade point average, percentiles and standard deviations of grades.While some universities determine the grade of CC, they allow the instructors to intervene in addition to the classroom grade point average (Ankara University and Istanbul Technical University).Besides, criterion-referenced assessment or norm-referenced assessment systems is implemented in many universities according to the grade In some universities that do not use the T score method, a criterion-referenced or norm-referenced assessment system is applied depending on the standard deviation of the grade distribution in the class as well as the number of students and classroom grade point average (İstanbul University, Çukurova University, Harran University, İnönü University, Kahramanmaraş Sütçü İmam University).Similarly, the upper and lower limits of the letter range in these universities are determined by the university administration.These different norm-referenced systems are presented in detail as follows:

Norm-Referenced Assessment System Only Applied According to T Score Conversion
This assessment system initially calculates students' raw success grades (RSG) through using midterm and final exam grades.RSG is calculated by taking 40% of midterm and 60% of the final exam.While determining students who will be included in the norm-referenced assessment system, RSG lower limit and the lower limit of number of students are used.Table 1 displays RSG lower limit and the number of students in universities applying norm-referenced assessment.As can be seen in Table 1, the lower limit of RSG and the number of students are determined as 15 and 11 in 65 state universities using norm-referenced assessment system in Turkey.Specifically, 11 students with RSG scores greater than 15 are required to use a norm-referenced assessment system.Otherwise, criterion-referenced assessment system is supposed to be used rather than norm-referenced assessment.
Following this stage, students' scores will be converted into T scores by using the following formulas in the norm-referenced assessment system, which requires only "T-score conversion" (Güler, 2017).
Here, N refers to the number of students participating in the assessment,   signifies students' RSG, µ, represents the students' RSG average, σ is the standard deviation of the students' RSG and T is the score converted from students' RSGs.After each student's T score is obtained, the letter grades are given to the students by using the value in Table 2 depending on the RSG average of the class.Table 2 suggests that a class in which the norm-referenced assessment system is applied is in one of eight different levels according to RSG average of the class.To illustrate, the class whose RSG average is between 80 and 100 is considered "outstanding", while the class whose RSG average varies between 57.5 and 62.5 is regarded as "Good".Taking the students' letter grades into account, the letter grade of a student whose T score is 58 and who is in the class with 55 RSG average (Good) is BB; whereas the student with the same T score but in the class with 65 RSG average (Very Good) has BA letter grade.

17
When the norm-referenced assessment regulations of the universities are examined, most of the universities use the chart as in Table 2 while calculating T score (Celal Bayar University, Kafkas University, Marmara University), while others use criterion-referenced assessment system for upperlevel classes.In no uncertain terms, some of the universities with 60 and over (Akdeniz University), 70 and over (Bartın University, İzmir Katip Çelebi University, Muğla Sıtkı Koçman University), 80 and over RSG average (Karadeniz Technical University, Ondokuz Mayıs University, Uşak University) use criterion-referenced assessment system.Table 3 depicts the letter ranges in the criterion-referenced assessment applied in some universities.Besides, some of the universities use norm-referenced assessment system by decreasing the number of class levels (below 8) and enlarging the class level intervals in Table 2 (Bülent Ecevit University, Çankırı Karatekin University, Niğde University, Süleyman Demirel University).
Table 3. Letter Intervals of the Universities Regarding Criterion-Referenced Assessment System

Norm-Referenced Assessment System Applying T Score Conversion and Quantiles
This assessment system is based on the number of students participating in the norm-referenced assessment.Upon examining the norm-referenced assessment regulations of the universities applying this assessment system, only T score conversion is conducted in cases when the number of students participating in the norm-referenced assessment is 30 or over just as in Table 2, while the letter grades are based on quantiles as in Table 4 when the number of the students is between 10 (11 in some of the universities) and 29 (30 in some of the universities).First, the percentage of the students participating in the norm-referenced assessment is calculated while determining their letter grades and then their letter grades are identified through using Table 4.For instance, in a class where the RSG average is 60, the letter grade of a student in the top 10% is AA, while that of a student in the top 30% is identified as BB.

Norm-Referenced Assessment System Implementing T-Score Conversion, Quantiles, and Standard Deviation
Within this system, the RSGs' standard deviation of some students participating in the normreferenced assessment in some universities is calculated in addition to the T-score conversion and the percentile method.If the standard deviation is below a certain value, a criterion-referenced assessment system is used.This value ranges between 4 (such as Yalova University) and 8 (Selçuk University) based on the regulations of the universities.

Standard Deviation Based Norm-Referenced Assessment System
This assessment system holds criterion or norm-referenced assessment systems by focusing on the standard deviation of the grade distribution in the classroom as well as the average of the student group (class) participating in the norm-referenced assessment.As indicated in the study conducted by Nartgün (2007), the effectiveness of the criterion or norm-referenced assessment systems depends on the standard deviation.In this system, moreover, criterion or norm-referenced assessment is applied depending upon the lower limit of the number of students participating in the norm-referenced assessment.This varies between 15 and 20 according to the regulations of the universities.Nevertheless, in this assessment system, the criterion-referenced assessment system is a prerequisite when the standard deviation of the grades in the class is below 8 (grade distributions are close to each other).In applying this system, RSG is calculated as the sum of 40% of the student's mid-term scores and 60% of their final scores.
Here µ represents classroom average and σ signifies the standard deviation of the distribution in the class.Formulas of µ and σ are presented as follows (Field, 2009).

19
In this formula,   represents the RSG of a student participating in the norm-referenced assessment, while N shows the total number of the students participating in this assessment.
Table 3 indicates the criterion-referenced assessment letter intervals in İstanbul University when the norm-referenced assessment system is not applied.Criterion-referenced assessment is applied in cases where the number of students in the class is less than 10, and/or the standard deviation of the grade distribution in the class is below 8.

The Significance and Aim of Research
Unlike previous studies focusing a particular norm-referenced system in Turkey (Atılgan et al., 2012;Duman, 2011;Nartgün, 2007), this research has compared aforementioned four different normreferenced assessment systems that are widely used in Turkey.In this regard, Aksaray University, which uses norm-referenced assessment for each grade level among the universities that apply only Tscore conversion, and the norm-referenced assessment system of Akdeniz University (for the classes with over 60 RSG mean) that applies criterion-referenced assessment systems for the upper-level classes have been chosen.Among the universities that use T-score conversion and quantile method, the sample of the norm-referenced assessment system of Karadeniz Technical University has been chosen; Selçuk University's norm-referenced assessment system has been preferred as an example for the university using T score conversion, quantile, and standard deviation.Finally, İstanbul University norm-referenced assessment system has been chosen as it does not use T-score conversion and only uses a standard deviation-based conversion.
In order to achieve this goal, answers to the following questions have been sought: 1. Is there a statistically significant difference between students' letter grades calculated through norm-referenced assessment and criterion-referenced assessment?
2. Is there a statistically significant difference between students' letter grades calculated by different norm-referenced assessment systems?

METHOD
This study can be considered a causal-comparative research approach, seeking to determine differences between groups by examining differences in the experiences of group members (Lodico, Spaulding, & Voegtle, 2010).In this study, the groups of individuals are students whose letter grades calculated through different norm-referenced assessment and criterion-referenced assessment.This section holds information regarding the research sample, process, and data analysis.

Research Sample
The research data have been collected through the midterm and final grades of the students studying at different faculties and vocational colleges in a state university during the fall semester of 2014-2015, and the total of 19,574 students' RSGs have been considered during data analysis.

Data Analysis
In order to identify the difference between norm-referenced and criterion-referenced assessment, which is the first research question, the paired t-test was used to analyze the difference between the two groups.For the second research question, the differences between the letter grades obtained by the norm-referenced assessment system of each of the universities mentioned above have been analyzed through one-way analysis of variance (ANOVA).

RESULTS
Figure 1 depicts the distribution of students' RSGs, are not converted scores through normreferenced/criterion-referenced assessment.
Figure 1.Students' End-of-Term Raw Success Grade (RSG) Distribution Upon examining Figure 1, the skewness coefficient of the students' RSGs distribution was identified to be -0.109 and this value was considered normal because it is between -1 and +1 (Büyüköztürk, 2009).
In the next stage, norm-referenced and criterion-referenced assessment systems used by Aksaray University, Akdeniz University, Karadeniz Technical University, Selçuk University, and İstanbul University have been applied for these RSGs.As shown in Table 6, the differences between the criterion-referenced and norm-referenced assessment scores for each university have been determined to be statistically significant and the scores increase in direction of norm-referenced assessment.ANOVA has been applied to explore the difference between the scores obtained from different norm-referenced assessment systems, which is related to other research question.Accordingly, a statistically significant difference has been determined (F (4, 789910) =4632.88,p<.05).As a result of the post-hoc test (LSD), the scores calculated with the normreferenced assessment score of each university are statistically different from those of the other university.The results are presented in Table 7.According to Table 7, the university which applies the most advantageous norm-referenced assessment system has been determined to be Karadeniz Technical University, which is followed by Aksaray University, Istanbul University, Akdeniz University, and Selçuk University, respectively.

RESULTS AND DISCUSSION
This research aims to explore whether students' letter grades differ across norm-referenced and criterion-referenced assessment methods and how this difference varies across universities.In this regard, the end-of-term raw achievement scores of 19,574 students who study at a state university during the fall term of 2014-2015 academic year have been converted into letter grades and 4-point

23
grading systems through use of both norm-referenced and criterion-referenced assessment regulations of the universities.After applying the paired samples t-test, the letter grades calculated via the normreferenced assessment have been identified to be statistically significant and high compared to those calculated through the criterion-referenced assessment.In the following stage, ANOVA has been used in order to determine the difference between the letter grades obtained by using the norm-referenced assessment systems of the universities and the result has been found to be statistically significant.Specifically, students with the same RSG appear to have different letter grades in different universities.
The research findings have notably shown that the students' letters grades decrease as standard deviation in the norm-referenced assessment systems and low cut-off scores in criterion-referenced assessment systems are used.To exemplify, considering the lower grades obtained through Selçuk University norm-referenced assessment system, criterion-referenced assessment system are used for classes with RSG mean of over 70 and/or standard deviation below 8. Likewise, criterion-referenced assessment is used for the classes whose RSG mean is over 60 in Akdeniz University norm-referenced assessment system, while the same system is used for classes whose RSG mean is over 90 and/or standard deviation is below 8 in İstanbul University.In the norm-referenced assessment system of Aksaray University, criterion-referenced assessment is not applied depending on the RSG mean, whereas criterion-referenced assessment is used for the classes with 80 and over RSG meaning the norm-referenced assessment system applied by Karadeniz Technical University.However, 80-89 scores in the system applied by this university refer to BA and the scores above 90 signify AA letter grade, which increases students' letter grades.
When the research results are considered in general terms, the norm-referenced assessment has been determined to be much more in favor of students' letter grades compared to the criterion-referenced assessment.A similar result has emerged in the study conducted by Sayın (2016); however, different results have been found by Atılgan et al. (2012).This may result from the different use of the normreferenced assessment algorithms and the small size sample group.
Based on the results of this research, two different evaluations can be made in terms of students and academic staff.On the basis of the student's perspective, students have higher grade point averages with the norm-referenced assessment than criterion-referenced assessment.This paves the way for the fact that norm-referenced assessment will lead to grading inflation as indicated by Başol (2015) in the related studies.Besides, different norm-referenced assessment systems reveal that the same RSG has been converted into different letter grades, meaning that the universities applying norm-referenced assessment system are more advantageous compared to the others.Both the difference between the norm-referenced and criterion-referenced grading system and the difference between the normreferenced assessment systems may cause injustice.The concrete indicator of this situation occurs when the students apply to graduate programs.The degree effect of the GPA is used up to 40% in some universities is the evidence of the injustice of assessment systems used in the universities.
Upon analyzing the research results in terms of academic staff, the norm-referenced assessment system will be able to reduce the grading errors that will arise from the structure of the tests prepared by the academic staff.Considering that the academic staff may be lacking preparing a sufficient test or question technique, errors emerging due to the structure of the test, such as misinterpretation of the question or not being included in the current program, will cause students to get poor grades.However, since the findings of the previous studies (Nartgün, 2007;Sayın, 2016) and this study provide the conclusion that the norm-referenced assessment increases the grades of students, the normreferenced assessment is likely to convert these lower grades, especially those of academic staff, into higher letter grades.The point to be noted here is that some students pass through the class without deserving it or being in high letter ranges.Thus, it is preferable to have an adequate level of the students included in the norm-referenced assessment and to determine the letter intervals rigorously.
What is more, it is of high significance to decide whether to use norm-referenced or criterionreferenced assessment depending on the purpose, structure, and results of the exams in terms of efficient measurement and assessment.Turgut and Baykul (2015) have noted that the scores will be distributed symmetrically when there is a normal distribution; otherwise, the grades of the students will be largely affected by the other people in the class.Because the extreme values in both the right skewed and left skewed grades will change the mean of the distribution as well as increasing the standard deviation (Turgut & Baykul, 2015).
Besides, the use of criterion-referenced assessment will lead to different results in the case of the skewness of the raw score distributions.As the grade point averages will be high in the left-skewed distributions, many people in the class will pass with high letter grades, while in a class with a rightskewed distribution, the grades will be lower, thus the letter grades will be low and many students may fail.Given the skewness of the grades derives from the fact that the exam questions are too difficult or too easy, it is probable that the academic staff does not prepare qualified questions in terms of measurement and assessment.Thorndike (2005) draws attention to the fact that while preparing a qualified test, 25% of the questions should be difficult, 50% of them are at a medium level and 25% easy.Thus, the distribution of the grades will be closer to the normality.Yücel (2015) has stated that the national exams in Turkey such as university entrance for which normreferenced assessment is used measure the objectives at the level of remembering, understanding and applying levels.The use of blueprint in the exams prepared by the academic staff may affect the distribution of grades.In particular, writing the questions that will measure the cognitive gains of the students in all levels will determine how much the student has learned.The most commonly used type of question, open-ended questions and project-based assignments, which are the most commonly used type of questions, will provide the students with the objectives that need to be gained in the upper levels, namely, analysis, synthesis, and evaluation.It is expected that the number of questions will be higher in the exams prepared in this direction, as a result of which both the scope validity of the exam will increase and the grades obtained as a result of the application of the questions will be expected to be distributed normally.In other words, the exam consisting of questions related to all cognitive levels (knowledge, comprehension, application, analysis, synthesis, and evaluation) will undoubtfully affect the difficulty of the questions, and as Thorndike ( 2005) stated, it will cause the questions to be distributed normally in terms of the degree of difficulty.In short, the preparation of an exam within the framework of measurement and evaluation will affect the grade distribution, and consequently, exams can be evaluated through criterion-referenced assessment without the need for norm-referenced assessment.

Table 2 .
Calculation of Letter Grades in terms of T Score

Statistical Comparison of Norm-Referenced Assessment Systems Used in Higher Education in Turkey
Atalmış, E., H. / A ___________________________________________________________________________________ ___________________________________________________________________________________________________________________ ISSN: 1309 -6575 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Journal of Measurement and Evaluation in Education and Psychology

Table 4 .
The Calculation of Letter Grades Depending on Quantiles

Table 5
presents the letter grade determination table of the İstanbul University which first applied this norm-referenced assessment system.

Table 5 .
İstanbul University Letter Grade Calculation through Norm-Referenced Assessment System

Statistical Comparison of Norm-Referenced Assessment Systems Used in Higher Education in Turkey
Atalmış, E., H. / A ___________________________________________________________________________________ ___________________________________________________________________________________________________________________ ISSN: 1309 -6575 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Journal of Measurement and Evaluation in Education and Psychology Students' RSGs have been calculated by taking into account 40% of the midterm score and 60% of the final score.Afterward, the RSGs of the students have been calculated as letter grades by adapting them depending on the above-mentioned assessment systems of Aksaray University, Akdeniz University, Karadeniz Technical University, Selçuk University, and İstanbul University.Besides, the RSGs of the students have been converted into letter grades considering the absolute assessment table of these universities as displayed in Table2.Thus, each student's letter grade calculated by means of both norm-referenced and criterion-referenced assessment has been determined and then converted to the to 4-point grading system.Given the students took more than one course in the 2014-2015 fall semester, the same process has been performed for each course's RSG taken by each student.In other words, the analysis in this research has been conducted for 19,574 students' 157,983 letter grade.
___________________________________________________________________________________ ___________________________________________________________________________________________________________________ ISSN: 1309 -6575 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Journal of Measurement and Evaluation in Education and Psychology 20 grade equivalent

Statistical Comparison of Norm-Referenced Assessment Systems Used in Higher Education in Turkey ___________________________________________________________________________________
Atalmış, E., H. / A ___________________________________________________________________________________________________________________ ISSN: 1309 -6575 Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi Journal of Measurement and Evaluation in Education and Psychology 21 Aksaray Uni.Figure 2. Criterion-Referenced Norm-Referenced Assessment Grade Distribution of the Universities

Journal of Measurement and Evaluation in Education and Psychology ___________________________________________________________________________________
suggests that norm-referenced assessment generally increases letter grades of the students contrary to criterion-referenced assessment.As a result of the paired t-test, the norm-referenced assessment mean score of the students (̅ =2.21, SD=1.51) has been noted to statistically differ from that of the criterion-referenced assessment (̅ =1.59, SD=1.37) (t(789914)=-590.92,p<.05).Table6displays t-test results conducted for each university.

Table 7 .
Comparison of Norm-Referenced Difference Scores among Universities