There are several
methods for the purpose of measuring and evaluating in education and one of
these methods is achievement tests which varies as true-false, matching or
multiple-choice. To prepare test items for different cognitive domain levels,
to make objective scoring and thus to compare different groups, and to save and
time in measuring and evaluating crowded classes' level of achievement; all of
these makes multiple choice achievement test more advantageous than other
measurement methods. A multiple-choice achievement tests whose validity and
reliability is ensured has a reusable feature. Creating an achievement test
that has reached this stage is an effort that requires care and attention.
Achievement tests prepared in accordance with measurement standards are one of
the measurement instruments that provide data on students' level of
understanding about the content, the level of accession to course's objectives,
the quality of actions in instruction and what kind of changes should be made.
The level of usefulness of the methods and techniques used in teaching the
content can also be evaluated by the data obtained from the achievement tests.
Clear, comprehensible and well-established tests provide a sound basis for
making objective and important decisions about both students and education
programs. One of the most important factors to be considered when preparing
test items is to develop ways to minimize the external cognitive load that may
occur in the student. Practices such as not using non-specific questions,
distractive content and images; paying attention to make all parts related to a
question to be on the same page; using visuals if they will facilitate the
understanding of the item; writing in bold type or underline the important
words in the item root; using clear and short guidelines have been considered
in constructing this achievement test.
One of the shortages
in the related literature is there is not any research about developing an
achievement test on solar system and beyond unit that belongs to 7th grade of
science curriculum which was revised in 2018. Because of this reason, the aim
of this study is to develop a valid and reliable multiple-choice achievement
test for the first subject of 7th grades namely Solar System and Beyond in 2018
Science Curriculum which was enacted in 2018-2019 academic year. Having a
contextual structure and requiring students to use problem solving skills are
the points considered during the item construction process. Because the test
items should be appropriate to the objectives of the unit, the related
curriculum and textbooks were examined. As it found that more than one
objective in the curriculum was expresses in a single objective, those ones
were written separately. Also, additional objectives were written by the
researchers by using the content in science textbooks without going out of the
curriculum. A total of 40 objectives were obtained and an achievement test
involves 43 items was developed which was analyzed by domain experts. A table
of specifications was prepared in order to see perspicuously which items
corresponds to which objectives. The participants in this study are 392 7th
grade students attending to public schools in Turkey. The survey method was
used in this study because the data was collected to identify some aspects or
characteristics of a sample representing the related population, rather than
each member of the population. The 43-item achievement test was applied to 392
students who were voluntarily contributed to the study, and in the analysis of
data TAP (Test Analysis Program) was used. Item difficulty index and item
discrimination index for each item and Kuder-Richardson 20 value for
reliability were examined.
In the analysis of the
data, some formulas and criteria were used which was given below;
D:
Item discrimination index
p:
Item difficulty index
Dh:
Number of students who answered the item correctly in the highest group
Dl:
Number of students who answered the item correctly in the lowest group
Nh:
Number of students in the in the highest group
Nl:
Number of students in the in the lowest group
Table 1. Item Difficulty Index and Interpretation of
Values
|
Item Difficulty Index
|
p
|
Means
|
|
p<0,35
|
Difficult
|
|
0,35<p<0,85
|
Moderate
|
|
0,85<p
|
Easy
|
Table 2. Item Discrimination Index and Interpretation
of Values
|
Item Discrimination Index
|
D
|
Quality
|
Suggestion
|
|
D < -0,01
|
Worst
|
Discard
|
|
0,00 - 0,20
|
Poor
|
Discard or review
in depth
|
|
0,20 -0,29
|
Mediocre
|
Check/Review
|
|
0,30 -0,39
|
Good
|
Possibilities for
improvement
|
|
0,39 < D
|
Excellent
|
Retain
|
As a result of the
analysis, 7 items in the multiple-choice achievement test were discarded
because of item discrimination indexes were not as desired levels. Those
questions are labeled as 11, 12, 13, 14, 26, 34 and 35. Their discrimination
indexes were 0,10; 0,15; 0,15; -0,04; -0,02; 0,15 and 0,16; respectively. The
initial test’s KR-20 value was 0,78. after discarding the mentioned above
items, a valid and reliable multiple-choice achievement test consisting 36
items with 0,44 difficulty index, 0,40 discrimination index and 0,80 KR-20
value was developed ultimately and made available for researchers and science
teachers who are interested in astronomy.
In addition, which
cognitive domain levels corresponds to those seven items that have high
difficulty index is also examined. Objectives at mostly
"Understanding" and one for each "Analyzing", "Evaluating"
and "Creating" levels were seemed to be difficult for participants.
Although the number of items in the test that require high level thinking
skills is limited, it would not be a mistake to look for reasons in-classroom
activities about why high-level thinking seemed difficult to students. In an
environment where the teacher and textbook are the primary source and the
course is being teach in a manner that will not allow students to make
analysis, evaluation and creating new ideas; may lead to an inability to
develop high level thinking skills. The objectives that correspond to the items
which have medium-quality discrimination index were also examined and two of
them are at "Evaluating" level, one is at "Understanding"
and the last one is at "Analyzing" level. Although three of these
four items require high-level thinking skills, they are not good at
distinguishing the highest group from the lowest one. This may mean that the
in-class activities may only remain at the basic levels such as remembering, understanding
and application in cognitive domain; and no place and time allocated for
activities that will promote high-level thinking skills of students. The fact
that students in the highest group made mistakes in items about telescopes,
constellations and galaxies may require teachers to make more detailed lesson
planning for such abstract contents. It is necessary to develop alternative
solutions such as developing related tools and materials, demonstrating videos
and simulation on the smart board and more importantly producing of such
technologies on a local basis.
Therefore, a valid and
reliable test is developed, and it is believed that this multiple-choice
achievement test is going to contribute to everyone working in this field
because it is developed for National Science Curriculum which was revised in
2018 and in the manner of renewed questions in transition exam to secondary
education.
