An Evaluation on Open and Distance Learning (ODL) Textbooks

Purpose: The present study aimed to investigate the compliance of ODE textbooks published by universities in and outside Turkey to textbook preparation criteria and identify the points that require improvement in those books. Methodology: A qualitative data analysis method known as document analysis was employed to achieve this. Using the convenience sampling method, the ODE textbooks published by 10 universities in and outside Turkey and were accessible online were determined as the sample of the study. The textbooks were evaluated based on the Open and Distance Education Textbook Evaluation Checklist, developed by a previous study. The checklist comprised 44 items distributed to four domains: individual learning, content, visual design, and language. Each item was scored based on the criterion's complete fulfillment, partial fulfillment, and non-fulfillment. Finding: The evaluation indicated that the textbooks showed good compliance to the criteria in terms of language and visual design while they needed improvement in the domains of individual learning and content, particularly in providing questions, examples, feedback sections, and in-text highlighting to improve the intelligibility of the content.


INTRODUCTION
As an alternative method to meet the educational needs of society, the history of ODL dates back more than a century.With the general definition of ODL, the instructor and the learner are in separate places, and the time limit disappears, printed materials, or a system in which learning and teaching activities are carried out using electronic tools in ODL systems (Moore, 2013).The interaction factor is also significant because learners and instructors are far from each other regarding time and space (Kaysi & Aydemir, 2017;Mutlu et al., 2005).An essential factor in the success of ODL experiences indicative interaction, affecting the academic achievement of learners, their continuity in the system, and their motivation to learn evaluated as a variable (Anderson, 2006).Thurmond and Wombach (2004) stated that the purpose of interaction.They stated that the content was better understood, and skill acquisition was achieved in reaching the gains.The interactive distance the Moore (1996) interaction proposes his theory; between the learner-content, the learner-tutor, and the learner-learner.Anderson and Garrison (1998) added the "Six Dimensions of Interaction" theory to Moore's interaction dimensions and added the dimensions of instructor content, lecturer-instructor, content-content interaction.These types of interactions are essential.However, learner-content interaction comes to the fore regarding individual learning in ODL.Contents in ODL environments.While it initially consisted of only texts, it was enriched with animation, sound, and images with the developing technology.In this way, the learner takes a more active role in the lesson since the learner, and the instructor is physically in different places.
Individual learning materials designed to study whenever, wherever and whenever they want have become very common over the years.One of the essential individual learning materials offered to learners in ODL is the course is the books.Textbooks are used to transfer, share, and store information in ODL (Bozkurt, 2013).The textbooks' preparation covers a long process, and the content must have specific characteristics (Bodur, 2016).Communication between them is provided mainly by printed or electronic materials (Soules, 2008). Soefijanto (2002) especially.The importance of learner-content interaction in distance education is increasing due to accessibility and low-cost states that he has won.The design of textbooks, especially individual learning, enables learners to be more effective and productive.It allows them to have a learning experience (Bozkurt et al., 2018;Senge, 2001).Individual Textbooks prepared based on learning have some outstanding features (individual speed, repetition, Etc.).Of these, one is the use of various activities to enhance and promote learning.These activities are a topic for learners.Engage learners with content by encouraging them to respond to the text rather than remaining passive while working on it to be in them; to think for themselves, to find solutions to the questions asked, and thus to learn what they have learned enables them to apply (Bullen, 2007).
Victim of the learner about your lessons about a hundred learners without learning from ODL and lesson violations may become appropriate (Karaağaçlı & Erden, 2008).This extra son is in ODL ideal for making suggestions and directing to benefit from learning emerges (Mandernach, 2005).Broadcast quality, writing, content and expression features, and interior and exterior visual design should show different features from other publications and formal education textbooks (Bodur, 2010).Also, the textbook should help learners reach their goals and needs in the fastest and best way (Aydın, 2006;Jacobs, 2015).For this reason, it is of great importance to determine whether the textbooks in OLS systems have these characteristics and present the rationale for the study.

Purpose of the Research and Research Questions
One of the essential components of ODL systems established with extraordinary financial expenditures is textbooks.Considering that it is also a mediator of learning content interaction, the importance of preparing these books becomes clear comes out.Ensuring the suitability of textbooks for individual learning in terms of content, structure, expression, and interaction, it is necessary to examine the reflections of scientific principles such as more practical design, content, language expression in practice.For this reason, the checklist previously created by the researchers (Yavuz et al., 2020) was used in the study, and the ODL determined the appropriateness level of individual learning, language and expression, visual design and content features in textbooks is intended.For this purpose, answers to the following research questions are sought: 1.What is the level of individual learning characteristics in the textbooks used in ODL? 2. What is the level of language and expression features in the textbooks used in ODL? 3. What is the level of visual design features in the textbooks used in ODL? 4. What are the content featured in the textbooks used in ODL? 5. What are the elements in the coursebooks used in ODL?Some need to be developed in individual learning, languageexpression, visual design, and content?

METHOD
In this section, the research model, sample, data collection process and data analysis are discussed.

Research Model
In the study, explanatory-descriptive case study, one of the qualitative research methods, is one of the types of case study used.A descriptive case study provides information and interpretation about a situation.It is a genre in which several situations are used (Datta, 1990).In this case study, the data is interpreted.Therefore, in the study, an explanatory-descriptive case study was used.

Sample
The universe of the study consists of ODL textbooks in the world.The sample of the study is from Turkey and outside of Turkey.It consists of the textbooks of 10 universities' institutions offering ODL.Distribution of universities included in the sample and the books found on the internet with the readily accessible sampling method belonging to universities are shown in Figure 1.
The easily accessible sampling method was preferred as the sample selection method.For this reason, only textbooks universities with internet access were selected.In addition, the books are informatics, social sciences, science, health, history, English and education fields.Universities depend on a standard form of their own in preparing ODL textbooks.It was thought to be sufficient to choose one book from each university and examine only the selected book.The reason is that the structure of each university book is like each other in itself.

Data Collection Process
As a data collection tool in the research (Yavuz et al., 2020), it was further investigated through document review.The previously developed "Open and Distance Learning Textbooks Evaluation Checklist" was used.Checklist, ODL It has been prepared to determine the criteria that should be included in the textbooks.The checklist consists of 44 items in total and these items were grouped under the categories of individual learning (13), content (12), visual design (13), and language-expression (6).In the excel program, the criteria in the checklist were tabulated in a row, with university names in the column.The researcher examined the books independently and the presence or absence of the criteria was scored (0, 1, and 2).
The creation of the "Checklist for Evaluation of Open and Distance Learning Textbooks" used in the study phase has been completed.In the first step to identify the items in the checklist, the literature was scanned, and a conceptual framework was created, and the items were arranged.In the literature, in the preparation of the textbooks used in ODL.These features were determined by analyzing 27 studies on what should be the features to be considered.Second at the second stage, Atatürk University Computer and Computer Science Department was used to evaluate the items in the draft form of the checklist.The opinions of three field experts from the Instructional Technology Education department were sought.Field Experts Distance Education Center and Open Education Faculty members regarding field experts' checklist items, the items that should be changed with their opinions were rearranged and the checklist took its final form.

Analysis of Data
Descriptive analysis, one of the qualitative data analysis types, was used in the research.Descriptive analysis is obtained with various data collection techniques.It includes summarizing and interpreting the collected data according to predetermined themes (Yıldırım & Şimşek, 2016).Previously created criteria are tabulated in MS excel.criteria in rows, university names in columns.The table is arranged as follows.The levels of finding the criteria were determined by the researchers and marked on the table.This 0, if there is no relevant criterion in the ODL textbook (for all criteria) to determine the level of presence of the criteria at the first stage, if it is at a certain level, it is scored as 1 and if it is at a certain level, it is scored as 2. Also, in the analysis the total score of the criteria based on the criteria and the average score of the criteria based on the item were calculated.Books from two universities analyzed separately by the researcher.Then, the consistency between these two researchers was examined.The consensus that gives consistency in the calculation should be at least 80% from each other (Miles & Huberman, 1994).The consistency between the researchers of these two books, which were evaluated independently, was calculated as 83%.Later on the other books to be examined were divided into two and the authors continued their analyzes separately.Researchers' reviews as a result, the criteria that differed from each other were reviewed and a consensus was achieved.

FINDINGS
In this section, the evaluations of the textbooks in terms of individual learning, language and expression, visual design, and content dimensions are given in the form of a table.University names are given in the tables by coding as U1, U2, U3, independent from Figure 1, due to ethical principles.Table 1 includes the evaluation results in terms of individual learning characteristics that should be included in ODL textbooks within the scope of the study.In terms of individual learning characteristics, the books were evaluated according to 13 criteria, and both the averages of the criteria and the total scores of the universities were given.
Table 1 includes the criteria for individuals to understand and apply the issues individually and evaluate themselves.At Table 1 "A comprehensive dictionary describing words and terms should be included", "The author should inform the reader about how to work with the textbook", and "The purpose and target of the subject should be given and planning should be made for timing" dimensions was found low mean.It is seen that the items with high averages are "informing the students about the target", "having a summary section at the end of the topic", "reinforcing the independent learning ability" and "giving students pre-regulators" have very high averages.Considering these criteria, it can be said that the books are at a good level in terms of preparing students for the concepts and information in the unit.However, it is noteworthy that pre-unit preparations and guidance for students are not sufficient.In general, it can be said that ALS textbooks should be developed in terms of paying attention to individual learning characteristics.In addition, some examples in terms of individual learning in the examined books are given below within the framework of the above items.

Figure 3. Examples of individual learning principles taken from books
When Figure 3 is examined, the picture on the left is an example of the statement "A repetitive question and summary should be included at the end of the chapters" in the fifth item.In contrast, the picture on the right is given as an example of the statement "It should contain activities to reinforce the independent learning in the eighth item.In the study's second research question, ODL books were examined according to 6 components in terms of language and expression features.As a result of the evaluation, both the averages of the components and the averages of the universities are given in Table 2.

Language and Expression
Sentence lengths should be appropriate for the student.

. Examples of context principles taken from books
When the Figure 6 is examined, the picture on the left is examples of the second item of the content category.It has been stated that the contents section should be in the item.The picture on the right is given as an example of "a general view of the subject" in the 11th item.When preparing ODL textbooks, it is necessary to keep eligible for individual learning, language and explanation, visual design, and content.However, it is seen that the textbooks examined within the scope of the study are not sufficiently provided by these features.Especially the elements that need to be developed in individual learning, visual design and content angle are given in Table 5.

Feedback Expressions to individual research (activities) orientation
Guides related to how students should work

Content
Additional reading or activities at the end of the unit

Keywords
Reinforcing samples related to subject

Visual Design
In-text highlights Inability to emphasize distinctions to be considered In Table 5, it is observed that the elements that make it easy to study in ODL textbooks, especially in the course books.In addition, it is also understood that the content should be considered in-text highlights to support examples, questions, and events and provide mindfulness in the text.

CONCLUSION and DISCUSSION
Today, ODL institutions, which are becoming more and more widespread, offer printed or digital textbooks to learners.Various features should use these materials to support their learning through these materials effectively needs to be prepared.In this study, individual learning, language expression, visual design and content in the ODL textbooks.It is aimed to determine the level of suitability of the features.In particular, the textbooks are suitable for individual learning preparation, an acceptable level of visual design, a more effective, efficient and satisfying learning experience for learners enables them to experience.In addition, the content is prepared correctly and effectively and reinforces the subject.The presence of examples, questions, and activities also facilitates the understanding and retention of the subject and increases satisfaction (Öztürk et al., 2017).
Students are informed about the objectives in the ODL textbooks prepared according to individual learning characteristics preparers are used (Moore, 2013), and feedback items are included same (Çepni et al., 2001).At the same time, the texts within the subject should enriched by considering different learning styles (BCcampus, 2018;Jacobs, 2015;Kaya, 2002;Lean, 1996).However, in the textbooks examined within the scope of the study, these features are sufficiently does not appear to have been complied with.Especially in the use of pre-preparers, it is generally accepted that the concepts related to the unit are given, but the unit is noteworthy that the instructions regarding the sections and applications in it are not given mistakes in the textbooks.It is also imperative to provide feedback to prevent learning.Facilitate learning and prevent mislearning to prevent this, unit evaluation questions should be given at the end of the unit (Kızılçaoğlu, 2003;Koçdar, 2006).In some of the books examined within the scope of the study, if the learners give wrong answers to the questions.There is guiding feedback.However, only the correct answer is given by giving multiple-choice questions in some of them.It is noteworthy that confirmation feedback is given by asking open-ended questions based on interpretation.This is the feedback type, and learners may not learn the reason for making mistakes.Instead, learners learn the source of the error.They should be given detailed feedback to understand (Shute, 2008).It is seen that the textbooks examined within the scope of the study do not include intriguing questions, but examples from daily life are presented.However, the only course content to get rid of orderliness, students' attention is drawn with intriguing and compelling questions (Ekici, 2003) and examples from daily life should be tried to withdraw (Willis, 2002).
One of the crucial criteria to be considered in the preparation of ODL textbooks is language and expression features.It is necessary to use plain and understandable language not to have difficulty understanding.Study in the course books examined within the scope of the study, it is stated that the terms and concepts suitable for the relevant content are not included and that the effective sentence structure is used.When the literature is examined, it can be seen that the comprehensibility of the content in the ODL textbooks using plain language (Kaya, 2002) and active sentence structure free of technical and complex words is emphasized (Lockwood, 2018).
Visual design and page structure are also fundamental in ODL textbooks.According to Megep (2011), the design should have integrity.While designing the books in the ODL books, the integrity of the design, the integrity of the pages in terms of the continuity of the student's attention must carry.In the textbooks examined, it was generally avoided to avoid page density, to work by taking notes.It is seen that there are spaces for those who want it.Intensive presentation of information in textbooks reduces productivity (Moore, 2013).According to Taş (2007), the density of the pages should be avoided, and students should take notes that required space should be left.In addition, it was ensured that the image related to the text was close to the text closeness was observed in the books examined.Considering the spatial proximity (Çepni et al., 2001;Kaya, 2002;Kızılçaoğlu, 2003).Visual according to Yılmaz and Yanarateş (2020).Design elements are also crucial in terms of mental development.
As a result, the ODL coursebooks of universities are generally prerequisites that increase readiness in terms of individual learning.Intermediate questions and activities supporting independent learning were included with the organizers.However, the term has determined that the dictionaries with explanations of the concepts are generally not given in terms of language and expression.As a result, the books are well prepared, the words and sentences are suitable for the level of the student, and there are no printing mistakes seen.The books are simply learning, the same standard in all units in terms of visual design.Features such as the adoption of the design and the presence of spaces on the margins of the page for the use of the learner came to the fore.In terms of content, unnecessary density is avoided in ODL textbooks, and an overview of the subject is given at the beginning of the unit and units were observed to be complementary.In short, language and expression and visual design of ODL textbooks.It has been determined that it is at a sufficient level in terms of individual learning and content, but it needs to be developed in terms of individual learning and content.The elements that need to be developed in the books are questions and examples that will facilitate understanding the subject, feedback presenting, and in-text highlighting.

SUGGESTIONS
Within the scope of the study, the critical suggestions that should be considered in the preparation of ODL textbooks are as follows; Individual learning characteristics should be taken into account in o Important terms should be bold or italicized.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest concerning this article's research, authorship, and/or publication.
OLS textbooks: o Engaging content with intriguing questions and examples to keep students' attention must be submitted.o It should be considered that students can learn in different styles.For this purpose, graphs and pictures should be used in different activities.o A comprehensive alphabetical glossary to explain words and terms should be included.o At the end of the unit, feedback should be presented to the learners in the questions given for the repetition of the topic.Content features should be taken into account in OLS textbooks: o The necessary information about the unit (such as activity, application and time) should be made.o More than one path in the transmission of content (such as text, pictures, and graphics) o The intelligence should be ensured that the content is made up of headings and subtitles.o A lot of samples and non-examples of transferred information should be given.Visual design features should be taken into account in OLS textbooks: o Pages should be visually exciting and straightforward.