Detection of coronavirus disease (COVID-19) from X-ray images using deep convolutional neural networks

COVID-19 is an epidemic disease that seriously affects elderly people and patients with chronic diseases and causes deaths. Fast and accurate early diagnosis has an important role. Although chest images obtained by computed tomography are accepted as a gold standard, problems are often encountered in accessing this device. For this reason, it is very important to diagnose with more accessible devices such as x-ray machines. These studies have been accelerated with deep neural network models and good results have been obtained. In this study, two different approach models are proposed for this purpose. At first study, training with the COVID-19 data set shared as open access and the test results with different classifiers. The other is the comparison of the results using a Pre-trained model MobileNet. COVID-19 patients, pneumonia patients and normal individuals were classified with 99.53% accuracy by the designed CNN with SVM model which was trained with the COVID-19 data set. As a result, because X-rays are a special type of image, a CNN model trained with X-ray images would be a good choice rather than using pre-trained deep networks with different images. As a result, since X-rays are a special type of picture, it was seen that a CNN model trained with X-ray images should be a better choice, rather than using pre-trained deep networks with different images.


Introduction
COVID-19 disease was first record in Wuhan and spread from Wuhan to the whole world in last quarter of 2019. The disease has affected the whole world in a short time. Because of its rate of spread it has been declared as a pandemic disease by the World Health Organization (WHO), At the end of 2020, WHO reported the number of patients worldwide exceeded 45 million and the death toll reached over 1 million in about a year and was increasing day by day (WHO, 2020) Real-time reverse transcription-polymerase chain reaction is used popular test method for diagnosis of COVID-19. This method was time consuming, costly method and low sensitivity of 60%-70% (Ardakani et al., 2020). The symptoms of disease can be detected in radiological images of patients even if test result was negative (Kanne et al., 2020;Xie et al., 2020). Chest X-ray (CXR) and computed tomography (CT) are important chest imaging techniques in early diagnosis and treatment of COVID-19 pneumonia (Zu et al., 2020). Although CT imaging provides more details, CXR are an easier, faster, more economical and less harmful alternative instead of CT (Narin et al., 2020;Ozturk et al., 2020). Edgar (2020) reported CXR images taken at continues days for a 50-year-old COVID-19 patient. Early diagnosis is very important to reduce the effect of the virus as with other critical diseases (Lin et al., 2005;Badnjevic et al., 2018). Since the virus attacks the lungs in a short time, it causes severe pneumonia. It shows some obvious symptoms such as dry cough, fever and difficulty breathing. Therefore, in modern healthcare systems, radiography examination can be used faster and more frequently, given the prevalence of imaging systems and the availability of portable units for chest radiology. This has made CXR imaging a part of the standard procedure, usually for patients with respiratory complaints (Wang & Wong 2020). However, due to the workload that will increase as the number of patients increases, correct diagnosis by visual inspection of medical images becomes challenging (Nihashi et al., 2019;Taylor-Phillips and Stinton, 2019). Moreover, it has been reported that the disease can be diagnosed easily with clear images that occur only 10-12 days after transmission and are understood by radiologists . For this reason, machine learning-based computer aided diagnostic systems have been developed to assist experts (Vasilakos et al., 2016;Faust et al., 2018). However, the scarcity of COVID-19 CXR image is a disadvantage for deep learning algorithms and it is insufficient to train a deep neural network effectively. Therefore, transfer learning could be a viable solution in this situation and has been widely adopted in many studies about detection of COVID-19 recently proposed (Zhang, 2019;Wang & Wong, 2020;Apostolopoulos & Mpesiana, 2020;Sethy & Behera, 2020;Falk et al., 2020;Narin et al., 2020). However, traditional transfer-learning models using pre-trained deep learning networks with the ImageNet database cannot be a good choice, as the properties of COVID-19 CXR images are different from other images. Furthermore, differentiating pneumonia patients with traditional viral or bacterial infections from COVID-19 patients with significantly overlapping characteristics is a challenging problem.
In the literature, various deep learning approaches to the diagnosis of COVID-19 from CXR images are available. Looking at some of these studies: the CXR images were classified COVID-19 cases, bacterial and viral pneumonia cases and normal cases using the Xception architecture (Khan et al. 2020). The diagnosis of COVID-19 from CXR images was reported to be performed using the dropweights based Bayesian CNN model (Ghoshal & Tucker, 2020) to be performed the Bayesian optimization based SqueezeNet model (Ucar & Korkmaz, 2020). Another study was performed to distinguish both COVID-19 patients from healthy individuals and from healthy individuals with COVID-19 and pneumonia using the DarkNet deep learning model (Ozturk et al., 2020). Using DenseNet and VGG19, COVID-19 was diagnosed from CXR images (Hemdan et al., 2020). Wang and Wong (2020) proposed the COVID-Net model for COVID-19 detection from CXR images (Wang &Wong, 2020). Mahmud et al. (2020) proposed a new CNN model named as CovXNet, to detect COVID-19 and other pneumonia (Mahmud et al., 2020). In another study, it was attempted to diagnose with a cost-sensitive learning model using CXR images (Li et al., 2020).
In this study, a CNN model has been developed in which CXR images are applied to classify COVID-19 patients from pneumonia patients and normal individuals. Firstly, the data was evaluated with image processing and data scaling approaches, then used in CNN trainings. In addition, a comparison was made using a pre-trained deep learning model. Throughout this study using open source data sets, deep learning algorithms were repeated for the dataset of COVID-19 patients, pneumonia patients and normal individuals.

Data Set
In this study, CXR images database known as "COVID-19 radiography database", which is available to open access by Kaggle, was used (Chowdhury et al., 2020). The database created by researchers from different university such as Qatar University, the University of Dhaka and their collaborators from Malaysia and Pakistan. The database consists of three type CXR images for COVID-19 positive cases, Viral Pneumonia images and Normal CXR images. There are 1200 COVID-19 positive case, 1345 viral pneumonia and 1341 normal images in COVID-19 radiography database. Figure 2 shows sample images belonging to these three groups. The number of data should be high enough to be better trained and to increase the classifier performance for the deep learning model, If the dataset size is small, over-learning often occurs when training classifier parameters. In order to prevent excessive learning, it is ensured that the number of data can be increased by adding synthetic data similar to the existing data. There are different methods of adding data depending on the image type and data set: for example, rotation, scaling and trimming, resampling, cropping, adding noise, zooming, horizontal displacement, vertical displacement etc.

Convolutional Neural Network
Convolutional neural networks (CNNs) are a machine learning model used to obtain results by directly processing applied data. That is a feature learning model. CNN has become very popular lately, as it has shown impressive results in image processing, especially for classification, detection and segmentation purposes (Goodfellow et al., 2006;Han et al., 2018). A standard CNN model consists of convolution, pooling layers and a fully connected layer, or in other words, the classification layer (Han et al., 2018). The convolution layer is main part layer of the model and gives its name to the model. It is performing feature transformation by processing the input data with filters on convolutional layers. The pooling layer is used to reduce the number of feature and parameters, thus reducing the computational cost. The fully connected layers work like the multilayer neural network in traditional machine learning. Instead of network learning various machine learning algorithms can be performed in this layer such as k-nearest Neighborhood, Support Vector Machines (Altan et al.,2018;Altan et al., 2019;Camgözlü & Kutlu, 2020).
The parameters are determined experimentally such as how many layer, how many filter, filter size of convolution layers etc. The selection of model parameters such as learning coefficient and number of iterations plays an important role in the training of the CNN model. For example, excessive selection of iterations causes over-learning of model. However, for training a deep learning model, a very large data set and a highly capable system are required. In the literature, there are some CNN models which trained with high images using super computer systems. CNN models such as imageNet, VGG, AlexNet, DenseNet, MobileNet, ResNet have been developed by arranging different layers and different combinations (Krizhevsky et al., 2012;Howard et al., 2017). With using trained models, the learning transfer method has been developed (Kaur & Gandhi, 2020) so that this problem can be overcome. Such deep learning models are described as "pre-trained models". In this method, parameters of a trained model are transferred to the model of interest by using a different big dataset. MobileNet is widely used Pre-trained model in many realworld applications which includes object detection, MobileNet was used in this study. It can work with pictures of different sizes. In this study, 224x224 color images were used. This frequently used model uses the weights of training with ImageNet data set. Figure 3. Example x-ray images of (a) patients with COVID-19, (b) patients with Pneumonia, and (c) normal subjects.

K-Nearest Neighbor
One of the most used machine learning algorithms in the literature is K-Nearest Neighbor (KNN). There is no training approach in this algorithm, which is basically a simple algorithm such as calculating the distance between two points in space. The training data set is available in this algorithm and the newly arrived data are determined according to the closest sample by calculating their distances to all samples in this set. KNN also uses different distance calculation functions such as city block, Euclidean, cosine, Mahalanobis. In addition, the other parameter is number of nearest neighbors.

Support Vector Machine
One of the most used supervised machine learning algorithms in the literature has been Support Vector Machine (SVM). SVM can be described as a vector space-based machine learning method. The algorithm allows the line to be drawn to be adjusted in two classes so that it passes from the furthest place to its elements (Noble, 2006). Different kernel functions can be used in SVM models such as linear, polynomial, sigmoid, radial basis, etc. SVM classifiers can also classify linear and nonlinear data.

Extreme Learning Machine
Extreme Learning Machine (ELM) is a single-layer feed forward networks (SLFFN) with random or fixed weights and a learning method that uses pseudo-inverse conditions. ELM was proposed by Huang (Huang et al., 2006) for SLFFN to achieve extremely fast training and high generalization performance. Training methods from traditional network learning algorithms are different from ELM. In ELM, input weights and latent neuron bias are randomly selected. Output weights are determined by Moore Pensore using generalized inverse conditions (Kutlu et al., 2015). That standard neural network has activation function, hidden nodes, etc. The algorithm can approximate these N samples with zero. It means that      The equation can be rewritten as

T H  
(2) Thus, a learning method for network called ELM can be described as Pseudo inverse methods,

Performance Measures and K-Fold Cross Validation
There are different methods to measure the performance. Accuracy, recall and precision are frequently used in classification problems and Mean Absolute Error, Mean Absolute Percentage Error are frequently used in function estimation applications in literature. (Kutlu, 2010). Four different measurements were used to evaluate the predictive performance of classifiers in this study.
Here, TP indicates the number of positive patients classified as correctly, FN is the number of positive patients who are incorrectly classified as unhealthy, TN is the number of non-patients classified as correctly, and FP is the number of people are incorrectly classified as patients but are not actually healthy (Kutlu, 2010;Kutlu et al., 2015).
In classification problems, the performance of the developed model against data, which it has not used before, is taken into account. For this reason, showing higher training performance does not mean that his performance will be high against test data which has not used before (Duda & Hart, 1973). Therefore, in all classifier studies, at least two data sets are created as train dataset and test dataset. Since the test data set consists of data not used during training, performance is evaluated over the test data set. In K-Fold cross validation method, the dataset is divided into k subsets. While k-1 subset of these is used as train set, one of them is used as test set. This process is repeated until the entire subset is used for testing. The classifier performance is calculated as classifier training performance and classifier test performance by taking the averages separately for training and testing results (Wong & Yang, 2017). In this study, the fold value of k is taken as 10 because the data set is large.

Results
In this study, all algorithms and calculations were performed using the Python programming language (Chollet, 2019) and the Tensorflow Keras library (Gulli & Pal, 2017). CNN trainings were carried out using the hardware that consisting of AMD Ryzen 5 3600x CPU, 32GB RAM and GTx 1080 GPU etc. The COVID-19 dataset which has approximately 7700 images in 3 different classes was used. Adam optimization algorithm was used for training deep neural networks (Kingma & Ba, 2014). The performances with different classifiers were examined using the trained CNN model. In addition, performance evaluation was made using the MobileNet deep learning model as a pre-trained model. The x-ray images were applied rescaling (1/255) before training the model. Thus, the proposed model and pre-trained model were compared. A 10-fold crossevaluation was applied to obtain the generalization performance of the classifier.
The model used in this study, analysis of performance rate changes according to pooling layer type and number. In addition, the model was created as a result of the filter analysis in the convolution layer (Camgözlü & Kutlu, 2019;Camgözlü & Kutlu, 2020). The model used consists of 6 convolution layers and 3 average pooling layers. Convolution filter size was determined as 3 and ReLu was used as the activation function in these layers. Finally, the pictures used in the training of the model are turned gray and are 250x250 in size.
Chest X-ray images consist of COVID-19 patients, pneumonia patients and normal individuals. A new CNN model have trained using CXR images. In addition, the Trained CNN model and the Pre-trained MobileNet model were used to examine their performance using different classifiers. The all images in dataset is separated as 80% training set and 20% test set. The CNN model training was provided by using training set. The training performance graph is shown in Figure 4. The test performance of the models was examined. At the end of the CNN training, 99.30% training performance and 96.5 test success were achieved. Another approach, the CNN model, whose training was completed, and the pre-trained MobileNet model were used to examine their performances with different classifiers. The ELM classifier performance is indicated in Table 1 according to number of neuron size. This table summarizes the results of studies in which COVID-19 patients were tried to be distinguished from pneumonia patients and healthy individuals within the same classifier and the results obtained in this study. Ozturk et al. (2020) achieved 87.02% classifier performance using the DarkNet model with 17 convolution layers. In another study, Wang and Wong were able to achieve 92.60% classifier performance using a new model called COVID-Net . Apostolopoulos and Mpesiana achieved 94.72% accuracy using the deep learning model of MobileNet architecture (Apostolopoulos & Mpesiana, 2020). In another study, Li et al, (2020) achieved a classifier accuracy of 97.01% using a cost-sensitive learning model. In this study, the classifier performance of 99.53% was obtained by using the developed CNN model with SVM classifier. This is the highest performance. Accordingly, it can be said that models trained with COVID-19 data give better results than the pre-trained model. The COVID-19 diagnostic tool has the potential to be a useful diagnostic support system for medical practitioners. If such a COVID-19 diagnostic tool is used, it is thought that the heavy workload of physician will decrease and the number of overlooked diagnoses due to workload will decrease.
Although the number of data in this study is greater than most studies in the literature, it is still insufficient for us to reach a general judgment about whether the proposed method is effective in such a fatal epidemic disease due to different symptoms. It is possible to say that pre-trained and parameter transfer-based deep learning models work as well as specially trained models.

Data availability statement
The data that support the findings of this study are openly available in Kaggle.com known as "COVID-19 radiography database",at [10.1109/ACCESS.2020.3010287], reference number [arXiv:2003.13145].