Automatic detection of COVID-19 in chest radiographs using serially concatenated deep and handcrafted features

1Introduction

The rapid development in modern life-saving facilities helped the human community to get a better medical facility through multi-speciality hospitals. Even though sufficient diagnostic and treatment facilities are easily accessible, the happening rate of life-threatening diseases is steadily rising [1, 3, 5, 6].

The infectious disease caused by viruses and Bacterium affects the human community rapidly, and detection and treatment is the only remedy to control the spread of the disease to a large population. Recently, the infectious disease; COVID-19 affected a large number of the human group globally, and because of its harshness and spread rate, the World Health Organization (WHO) declared it a pandemic [2, 4, 37]. The SARS-CoV-2 virus causes this disease that initiates mild to severe pneumonia in patients based on their immunity level. Even though the patient vaccinates with the appropriate drug, the spread of COVID-19 is still uncontrollable, due to the rapid mutation in the virus.

The COVID-19 infection in a human is commonly diagnosed using the following clinical methods. (i) Reverse transcription-polymerase Chain Reaction (RT-PCR), in which the collected samples from the patient are examined using the approved clinical protocol, and (ii) Radiological image supported detection of the infection rate. In this technique, either the chest radiograph (X-ray) or the Computed-Tomography (CT) was collected from the patient. Finally, the collected X-ray/CT images were examined by the doctor to implement and to recommend treatment to cure the disease [8–12].

When the COVID-19 rapidly spread person to person, hospitals filled with patients and reduced the doctor to patient ratio. This situation causes a heavy diagnostic burden in hospitals. Therefore, computerized screening is always adopted to support the prescreening of the patient to generate the First Information Report (FIR) about the disease and its infection rate. The generated FIR is then examined by the doctors, and based on the report and doctor’s observation, the treatment is planned and implemented to recover the patient to reduce the disease impact [18–21].

This research aims to develop an automatic system to detect the COVID-19 infection in chest X-rays with better accuracy. This scheme implemented a Deep-Learning-Method (DLM) using the following stages; (i) Image collection and pre-processing, (ii) Pre-trained DLM implementation and performance evaluation, (iii) Combining the deep and machine features, (iv) Feature optimization with Firefly-Algorithm (FA) and (v) Feature integration and classification.

In this work, initially, the performance of the chosen DLM is verified using the SoftMax classifier. After selecting the DLM with the best performance, its features are then combined with the machine features to verify the performance of COVID-19 detection. During this work, along with the SoftMax classifier, other two-class classifiers from the literature are considered. The detection performance of the system is verified using a 5-fold cross-validation process. In this research, the SVM-RBF classifier helped to get a better COVID-19 with the deep features (> 92%) and, Fine KNN provided improved accuracy (> 96%) for combined optimal features.

The main contributions of this research work are;

The remaining parts of this study are as follows. Section 2 presents the context, section 3 presents the various phases existing in the proposed scheme, and sections 4 and 5 depict the results and conclusion of the present work, respectively.

2Context

The detection of lung infection using chest X-ray is a commonly approved technique and, several previous works can be found in the literature using conventional machine learning and advanced deep learning to detect the virus cases [1, 3, 23]. COVID-19 is a deadly disease, which causes severe lung infection. Due to its rapid spread nature, it affected a large human community globally and, several studies are proposed to diagnose this infection with better accuracy. The detection of COVID-19 using X-ray is a commonly adopted scheme and several earlier research works existing in the literature are summarized in Table 1.

The earlier work presented in the above table either uses a new deep learning method (DLM) or a conventional machine learning method (MLM) to detect the COVID-19. The former study in the literature authenticates the combination of the features of DLM and MLM, which helps to get better disease detection in various modality medical images [26]. The proposed work tests the value of the developed structure using DLM first, then the features of the DLM and MLM are optimized with FA. After reducing the feature vector to a lower level, a serial concatenation is then implemented to get a new feature vector, which is then considered to train and validate the disease detection performance of the proposed scheme. The main merit of the proposed scheme compared to the other methods of Table 1 is the integration of features and FA based feature optimization. This approach helps to get a better result compared to other methods considered in this work.

3Method

This part of the work presents the various methods integrated to form the proposed scheme. Figure 1 presents the implementation of the DLM to classify X-ray images into normal/COVID-19 section. In this work, the well-known DLM, which uses an image dimension of 224×224×1 pixels, is considered in the investigation. Based on the attained accuracy, the appropriate DLM for the chosen database examination is confirmed. The considered classification task is implemented using the traditional deep features extracted using the DLM. The SoftMax based classification is employed and based on its result, the VGG16 model is selected.

Fig. 1

Pre-trained DLM based chest X-ray classification.

Figure 2 presents the proposed scheme, which integrates the FA optimized deep and machine features. The necessary machine-learning features were extracted using the Local Binary Pattern (LBP) of various weights as discussed [7]. The FA optimized deep and LBP features are serially integrated, and this feature vector is then considered to train and validate the two-class classifier. In this work, 5-fold cross-validation is employed and based on the achieved result, the performance of the proposed scheme is confirmed.

Fig. 2

Structure of the proposed COVID-19 detection system.

3.1Image database

The performance of the computerised disease assessment scheme mainly depends on the medical database considered to train the system. In this work, the available chest x-ray database [38] is considered for the assessment. From this dataset 3600 greyscale images (1800 disinfected and 1800 COVID-19) are taken for the experiment. The number of training (70% of the total images) and validation (30%) adopted in this study is presented in Table 2 and, the sample test images of this database are presented in Fig. 3. Each depiction is resized to pixels, before the assessment with the DLM.

Table 2

Test imagery considered for investigation

Image type	Dimension	Images
	Total	Training	Testing
Healthy	224 × 224 × 1	1800	1260	540
COVID-19	224 × 224 × 1	1800	1260	540

Fig. 3

Sample X-ray images of the chosen database.

During the 5-fold cross-validation process, the sample test images are shuffled arbitrarily and, the new dataset consists of 1080 images (540 disinfected and 540 COVID-19 class) to get a dataset of 5-groups. In each class, 540 images are considered during the cross-validation, and the performance of the classifier is tested and verified.

3.3Feature optimization

This work consists of the deep and machine features obtained using a chosen procedure. The deepfeatures are initially extracted using the chosen pre-trained DLM. Initially, an experimental investigation is conducted with the chosen DLM and the classification outcome of SoftMax is then verified. This outcome confirms that the result of VGG16 is better compared to other DLMs considered in this research.

The complete information about the VGG16 can be found in [5, 24] and in this work, the following parameters are assigned for the VGG16; conventional image augmentation technique.It is assigned during the learning. The learning rate is fixed as 1e-5, Linear-Dropout-Rate (LDR) based training with and Adam optimization. The numbers of iterations are assigned as 2000 with 50 epochs and SoftMax classifier with 5-fold cross-validation. This DLM helped to get a total feature of 1 × 1 ×4096 from the final Convolutional + Maxpool layer. These features are then passed through three fully connected layers having a dropout value of 50% and after this dropout, the original deep features are then reduced to 1 × 1 ×1024. These features are the prime deepfeatures to check the merit of the proposed system. Equation (1) presents the deep-feature of VGG16;

(1)

DeepfeatureVGG16(1×1×1024)=VGG16(1,1),VGG16(1,2),...,VGG16(1,1024)

The necessary machine-features are then obtained using the Local-Binary-Patterns (LBP) developed using varying weights (W = 1 to 4) as discussed in [7]. Every pattern provides a feature vector of dimension 1 × 1 ×59 and, the total features obtained from these images is 1 × 1 ×236 and this information is depicted in the Equations (2) to (6);

(2)

LBP1w1(1×1×59)=W1(1,1),W1(1,2),...,W1(1,59)

(3)

LBP2w2(1×1×59)=W2(1,1),W2(1,2),...,W2(1,59)

(4)

LBP3w3(1×1×59)=W3(1,1),W3(1,2),...,W3(1,59)

(5)

LBP4w4(1×1×59)=W4(1,1),W4(1,2),...,W4(1,59)

(6)

MachineFeatures(1×1×236)=LBP1+LBP2+LBP3+LBP4

The FA based feature optimization is then employed to reduce the features of Equations (1) and (6). The necessary information about the heuristic algorithm-based feature selection is available in [8] and in this work, the maximization of the Cartesian distance is chosen as the objective value. The features whose values are more is selected and the features whose values information is less are discarded. In this work, 1 × 1 ×611 number of deep features and 1 × 1 ×117 machine features are selected by the FA and a serial concatenation is then implemented to integrate these features.

The FA is initially invented by Yang in 2009 [34, 35] based on the mimic of the Firefly’s foraging strategy and due to its merit, the traditional and modified form of FA is widely adopted to solve a variety of optimization tasks [15, 22, 30]. The traditional FA consist a Lévy flight operator, which helps to move the flies towards the optimal solution with better accuracy. In the proposed work, the FA is employed to optimize the features by maximizing the Cartesian Distance (CD) among the features. This process can be depicted as in Equation (7);

Let us consider, there exist two fireflies; i and j located in d_i and d_j in a D-dimensional search location. Mathematically, this CD can be expressed as;

(7)

Dijt=∥djt-dit∥2=∑k=1n(dj,k-di,k)2

During the optimization task, the FA is allowed to find the feature subset whose Dijt is maximum.

The movement of a Firefly towards another Firefly is monitored. It is based on the Cartesian distance and this concept is utilized in this work to find the optimal features. The FA is allowed to compare the feature value of normal and COVID-19 class and, if the distance between the features is more, it is selected as the optimal features. Other information in relation to the FA- based feature selection can be found in [28, 33, 36].

(8)

Combinedfeatures=(1×1×611)+(1×1×117)=1×1×728

The feature vector in Equation (8) is then considered to train and validate the system.

The feature optimization process is depicted in Fig. 4. The FA based optimization is continued till the optimal features are identified from the considered feature set.

Fig. 4

FA based feature selection procedure.

4Results and discussion

This the part of research reveals the outcome attained with the workstation; Intel i5 processor with 20GB RAM and 2GB VRAM operational with MATLAB®.

During the experimental investigation, the necessary images are collected from the database, and then every image is resized to 224 × 224 × 1. The resized image is then considered for the experimental investigation. In this work, 1800 numbers of the test images of Normal/COVID-19 class are considered for the examination. Initially, the performance of pre-trained DLM is established by this image database and the classification accuracy achieved with SoftMax is then considered to choose the appropriate DLM for the COVID-19 detection with X-ray.

The heat map obtained from the chosen test image of disinfected/COVID-19 class during the VGG16 implementation is depicted in Fig. 5. This image confirms the existing dissimilarity between the images and the visibility of the lung section in disease infected section is less compared to the normal section. The results obtained for the pre-trained DLM is depicted in Table 3 and this result confirms that the classification accuracy achieved using the VGG16 is better than other methods. Hence, in this work, the VGG16 scheme is considered to examine the chest X-ray with the proposed method.

Fig. 5

Heat map of the X-ray image for normal and disease class.

The performance of VGG16 is depicted in Fig. 6, in which Fig. 6(a) presents the search convergence along with obtained values of accuracy and error rate. The confusion achieved during the classification is presented in Fig. 6(b). From this result, it can be noted that the VGG16 scheme with traditional deep feature helped to achieve a categorization accuracy of > 90%.

Fig. 6

Performance achieved with the VGG16 scheme.

Table 4 presents the outcomes during the 5-fold cross validation process and the achieved accuracy variation is depicted in the Fig. 7(a) and (b), presents the overall performance of the pre-trained DLM, which confirms that the Glyph-pattern developed for VGG16 is bigger (superior) compared to other schemes.

Fig. 7

Performance assessment of the DLM.

After selecting the appropriate DLM for evaluating he X-ray database, the FA based feature optimization is selected. It is then employed to reduce the deep features and machine features.

Figure 8 depicts the LBP pattern generated with weights (4) with a value of 1 to 4 and the number of features obtained with these images is 1 × 1 ×236. It is then reduced to 1 × 1 ×117. This feature is then reduced with the optimal deep features and the classification task is implemented with the classifiers considered in this work. The result achieved during the experimental investigation is presented in Table 5. This table separately presents the outcome with optimal deep-features and combined deep and machine features. This table confirms that the proposed work helps to get the better result with deep and integrated features. Figure 9 depicts the confusion matrix achieved with SVM-RBF and Fine KNN classifier. This confirms the merit of this work. Figure 10 presents the Receiver Operating Characteristic Curve (ROC) achieved for Fine KNN classifier with Deep + Handcrafted features.This confirms the merit of the proposed scheme,which helped to get better values of Area Under Curve (AUC) and p-value.

Fig. 8

LBP generated using weights W = 1 to 4.

Fig. 9

Confusion matrix obtained with SVM-RBF and Fine KNN.

Fig. 10

Performance evaluation of proposed technique with Fine KNN classifier.

The overall performance depicted in Fig. 11 also considers the superiority of proposed scheme. Figure 11(a) and (b) presents the Glyph-Plot, constructed using ACC, PRE, SEN, SPE and NPV of Table 5. Every pattern in this plot is the combination of these features and every feature forms a straight line in this pattern. For example, this pattern consist of 5 lines starts from the origin of the pattern, which represents the 5 parameters, such as ACC, PRE, SEN, SPE and NPV. When the value of the considered parameters is more, the dimension of the pattern is also more. By simply, observing the pattern dimension, the merit of a particular classifier can be measured.

Fig. 11

Glyph-plot to demonstrate the overall performance of deep features and integrated features based COVID-19 detection.

Table 5 also confirms that the classification accuracy in Deep + Handcrafted features is good compared to the investigation implemented only with the Deep features. The proposed scheme helped to improve the detection accuracy. Figure 12 presents the disease detection performance of the Fine KNN with Deep + Handcrafted features and this result confirms that the images are detected without the overfitting.

Fig. 12

Probability of detected image with its real class.

The outcome of the proposed scheme’s classifiers accuracy is then compared as presented in Fig. 13. This confirms that the classification accuracy of proposed technique is good in all the considered two-class classifiers.

Fig. 13

Classification accuracy of proposed technique.

The proposed work implemented integrated feature based classification of the X-ray and in future, the proposed technique can be improved by considering other handcrafted features available in the literature.

5Conclusion

COVID-19 is one of the fast-spreading infectious diseases in humans and the timely detection and treatment helps to treat the patient with appropriate procedure. Accurate detection of the COVID-19 is necessary to treat the patient with suitable medication. DLM based detection of the COVID-19 infection using X-ray images are demonstrated in this work. This work implements; (i) Detection of the lung infection using the optimized deep features and (ii) Infection detection using the combined features. In this work, the FA based feature optimization is employed to select the dominant and reduced feature vector, which helps to get better disease detection with a better accuracy. This investigation is executed using 1800 images of normal/COVID-19 using the VGG16 extracted features and LBP-based features. During this investigation, 70% of test pictures are considered for training the scheme and remaining 30% images are considered for the validation. The classification task is executed using binary classifiers with a 5-fold cross validation. The outcome of the present research confirms that this research offered an accuracy of > 92% with SVM-RBF and > 96% with Fine KNN. In future, this technique can be tested and confirmed with the real patient images collected from hospitals.