Soft computing and image processing techniques for COVID-19 prediction in lung CT scan images

1.Introduction

Humans have encountered many contagious diseases throughout history, resulting in pandemics and epidemics [1]. There were no advanced answers to these difficulties in the past, which resulted in several negative human consequences. COVID-19 [2] is a novel disease kind that has arisen. It’s a member of the SARS family [3]. According to WHO, there were 198,778,175 confirmed cases of COVID-19 as of mid-August 2021, with 4,235,559 deaths and a total of 3,886,112,928 vaccination doses [4]. However, unlike in the past, there are now significantly more effective methods for detecting and diagnosing COVID-19, including RTPCR [5]. However, RT-PCR is not only time expensive, but it also has a high false negative rate [6]. As the prevalence of covid 19 grows, a better approach is required to address these issues. This is where AI [7] comes in. In the past, AI has proven to be extremely beneficial in the medical field [8]. AI has come a long way. In [147]. Elleuch Mohamed et al. employed a pre-trained VGG-16 architecture to recognise characteristics in plant leaves in agricultural fields. Valappil et al. applied CNN-SVM machine learning method [148] for vehicle detection utilising Unmanned aerial vehicles (UAVs). For Arabic word detection from natural photos, Oulladji et al. used ensemble learning approaches such as Support Vector Machine, Neural Networks, and Adaboost boosting algorithm in [149]. However, because AI alone cannot solve the problem, CT scan and X-ray images are used. Using CT scan images and transfer learning techniques, Souza et al. utilized Mask R-CNN for lung segmentation in 2021 [150]. In the medical industry, CT scan and CXR images are employed for a variety of purposes [9, 10]. Deep Learning [11], Machine Learning [12], and Image Processing [13] are some of the AI technologies that can be utilised to distinguish COVID-19 patients from CAP (Community Acquired Patients) [14].

This paper discusses a few ways for detecting COVID-19 in CT scan and X-ray images utilising Machine Learning, Image Processing, and Deep Learning techniques. The models that we implemented using Deep Learning and Pretrained models are also included in this work [15]. Websites such as IEEE Xplore, SpringerLink, and Arvix were used to choose the papers. Google Scholar was utilised to find papers, and the key words used were COVID-19, CNN, Machine Learning, Image Processing, Deep Learning Techniques, Pretrained model and Datasets. The purpose of this publication is to give future scholars with knowledge about COVID-19 prediction via AI techniques and also predict COVID-19 with different AI technique. To work with these models, there should be a suitable dataset; the dataset should be divided into training and testing sets at a ratio of 10:90, but this might vary depending on the model. ImageNet is one of the datasets which has over 10 million images [112]. The data that is used to train the model, also known as seen data, is data that is processed repeatedly in order to enhance the model’s accuracy. The data used to evaluate whether the model is performing properly or not is known as testing data or unseen data. To minimise overfitting, the dataset is sometimes segregated into validation data, which is used to validate the data after using the testing data to ensure that the model is not becoming accustomed to the testing data. It occurs as a result of the model being trained in the training data for an excessive amount of time. It removes irrelevant data from the training data and adjusts the model’s memory to be as near to it as possible. Overfitting is caused by a low error rate in training data and a high error rate in testing data. Underfitting refers to when a model is unable to create a relationship in the model, resulting in a high error rate in both training and testing data. It is caused by a model that is too simplistic, has been trained insufficiently, or has a dataset with insufficient characteristics. It is also created by forcing the model to end training in order to minimise overfitting. This procedure is known as early stopping. To prevent the issues of underfitting and overfitting, the machine learning algorithm should ensure that the data fits the model perfectly. The first section of this paper consists of Introduction, the second section provides information about different research papers of deep learning techniques, the third section provides information about different studies which have used image processing technique, fourth section mentions the different papers that have been used in this study using machine learning techniques. Fifth section is about our COVID-19 prediction pretrained model. Sixth section is about our deep learning model for COVID-19 prediction, seventh section is the comparison table of different papers. Eight section is conclusion and Ninth Section is References.

2.Methods for deep learning technique

Neural networks are the most common deep learning method. Deep learning is utilized in image recognition, speech recognition, and other applications. It’s also used to categorize things. Deep learning is a subset of machine learning, in which basic concepts are employed to train the model, whereas artificial neural networks are utilized to mimic human behaviour in deep learning. The number of layers in an ANN indicates how deep the network is. The neural network works in a similar fashion to the brain. Artificial neural networks, like neurons in the brain, include nodes that carry signals. Thousands of these nodes are connected to each other and carry the signals all over the place. Deep learning, as a result, necessitates the use of sophisticated hardware. When there are several output categories, the dataset should be labelled dataset. Because the dataset being used is so enormous, training takes a long time, sometimes even weeks. As a result, several researchers apply transfer learning techniques to overcome the time problem. Transfer learning is the act of taking an existing model that has previously been trained for a short period of time and then inputting new data into the model that the researchers want to model by making modest changes to the network. The time it takes to train the model is reduced to hours or potentially a few days because the model has already been taught for a while and does not have to start from scratch. Deep learning is used for feature extraction, and the performance of deep learning models can be improved by using GPUs.

Zhao et al. suggested [119] in 2020. They provided datasets and used DenseNet to create their model, which had an accuracy of 84.7 and precision of 97 using CT scan pictures. In 2020, Hilmizen et al. proposed [31] for covid 19 detection using CT scan and X-ray images. They used various models such as ResNet, DenseNet, and others, but the best results were obtained by combining ResNet50 and VGG16, as well as by combining Densenet121 and MobileNet. Both methods had a similar accuracy of 99.87 and sensitivity of 99.74. Islam and Matin proposed [32] in 2020 that they employed Lenet CNN for COVID-19 detection and achieved accuracy of 86.06 percent and precision of 85 percent with CT Scan images. Anwar and Zakir proposed [33] in 2020 that they used EfficientNet b4 with CT Scan pictures to detect COVID-19. They employed three learning rates: plateau, cyclic, and constant, however plateau produced the best results, with an accuracy of 0.90. Sari et al. proposed [34] in 2020, using CNN with CT scan pictures to achieve a precision of 98.0 and an accuracy of 97.57. Tabik et al. suggested [37] in 2020, using COVID-SDNET, which combines many approaches such as ResNet, U-Net, FuCiTNET [38], and CNN, to achieve an accuracy of 81.00 percent. In 2020, Khan et al. employed inflated Inception and ResNet50 with CT scan images for COVID-19 detection [39], with an accuracy of 0.84. They used two separate datasets, CC-19 and Covid-CT. In 2020 [40], James and Sunyoto employed CLAHE (Contrast Limited Adaptive Histogram Equalization) [41] in conjunction with CNN to achieve an accuracy of 83.28 percent and precision of 81.57 percent. In 2020, Dastider et al. proposed [42] that they used Otsu Thresholding [43] and CNN in their work for not only COVID-19 detection but also for distinguishing between normal, COVID-19, Viral Pneumonia, Bacterial Pneumonia, and Mycoplasma Pneumonia patients using CT scan images while also working with ResNet152V2 as backbone for their ResCovNet architecture. In 2020, Mohammed et al. applied CNN and updated ResNet in COVID-19 detection using CT scan images, achieving a precision of 0.819 and accuracy of 0.776 [44]. Wang et al. used 3D U-net [45] and 3D ResNet combined with CNN for COVID-19 identification in 2020 [46], achieving 93.3 percent accuracy and 87.6 percent sensitivity. In 2020, Cia et al. proposed [47] using CT scan images to diagnose COVID-19. They utilised CNN and ResNet and obtained an accuracy of 0.94 and precision of 0.97. Tabarisaadi et al. used a Bayesian approach for COVID-19 identification using CT scan in 2020 [48], along with three Bayesian algorithms, CNN and VGG16 in their work, and the results for regularised ensemble learning were 82.6 accuracy and 0.84 precision. Seum et al. proposed [49] in 2020, using several variations of ResNet and DenseNet, as well as VGG16 and VGG19, to detect COVID-19 in CT scan images. In DenseNet201, they had the best overall performance, with an accuracy of 89.92 percent and precision of 92.74 percent. For COVID-19 detection CT scan pictures, Kaya et al. [50] employed a variety of algorithms, including CNN, VGG16, EfficientNetB3, ResNet50, and Mo-bileNetV2, however EfficientNetB3 produced the best results, with an accuracy of 0.97 and sensitivity of 0.97. Padma and Kumari proposed employing CNN for COVID-19 detection using X-ray imaging in 2020 [51]. They achieved a precision of 100 percent and accuracy of 98.3 percent. In 2020, Qaqos and Kareem proposed [52] for COVID-19 detection using X-ray images. They used CNN for their work, and they divided the research into four classes, with the fourth class distinguishing between normal, COVID-19, Pneumonia, and Tuberculosis patients. They obtained an accuracy of 94.53 and precision of 92.67, 95.83, 95.97, and 95.65 for normal, COVID-19, Pneumonia and Tuberculosis patients respectively. Yener and Oktay proposed [53] in 2020 that they employed VGG16, VGG19, and Xception for COVID-19 detection using CT scan pictures, and that VGG16 had the best results with a learning rate of 10-4, accuracy of 0.91, and precision of 0.94. In 2021 Bougourzi et al. used algorithms such as ResneXt-50, Densenet-151, Inception-V3, WideResneXt for patient level and slice level classification for distinguishing normal, COVID-19, and CAP (Community Acquired Pneumonia) patients using CT scan images in 2021 [54], with an overall accuracy of 87.75 percent and sensitivity of 95.83 percent, 96.36 percent, and 52.63 percent. Chaudhary et al. employed EfficientNet to detect COVID-19 and CAP in 2021 [55], with stage 1 accuracy of 97.7% for COVID-19 and 94.7 for CAP, and stage 2 accuracy of 89.3%. In 2021, Wu et al. proposed [56] a Joint Classification and Segmentation (JCS) based diagnosis system to deliver diagnosis findings for COVID-19 detection. They employed ResNet and achieved a sensitivity of 95% and a specificity of 93%. In 2021, Jiang et al. proposed a Siamese network-based model for COVID-19 identification utilising Xception with CT scan images [57], with an accuracy of 0.8040 ± 0356 and an F1 score of 0.7998 ± 0384 in their research. Xue [58] used 3D based learning for COVID-19 diagnosis in 2021, with an accuracy of 86.84 percent and a sensitivity of 87.79 percent using 3DResNet50 and CNN. In 2021 [59], Sanagavarapu et al. employed the CLAHE algorithm for COVID-19 diagnosis, as well as CNN and ResNet, and obtained an accuracy of 0.87 and precision of 0.91. In 2021, Dutta et al. used Deep Neural Network and inception in their CNN model to diagnose COVID-19 using CT scan images [60], with an accuracy of 84 percent. Amran et al. proposed [61] in 2021 utilising CNN and variations of U-Net to distinguish be-tween COVID-19 and CAP, with an accuracy of 87.9 and sensitivity of 90.7. Berrimi et al. proposed [62] in 2021 for creating an automated for assisting doc-tors in COVID-19 diagnosis using X-ray and CT scan images. They used Inception V3 and DenseNet in their work, and they achieved an accuracy of 85% with Inception V3 in CT scan images and 95% with enhanced DenseNet in X-ray images. In 2021, Heidarian et al. introduced the “CT-CAPS” [63] capsule network-based architecture. They employed the R231CovidWeb U-Net model and yielded an accuracy of 89.8% and sensitivity of 95.5 percent. Rodrguez et al. proposed [64] using 3D CNN with CT scan images to diagnose COVID-19. They used LTC (Long Term Convolutional Nets), I3D (Inflated 3D ConvNet), and 2D U-Net for lung segmentation and got a precision of 98 percent for non-Covid and 96 per-cent for Covid in LTC and 96 percent for non-Covid and 100 percent for Covid in I3D. Sharma employed the CNN model with X-ray images in 2021 [65], but they also used VGG16 and VGG19 since they both produced superior accuracy of 0.97 compared to 0.94 for CNN, but their CNN model required less computational resources than VGG16 and VGG19. Kumar et al. proposed [66] in 2021, in which they tested VGG16 and ResNet models for COVID-19 identification using CT scan and X-ray pictures, as well as CLAHE, U-Net, and CNN for image cropping and histogram equalisation. They discovered that VGG16 provided superior accuracy while ResNet was more dependable, resulting in an accuracy of 0.974 and an F1 score of 0.979. In 2021, Nawshad et al. proposed [67], in which they used different approaches such as VGG16, ResNet, and Xception with little variation with X-ray images, which was not only useful for COVID-19 detection but also proved useful for viral pneumonia, but they got the best accuracy in ResNet model 96.79 percent, which had attention module in each block. Garg et al. proposed [68] that they employed CT scan to train three level classifiers (pretrained, patient level, and slice level) using ResNet50 to differentiate between COVID-19, CAP, and normal patients, with an overall accuracy of 88.57. In 2021 Subrato Bharati et al. [146]. used CO-ResNet for COVID-19 detection using X-ray images and they got accuracy of 90.90% and precision of 90.20%.

3.Methods for image processing technique

This section contains details on the various machine learning and image processing techniques utilized in the research articles presented here. Image processing is a technique for extracting or obtaining usable information from an image, such as characteristic features, by performing operations on it. An algorithm is used to convert the low-resolution image to a higher-resolution image. It was originally implemented in 1960 with the sole objective of improving image quality. In 1972, British engineer EMI Hounsfield invented x-ray computed tomography for head diagnosis utilizing image processing. Feature extraction, categorization, and pattern recognition are the most basic image processing applications. The purpose of image processing is to locate areas of greyscale. There are two types of image classification: supervised classification and unsupervised classification. In supervised categorization, particular training data is used with the use of “training sites” to determine what information is known. The huge unknown data with natural groupings is employed in unsupervised classification.

Luqy Nailur Rohmah and Alhadi Bustaman [75] proposed a study in 2020 about classification using image processing techniques to identify COVID-19. They utilized various combinations of GLCM, HOG, and LBP, as well as PCA and SVM, and reached accuracy of 99.4 using CT scan pictures and 97 using x-ray images. Zebari et al. proposed a paper in 2020 [76] for detecting COVID-19 by extracting different features from images and then feeding into classifiers. They used FD, GLCM, and LBP for feature extraction and KNN, ANN, and SVM for classifiers, and the best results were obtained using ANN and KNN, with an accuracy of 96.91 and precision of 95.77. Mucahid Barstugan, Umut Ozkaya and Saban Ozturk published a paper [77] in 2020 about classification of COVID-19 using feature extraction and machine learning. They used GLCM, LDP, GLRLM, GLSZM, and DWT [86] for image processing and SVM for machine learning. They divided the dataset into 5 parts in terms of subset while obtaining the results. The findings of the 76 non-infected and 106 infected patients were included in the fifth subset. Using the DWT feature extraction approach, they extracted 1024 features and performed 10-fold cross validation to get a best result of 97.28 ± 2.9 accuracy and 100 precisions. In 2021, Yasar and Ceylan proposed a paper [78] for COVID-19 detection using texture analysis, machine learning, and deep learning. KNN and SVM were used as machine learning techniques, and it was discovered that SVM was more successful. It was also discovered that using GLCM and LE for-feature extraction would improve SVM results, whereas using feature LBP and LE for-feature extraction would improve KNN results. Using CNN and deep learning, it was discovered that MobileNet had the best result, although AlexNet was the fastest in terms of classification time. Using all of these factors, they obtained a mean accuracy of 0.9473 in 2-fold cross validation and 0.9599 in 10-fold cross validation.

4.Methods for machine learning technique

This Machine learning is a data science approach that improves over time without requiring the usage of computer code. It develops itself by incorporating various user inputs. In general, machine learning in AI is used to replicate human thinking by using data provided by people. Big firms like Facebook and Google build and enhance their apps by asking consumers a variety of questions and incorporating their responses. It takes the inputs and processes them in order to predict new values. supervised machine learning, unsupervised machine learning, semi-supervised machine learning, and reinforcement machine learning are the four types of machine learning. To acquire its findings, supervised machine learning employs labelled datasets; it’s utilized for classification, and some of the methods employed include linear regression, naive bayes, and support vector machines, among others. The employment of algorithms to deal with unlabeled data is known as unsupervised learning. It can be beneficial in locating hidden elements in data. It’s utilized for both picture and voice recognition. Neural networks, KNN, and other approaches are employed in the algorithm. Semi-supervised learning is a combination of supervised and unsupervised learning. The train data used is small labelled data whereas the testing data is used in large unlabeled clustered data, it is sued to solve the problem of the having less training data. Reinforcement learning is like supervised learning but instead of using supervised learning it uses constant trial and error and with each successful outcome it reinforces the data and thus the name reinforcement learning. Machine learning helps in business and provide market. Applications of machine learning include image recognition, speech recognition, whether predictions, etc. Ensemble learning [79] is also a machine learning technique which combines different machine learning techniques in order to make a better model. Ensemble learning increases the performance of the model by achieving greater accuracy also makes the model robust.

Annisa Utama Berliana and Alhadi Bustamam suggested [95] a method for ensemble learning for categorization of COVID-19 in 2020. They employed SVC, KNN, and RF to obtain a 99 percent accuracy. X-rays have a precision of 98 percent and a recall of 100 percent, whereas CT scans have a precision of 97 percent and a recall of 97 percent. Tabrizchi et al. proposed a paper [96] in 2020 for using various machine learning, ensemble learning, and neural networks, such as SVM, Nave Bayes, CNN, MLP, Adaboost, and GDBT, to find an intelligent and accurate solution for COVID-19 detection, and they discovered that SVM and CNN outperformed all of them, with accuracy of 0.9920 and 0.9670, and precision of 0.9819 and 0.9724. Sharma submitted research [97] in 2020 that looked at CT images of 200 infected individuals and utilized machine learning to detect COVID-19. They used ResNet and Gradcam [36] and got a 91 percent accuracy rate. In 2020, Luca Brunese, Fabio Martinelli, Francesco Mercaldo, and Antonella Santona published a study [98] that employed machine learning approaches to detect COVID-19 patients. They used KNN and obtained a precision of 0.968 for COVID-19 patients and 0.955 for other patients. In 2020, Rezaee et al. proposed [99] a hybrid deep transfer approach for COVID-19 identification, which used both machine learning and deep learning. They employed SVM, CNN-CaffeNet, AlexNet, VeggF, KNN, and MLP to achieve a precision of 0.995 and accuracy of 0. 994. An ensemble learning approach for diagnosing COVID-19 using CT scan images was suggested by Bingyang Li, Qi Zhang, Yinan Song, Zhicheng Zhao, Zhu Meng, and Fei Su in a publication [100]. They achieved accuracy of 86.73 percent and sensitivity of 87.27 percent for COVID-19 patients, 68.42 percent for CAP patients, and 100 percent for normal patients using flip-and-shift augmentation, RandAugment, AdaBoost, EfficientNet, K-means, and ImageNet. A review study on machine learning approaches was offered by Siddiqui et al. in 2021 [101]. SVM, KNN, Logistic regression, and LSTM were the machine learning algorithms employed. They discovered that LSTM surpasses them all, with an accuracy of 99.68 percent, whereas SVM, KNN, and logistic regression only managed 96.20 percent, 95.30 percent, and 97.20 percent, respectively. A review study [102] by Ameer Sardar Kwekha Rashid, Heamn N. Abduljabbar, and Bilal Alhayani on covid- analysis of 19 They looked at 14 publications, five of which were about logistic regression, three were about ANN, and two were about CNN. They also looked at one study each about linear regression, K means, and KNN.

5.Prediction using pretrained model

In this study, a Pretrained model was used to identify COVID-19. Pretrained models are those who have previously been trained by someone else. As a result, rather of creating a model from scratch, one might use an existing model to solve their problem. Transfer learning is commonly used by pretrained models since it relies on prior information to tackle present problems. Feature and parameter transfer are both part of the pretraining process. In feature transfer, pre-train effective feature representations are used to improve the model’s performance. Parameter transfer methods work on the assumption that source and target tasks can exchange model parameters or previous distributions of hyper-parameters. The capacity to work with less data, the removal of the requirement to construct a model because one already exists, and the ability to attain high performance in a short amount of time are all advantages of using a pretrained model. We employed the VGG16 model, which had been pre-trained. Zisserman and Simonyan “VERY DEEP CONVOLUTIONAL NETWORKS FOR LARGE-SCALE IMAGE RECOGNITION” [22] was suggested in 2015. They used the ConvNet (Convolutional Neural Network) Model in that study to improve the model and raise its accuracy. They used their model in the ILSVRC competition. In 2014, they won with a 25.3 percent loss. They implemented VGG Net-D (16 layers) and VGG Net-E (19 layers), and their picture classification results were 89.7 mean AP in VOC-2007, 89.3 mean AP in VOC-2012, 92.7 0.5 mean class recall in Caltech-101, and 86.2 0.3 mean class recall in Caltech-256 [104]. For picture levee classification, we use Xgboost [103], which can also be done using deep learning, however deep learning takes a large amount of data, often millions to thousands of photographs. However, if that much data isn’t accessible, deep learning models produce poor outcomes and underperform. Boosting techniques such as Xgboost, Adaboost, random forest, and others are better to employ at that time. Because employing boosting algorithms instead of a deep learning model will yield far better outcomes.

5.4Performance metrics

For getting the performance of the model and confusion matrix we imported ‘sklearn.metrics’, it shows the results in true positives (TP), true negatives (TN), false negatives (FN) and false positives (FP). Let TN stand for COVID-19 negative photos that have been correctly categorized, TP for COVID-19 positive images that have been correctly classified, FN for COVID-19 positive images that have been wrongly classified, and FP for COVID-19 negative images that have been incorrectly classed.

Table 1

Result table of our pretrained model

Model	Algorithm	Accuracy	Precision	Recall	Specificity	F1Score
Proposed	VGG16	98%	98%	97%	97%	98%
[44]	Updated ResNet	78.8%	81.9%	97.5%	83.4%	79.3%
[46]	3D U-Net 3D ResNet	93.3%	87.6%	95.5%	88.4%	87.8%

Confusion matrix shows the value in TP, TN, FP and FN (see Fig. 1).

Figure 1.

A confusion matrix of true positive, true negative, false positive and false negative.

Accuracy-Accuracy is the how much correct predictions obtained divided by the total prediction.

Accuracy = TP + FP/TP + FP + FN + TN

Recall (Sensitivity) – is the correct predictions divided by the total prediction of that class i.e., COVID-19 which will include the correct COVID-19 prediction and Incorrect COVID-19 prediction.

Recall = TP/TP + FN

Precision – is the correct prediction divided by the total positive prediction, which include all the prediction that the model thinks is COVID-19.

Precision = TP/TP + FP

F1 score – F1 score takes account of both false positive prediction and false negative predictions. Thus, it takes the values of both precision and recall.

F1 Score = 2 × Precision × Recall/Precision + Recall

Specificity – is the incorrect predictions divided by the total prediction of that class. Thus, it shows the correct detection on Non COVID-19.

Specificity = TN/TN + FP

5.5Result

We used Vgg16 model for image classification along with Xgboost boosting algorithm. The results depend on the total number of images used. The higher the number of images the more accurate the results are. For comparison, we compared our model to that of [44], who utilized ResNet, and [46], who used 3D U-Net and 3D ResNet. Results are provided in Table 1. We got overall good results with accuracy, precision and f1 score of 98%. We got 97% of recall and specificity. Model presented the COVID-19 infected patient image (see Fig. 2).

Figure 2.

A COVID-19 infected image displayed by the model.

6.Prediction using deep learning model

We also used a deep learning model based on the U-Net architecture to detect COVID-19. Olaf Ronneberger, Philipp Fischer, and Thomas Brox proposed the U-Net architecture in 2015 [29], which is based on data augmentation. Data augmentation refers to methods for enhancing the amount of data available by adding slightly modified copies of existing data or synthesizing new data from existing data.

Figure 3.

U-Net model that we employed.

Figure 4.

Images when loaded to preview of original image, lung mask, infection mask and lung and infection mask.

6.1Model

U-net is an image segmentation architecture that, like many other deep learning models, consists of a combination of convolutional and maxpooling layers that are structured in such a way that they provide a result. Even though they employ the same layers that are included in all deep learning models, the arrangement and number of layers for each model will be different, resulting in variability in each design. The input layer is processed first, which will have a certain height, width, and channel size, such as 224 × 224 × 3. If the image channel is 3, the picture is colored, or more particularly, an RGB image (red, blue, and green), but if it is 1, the image is greyscale. Because of the convolutional filters, which essentially execute matrix multiplication, the image’s dimensions vary as it progresses down the model. The value depends on how much padding is done, among other things. The method of padding involves adding an extra layer of zeros to the image’s border. A tiny matrix determines the value from the picture that will be multiplied farther down the model in the maxpooling layers. As a result of this procedure, the image’s size is reduced, and padding is used to keep the image’s size consistent. U-net is a semantic segmentation tool that is mostly utilized in the field of medicine. The U-net architecture is called after the form of the letter ‘U’ in the alphabet. The contraction/encoder path is the one that goes down sampling path, while the expansion/decoder path is the one that goes up sampling path. The picture is initially sent to the convolution layers, then to the maxpooling layers, and finally to the up-sampling path. The model will have perfect symmetry; therefore, the down sampling path’s convolutional layer will be concatenated with the up-sampling path’s convolutional layer, which is exactly opposite each other. They’ll be the same size, thus adding the layer won’t be a problem. There is a dropout layer between the convolutional layers that randomly chooses certain pixels from the convolutional layer and removes them from the model. To avoid overfitting, dropout layers are employed (see Fig. 3).

Table 2

Result table of our deep learning model

Model	Algorithm	Accuracy	Precision	Recall	Loss	F1Score	AUC
Proposed	U-Net	99.86%	90.90%	88.88%	0.039%	89.82%
	99.87%
[44]	LeNet	86.06%	85%	89%	0.369%	87%	86%
[46]	CNN	99.2%	100%	99.91%	0.3%	–	–

Figure 5.

Performance metrics of the model compared to their validation.

Table 3

Review of image processing, machine learning and deep learning papers

References	Dataset references	Image processing techniques	Machine learning and deep learning techniques	Result
[75]	[106], [109]	GLCM, HOG LBP	SVM, PCA	CT Scan Accuracy = 99.4 Recall = 98.7 Precision = 100 Xray Accuracy = 97 Recall = 97 Precision = 97
[95]	[106], [109]		KNN, RF, SVC, ANN	CT Scan AUC = 100 Accuracy = 99 Precision = 98 Recall = 100 Xray AUC = 99.2 Accuracy = 97 Precision = 97 Recall = 97
[76]	[110]	GLCM, FB, LBP	SVM.ANN.KNN	Accuracy = 96.91 Sensitivity = 95.77 Specificity = 98.03 Sensitivity = 96.98
[96]	[111], [112]		SVM, NB, LBP, GDBT, AdaBoost, CNN	SVM CNN Accuracy 0.99 0.97 Precision 0.98 0.97 Recall 1.00 0.97 F1 0.99 0.97 MCC 0.98 0.95
[97]	[113], [114], [115], [116]		ResNet (CNN), Gradcam	Accuracy = 91% Specificity = 90.29% Sensitivity = 92.1%
[77]	[113]	GLCM, LDP, GLRLM, GLSZM, DWT	SVM	DWT (10-Fold) Accuracy = 97.28 ± 2.9 Sensitivity = 93.39 ± 7 Specificity = 100 Precision = 100 F1 = 96.46 ± 3.7
[98]	[117]		KNN	FP Rate = 0.068 Precision = 0.965 Recall = 0.965 F-Measure = 0.964 ROC Area = 0.989
[99]	–		SVM, CNN (CaffeNet, AlexNet, and Vegg-F), KNN, MLP	Accuracy-0.995 Precision-0.994 Recall-0.993 F1 score-0.991 Specicivity-0.995
[100]	[111], [112]		d flip-and-shift augmentation, Rand Augment, AdaBoost, EfficientNet, K-means, ImageNet	Accuracy = 99.73% Sensitivity (COVID-19 = 87.27% CAP = 68.42% Normal = 100.0%)
[101]	–		SVM, KNN, LSTM, Logistic Regression	Accuracy = 99.68% TP (COVID-19 = 100% CAP = 100% Normal = 98.90%)

Table 3, continued
References	Dataset references	Image processing techniques	Machine learning and deep learning techniques	Result
[102]	–		Logistic Regression, ANN, CNN, Linear Regression, Naive Bayes, K Means, KNN	Out of 14 articles (86% classification, 7% regression and 7% clustering). Methods used (5 Logistic regession, 3 ANN, 2 CNN, 1 Linear Regression, 1 Naive Bayes, 1 K Means and 1 KNN).
[78]	[117], [118], [119]	GLCM, LBP, LE	SVM, KNN, CNN	SVM CNN Accuracy 0, 91 0, 92 Sensitivity 0, 91 0, 92 Specificity 1, 00 0, 98 F1 0, 92 0, 92 AUC 0, 98 0, 99
[119]	[120], [121], [122]		DenseNet	Accuracy-84.7 Precision-97.0 Recall-76.2 F1 score-85.3 Area under ROC curve (AUC)-82.4
[31]	[106], [123], [124]		DenseNet, MobileNet, Xception, Inception, ResNet, VGG	Accuracy = 97.87 Sensitivity = 99.74 Specificity = 100
[32]	[119]		LeNet-CNN	Accuracy-86.06% Loss-0.369 F1-87%, Precision-85% Recall-89% Area under ROC curve (AUC)-0.86
[33]	[119], [112]		EfficientNet b4	(Plateau LR) Accuracy-0.90 Precision-0.90 Recall-0.89 F1 score-0.90 Area under ROC curve (AUC)-0.90
[34]	[125]		CNN	Accuracy-97.57 Precision-98.0 Recall-98.0 F1 score-98.0
[37]	[118], [126], [127],[128], [129]		COVID-SDNet	Negative Specificity-85.20 Negative Precision-79.94 Positive Sensitivity-76.80 Positive Precision-84.23 Accuarcy-81.00
[39]	[130], [119]		Inflated inception (Stream 13D) Resnet-50	CC-19 Accuracy = 0.85 Sensitivity-60 Covid-CT Accuracy = 0.84 Sensitivity-35
[40]	[123]		Contrast Limited Adaptive Histogram Equalization (CLAHE), CNN	Accuracy = 83.28% Precision = 81.57% Recall = 88.07%
[42]	[131]		CNN architecture, ResCovNet153V2 ImageNet	Accuracy-0.881 Sensitivity-0.821 F1 score-0.758 Specicivity-0.899
[44]	[118]		CNN ResNet+ (Upgraded ResNet)	Accuracy-0.788 Precision-0.819 Sensitivity-0.975 F1 score-0.834 Specivity-0.793

Table 3, continued
References	Dataset references	Image processing techniques	Machine learning and deep learning techniques	Result
[46]	–		3D-Unet 3D-ResNets CNN	Accuracy-93.3 ±0.8 Precision-87.6 ± 4.3 Recall-95.5 ± 2.1 F1 score-88.4 ± 4.1 Specificity-87.8 ± 1.5 AUC-97.3 ± 1.1
[47]	[119], [132]		CNN Resnet	Accuracy-0, 943 Precision-0.971 Sensitivity-0.941 F1 score-0.942 Specificity-0.973 AUC-0.985
[48]	[119]		CNN Three Bayesian Algorithm ImageNet VGG16(OxfordNet)	Accuracy-83.9 Precision-0.83 Recall-0.86 F1 score-0.84 Specificity-0.77 Sensitivity-0.886
[49]	[106]		AlexNet, VGG16, VGG19, ResNet18, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, DenseNet201, Inception_v3 and GoogleNet	(DenseNet201) Accuracy-89.92% Precision-92.74% Sensitivity-86.80% F1 score-89.67% Specivity-93.09%
[50]	[133]		CNN VGG-16 EfficientNetB3 ResNet-50 MobileNetV2	(EfficientNetB3) Accuracy-0.9792 Sensitivity-0.9722 F1 score-0.98 Specivity-0.9861
[51]	[118]		CNN	Accuracy-98.3% Precision-100% Sensitivity-99.1% Loss-0.3 Specivity-98.8%
[52]	[109], [134]		CNN	(COVID-19 4 classes) Accuracy-94.53 Precision-95.83 Sensitivity-80.23% F1 score-87.33% Specivity-99.65%
[53]	[119], [135]		VGG16, VGG19 and Xception	(VGG16 LR4) Accuracy-0.91 Precision-0.90 Recall-0.94 F1 score-0.92 AUC-0.91
[54]	[111]		CNN, ResneXt-50, Densenet-161, Inception-V3 and Wide-Resnet	Accuracy-87.95% Sensitivity-96.36%
[55]	[111], [119]		CNN EfficientNet InceptionV3 ResNet DenseNet	Accuracy-89.3
[56]	[118], [119], [133], [136]		CNN-RESNET	Sensitivity-95% Specificity-93.0% Dice-78.5%
[57]	[133], [137]		Xception	Accuracy = 0.8040 ± 0.0356 F1 score = 0.7998 ± 0.0384

Table 3, continued
References	Dataset references	Image processing techniques	Machine learning and deep learning techniques	Result
[58]	[111]		3D ResNet50 CNN	Accuracy = 86.94% Sensitivity = 87.79% Specificity = 89.88%
[59]	[123]		Contrast Limited Adaptive Histogram Equalization (CLAHE), Resnet CNN	Accuracy-0.87 Precision-0.91 Recall-0.81 F1 score-0.84
[60]	[105]		Convolutional neural network (CNN) Inception v3	Accuracy-84 Precision-0.8636 Recall-0.8143 F1 score-0.8382
[61]	[138]		CNN U-Net	sensitivity = 90.7% specificity = 83.3% accuracy = 87.9%
[62]	[139]		DenseNet Inception V3 ImageNet	Accuracy Ct Scan X ray 85% 95.98%
[63]	[111]		U-net (R231CovidWeb)	Accuracy-89.8% sensitivity-94.5% specificity-83.7% Area Under the ROC Curve (AUC)-0.93 probability threshold-0.5
[64]	[140]		U-Net 3D CNN (Long Term convolutional nets (LTC), Inflated 3D ConvNet (I3D))	Prec Recall f1 LTC 96% 98% 97% ConvNet 100% 96% 98%
[65]	[141]		CNN VGG16 VGG19	Accuracy-0.94 Precision-1.0 Recall-1.0 F1 score-1.0 AUC-0.985
[66]	[142], [143], [144], [35]		CNN ImageNet U-Net, CLAHE VGG16, ResNet	Accuracy-0974 F1 Score-0.979
[67]	[30]		CNN, ImageNet Res-Net32 CBAM VGG16	(ResNet32 with attention block module) Accuracy-97.43%
[68]	[26]		ResNet50	Accuracy (%) Normal CAP COVID 88.89 88.57 90.00 Sensitivity 88.57
[145]	[146]		Co-ResNet	Accuracy-90.90% Precision-90.20%

7.Comparative study table

Information of the results about different machine learning and image processing papers is provided in Table 3. The purpose of this paper to provide information about the research done on covid-19 diagnosis and provide some help for future researchers to work from the information gathered. The literature review of image processing and machine learning was done in our previous work [108].

8.Conclusion

SARS-CoV-2 virus that causes COVID-19, a respiratory disorder. It spreads from person to person in close proximity, and people are most vulnerable to contracting COVID-19 in crowded and poorly ventilated areas. As a result, it’s critical to have a quick diagnosis of COVID-19. AI approaches are also being used by researchers to detect COVID-19. This paper summarizes the results of various different researchers that have worked on COVID-19 detection. Because dataset is so crucial, different datasets have been employed for different projects, resulting in a variety of results. As a result, each dataset will provide various outcomes. This paper compiles information from various research papers that have used x-ray and CT scan lung images to study COVID-19. As well as the models for COVID-19 prediction. Image processing, machine learning approaches, and deep learning are the three aspects of the literature review. DenseNet and its variants were utilized in a number of deep learning articles, including [119, 31, 49, 54, 56, 62]. ResNet and its variants were also widely utilized, with articles [97, 31, 39, 44, 46, 47, 49, 50, 54, 55, 58, 59, 66, 67, 68] mentioning them. Researchers that have employed image processing and feature extraction for COVID-19 detection can be found in the image processing section. GLCM has been used in all feature extraction publications since it offers the pixel value of texture images. LBP has also been employed to extract features in [75, 119, 77]. The paper’s machine learning section examines works by researchers who have utilized machine learning techniques to detect covid 19. SVM appeared to be the most widely used technique, as it was utilized in [75, 119, 77, 78, 96, 99, 101]. Many articles [119, 78, 95, 98, 99, 101] used KNN because it produced good results. In addition, we applied our own algorithm to predict COVID-19 in Lung CT scan pictures. For feature extraction in images, we employed a VGG16 pretrained model along with the Xgboost boosting approach to distinguish between COVID-19 and non-Covid. We got an accuracy of 98%, precision of 98%, recall of 97%, specificity of 97% and f1 score of 98% for COVID-19 images. We also implemented our Deep learning model by using U-Net and got an accuracy of 99.86%, precision of 90.90%, recall of 88.88%, Loss of 0.39%, F1 score of 89.82% and AUC (Area under Curve) of 99.87%. The goal of “Soft Computing and Image Processing Techniques for COVID-19 Prediction for COVID-19 in Lung CT Scan Images” paper is to help future researchers to work on COVID-19 diagnosis and also to predict COVID-19 with different AI models. It would make their work easier as the necessary information to start working on COVID-19 is provided.