Novel neural network architecture using sharpened cosine similarity for robust classification of Covid-19, pneumonia and tuberculosis diseases from X-rays

2.1Motivation of the research

According to earlier findings, most of the covid-19 infected patients are identified as having pneumonia and tuberculosis disease. As a result, radiographic testing plays a more significant role in early childhood infection diagnosis of covid-19 infection. It is challenging to categorize a patient based solely on CXR scans. The deep learning architectures currently used for classification have high model complexity and numerous training parameters. A novel neural network with nominal training parameters and greater accuracy is suggested to solve this issue using the SCS layer. The neural network model with SCS layer is considered for covid-19 classification to deploy on computing- or power-limited devices to make rapid decisions.

2.2Related work

The influence of artificial intelligence on the clinical care of covid-19 is briefly discussed in this section, along with studies that used CXR/CT scans to diagnose pneumonia, tuberculosis, and covid-19. In the recent decade, AI has shown considerable promise in medical image processing [15]. DL has been used widely in medical domains [16], including disease prediction [17], identification of pulmonary nodules [18], and classification of benign/malignant cancers [19]. A technique to create artificial CXR images called auxiliary classifier GAN based on covidGAN was proposed by Waheed et al. (2020) [20]. The CNN model is trained using synthetic images. The approach classified Covid-CXR and Normal-CXR with a 95% accuracy rate. A covid-DeepNet model was suggested by Panthakkan et al. to swiftly detect covid-19 and non-covid pneumonia infections from lung X-rays of critically sick patients (2021) [21]. The model can quickly test for covid-19 and has a classification accuracy of 99.67%. The major drawback are large number of layers and nodes which leads to large number of parameters. Waisy et al. (2021) suggested a COVID-DeepNet model with contrast constrained adaptive histogram equalization and Butterworth filter to improve contrast and remove noises in order to properly predict covid-19 infection, and obtains the accuracy of 99.93% [22]. The biggest downside of the Covid-DeepNet system is that it was trained to divide the input CX-R picture into two different groups (e.g., healthy and Covid-19 infected).

To distinguish between covid and non-covid, Turkoglu et al. (2021) suggested a multiple kernel extreme learning deep neural network model and achieved an accuracy of 98.36% [23]. This method’s drawback is that it can only be used for binary categorization. To improve model performance, Murugan et al. (2021) combine the backpropagation approach and the Whale Optimization Algorithm (WOA) [24]. The method experimented with different pre-trained models like Alex net, GoogLeNet, squeeze net, resnet50, and resnet50 with WOA. The resnet50 with WOA achieved an accuracy of 98.78%, but the number of trainable parameters is vast. Based on the U-net concept, Tianbo et al. (2021) created an architecture that included down sampling, skip links, and ultimately connected layers. [25]. Using a publicly available dataset that contained binary and multiclass classification, the model detects COVID-19 with an accuracy of 99.53% and 95.35% in binary and multiclass tasks, respectively. Deep transfer learning combined with super pixel-based segmentation was offered by Prakash et al. (2021) for detecting and localization of COVID-19 [26]. The method’s overall accuracy for binary classification was 99.53%, while for multiclass, it was 99.79%.

Agarwal S et al. in 2022 have proposed model for four classes each: normal, covid-19, bacterial pneumonia, and viral pneumonia [27]. Set 1 of the datasets comprises normal, covid-19, and pneumonia. They looked at various pre-trained CNN models, including mobilenetv2, inceptionv3, Xception, VGG16, InceptionResNetV2, NASNet mobile, resnet50, and DenseNet121, which are improved by swapping out the architecture’s last layers. Based on the results, densenet121 had an accuracy of 97% for the set 1 dataset, while mobilenetv2 had an accuracy of 81% on the set2 dataset. The disadvantage of this work is that the accuracy level for four class classifications is poor due to the model’s inability to discriminate features.

A covidDetNet architecture with nine convolutional layers and one fully connected layer was proposed by Naeem et al. (2022). Two activations are utilized in this architecture. Leaky ReLU and ReLU, along with batch and channel normalization [28]. The architecture has classified the covid, pneumonia, and normal, which obtained an accuracy of 98.40%. The model complexity is more regarding the number of parameters to be trained. Abhijit et al. (2022) suggested combining image segmentation and classification architecture using deep learning algorithms to identify lung abnormalities [29]. The approach was carried out in three phases. The segmentation technique was first carried out using CXR images created using conditional GAN architecture. Then segmented image features are extracted using VGG 19 model, which is merged with the BRISK key point extraction and classified with a random forest algorithm in the final layer. The testing accuracy of image classification is 96.6% when utilizing the architecture of the Visual Geometry Group-19 model in conjunction with the binary robust invariant scalable key points approach. Syed et al. (2022) proposed a novel image classification strategy for determining whether or not a patient is infected with covid19 [30]. To boost accuracy, the last layer of SoftMax is substituted with a KNN classifier in this technique. The suggested evolutionary approach is utilized to automatically discover the ideal values for CNN’s hyperparameters, resulting in a considerable improvement in classification accuracy. Larger and more diversified databases are still needed for validation before claiming that deep learning may help clinicians recognize COVID-19 in X-ray images since cross-dataset analysis shows that even cutting-edge models lack generalization power.

S.K. Ghosh et al. (2022) introduced the enhanced residual network (ENResNet) for the visual categorization of COVID-19 from chest X-ray images. The ENResNet is used to build the residual images and is normalized using patches [31]. The ENResNet achieves a classification accuracy of 99.7% in binary and 98.4% in multiclass detection. Tripti Goelr et al. (2022) proposed a two-step DL design for COVID-19 diagnosis leveraging CXR images [32]. The two steps of the recommended DL architecture are feature extraction and classification. The “Multi-Objective Grasshopper Optimization Algorithm (MOGOA)” is used to optimize the DL network layers; as a result, these networks are known as “Multi-COVID-Net.” A covid fuzzy ensemble network (COFE-Net) was presented by Banerjee et al. (2022) in ensemble deep learning architectures [33]. The classification accuracy for the COFE-Net model is 99.49% for the binary class and 96.39% for the multiclass. The proposed network was compared to an existing pre-trained network to establish the network’s efficiency. SARS Net was introduced by Kumar et al. (2022) to classify COVID-19 using the images CXR [34]. In this paper, SARS-Net combines a 2L-GCN model and a CNN model. The model’s accuracy and sensitivity on the validation set were 97.60% and 92.90%, respectively. Gurkan Kavuran et al. (2022) introduced the MTU-COVNet hybrid architecture to abstract visual features from volumetric thoracic CT images to identify COVID-19 [35]. Then features are optimized from the concentrated layers and obtain overall accuracy of 97.7%.

2.3Limitations of the existing studies

Several researchers have tried to use contemporary DL frameworks to automate the analysis of pneumonia and COVID-19. However, the following drawbacks and difficulties affect several of the notable algorithms:

3.1Description of the datasets

The CXR datasets are collected via Kaggle and GitHub and include diseases such as Covid-19, Normal, Pneumonia, and Tuberculosis in binary and multiclass form [36–39]. The images have different dimensions and are resized to 224 by 224 pixels. In cases of viral pneumonia, the white opacity surrounding the lungs on an X-ray image reduces and is almost non-existent in Covid-19 images. Model X-ray images from the database are shown in Fig. 1. The distribution of categories across the obtained dataset is depicted in Fig. 2.

Fig. 1

Samples from dataset a) Covid-19 (b) Normal (c) Pneumonia (d) Tuberculosis.

Fig. 2

Dataset distribution of multiclass experiment.

In total, there are 7135 images in the datasets shown in Table 1, of which 576 are COVID-19 images, 1583 are normal images, 4273 are pneumonia images, and 703 are tuberculosis images. The dataset was split into 710 images for validation and 6425 for training. The validation data includes 61 COVID-19 images, 174 normal images, 411 images of pneumonia, and 64 images of tuberculosis—a total of 1564 normal and 573 COVID-19 images used for binary classification. For data validation, 161 normal images and 54 COVID-19 images were employed. Different image augmentation techniques, including rotation, random crop, width shift range, and height shift range, to prevent the model from overfitting are used to enhance model performance.

3.2Problem with CNN

Convolutional layers, which include a collection of filters whose parameters are learned throughout the training, are the essential building component of a convolutional neural network. While each filter convolves with the image, the feature map is produced. A feature map is created when a filter is applied to the input, revealing the locations and intensities of a detected feature in the input. The kernel and an image are confined to two values, a high and a low. Positive values imply “prefer the signal value to be as high as possible,” which is ideal for understanding a convolutional kernel. Negative values indicate “prefer as low a frequency value as possible,” and zeroes suggest that “it doesn’t matter what the value.” The convolution layer output size is computed using equation 1. Convolution results from a sliding dot product between the kernel and an image portion. But dot product alone is a poor metric and results in a poor feature detector [40]. The fundamental shortcoming of the convolution layer is that it does not encode information when the objects’ location and orientation change. Furthermore, the intrinsic data of the image is entirely lost, and the information is sent to the same neurons, rendering them incapable of dealing with this type of information.

Where, W = Size of the input, F = Size of the filter, P = Padding, S = Stride

Equation 2 represents a convolution operation, a strided dot product between an input and a kernel. CNN has a very high level of accuracy when it comes to image recognition tasks and automatically selects critical information without human intervention. The convolution layer’s primary issue is spatially invariant to input data. Furthermore, a lot of training data is required because the convolution layer does not encode the location and orientation of the object. Again, the Max pooling layer operation makes a CNN much slower. CNNs have millions of parameters, and a tiny dataset would lead them to overfit since they require vast data to make them more generalized. To solve this issue, we replace the convolution layer with a sharpened cosine similarity layer.

Where, X = Input, Y = Kernel

3.3Proposed SCS-Net

The flowchart of the proposed work is shown in Fig. 3, and the pipeline starts X-ray machine used to capture the X-ray of the patients and stored in the database. The produced images from the X-ray are used to train the model for classification of binary class and multiclass classification labeling (covid-19, pneumonia, tuberculosis, and normal. The images are preprocessed to fit the model for training. Train data and validation data are split from the dataset. The SCS-Net architecture is developed with different layers consisting of a cosine similarity layer, max-pooling layer, and flattened and dense layer. Figure 4 depicts the SCS-Net architecture with layers combined. The training set is employed to create the SCS-Net model. After the model has been trained, validation data are used to evaluate it, and performance metrics, including accuracy, precision, recall, F1 measure, and Cohen’s kappa score, are generated. Moreover, the SCS-Net model’s accuracy is evaluated in various samples of state-of-the-art architecture. Instead of applying a convolution layer, we employed the SCS layer to build the feature map. SCS is a feature-building method for neural networks that is an alternative approach to convolution. Similar to convolution, SCS is a stride operation that takes features out of an image. But before calculating the dot product, the image patch and kernel are adjusted to have a magnitude of 1, producing a cosine similarity, also known as cosine normalization [41]. The cosine of the angle formed by the two-dimensional signal and kernel vectors is given in equation 3.

Fig. 3

Flowchart for SCS-Net model.

Fig. 4

Overall architecture of SCS-Net.

Where, X = Input, Y = Kernel, Q = Scale invariance, P = Sharpening exponents.

Convolution-like stride operations, such as SCS, extract features from images. The Sharpened Cosine Distance (SCD) is described in equation (4), and the Sharpened Cosine Similarity (SCS) is in equation (5). The sharpening exponents must be learned for each unit. This peak can be sharpened by raising it to a power of an exponent [42]. It performs better than a convolution neural network with 10x-100x fewer parameters and doesn’t require normalization or activation functions after SCS layers. Since the cosine has a wide range, two very different vectors might have a high cosine similarity. It can be sharpened by keeping the sign while increasing the size of the result to a power P. If the signal or kernel magnitude ever approaches 0, this measure may become numerically unstable, increasing the input strength by a modest amount, Q. The kernel magnitude doesn’t grow too narrow.

The benefits of sharpened cosine similarity are as follows:

One of the essential constraints of the SCS is training the model on GPUs and TPUs by raising operations to the power of P often don’t parallelize efficiently. On GPUs, reducing the parameters greatly accelerates performance, but SCS loses its “sharpened” quality.

3.4Model parameter tuning

The SCS-Net model is trained using 40 epochs, 64 batch sizes, and a 0.0001 learning rate. The momentum and RMS prop optimizer combined model are trained using the Adam optimizer. The Keras tuner library is used to fine-tune the hyperparameters to make the proposed SCS-Net robust. The key reason for utilizing the Adam optimizer is that it uses very little memory and is computationally efficient. The learning rate affects the pace of learning of the DL model, which determines the number of movements required to minimize the loss function value. The momentum enhances both model training speed and accuracy. For binary classification, the sigmoid function is utilized at the output layer, while SoftMax activation is used for multiclass classification.

4.1Results and discussion

The experiments are conducted for binary classification and multiclass classification. We have used the 10% dataset for validation and the 90% dataset for training. DL methods use this split ratio frequently for model training and validation [43–45]. The model is trained using the same parameter with a 40-epoch, a 0.0001 learning rate, and a 64-batch size. For training and validation, the 224x224 CXR images are utilized. The model keeps track of the validation accuracy and modifies the initial learning for better feature extraction by using checkpoint callbacks. An Adam optimizer with a momentum of 0.9, a binary cross-entropy loss function for binary classes, and a categorical cross-entropy loss function for multiclass classification are utilized for training the SCS-Net model.

Figure 5(a) and 5(b) show the training and validation accuracy of binary and multiclass classification. Figure 6(a) and 6(b) show the loss of binary and multiclass classification training and validation. Accuracy, precision, recall, F1-score, and kappa score are the metrics used to evaluate the model performance. The formulae for the performance metrics are given in Table 3. Figure 7 shows the confusion matrix of the SCS-Net for multiclass and binary classification, which is used to determine performance metrics for SCS-Net. The SCS-Net model is developed to achieve a minimal false-positive class for binary classification; we achieved only four images that fall into the false positive category for covid-19 disease, and in multiclass, seven images fall into false positive classes for covid-19 disease. In Table 4, the proposed work is compared to different SOTA models reported in XiangYu [46], Stephan [47], Gaobo Liang [48], Gayathri J [49], Soni M [50], Mehedi M [51], Lokeswari V [52], Shikhar J [53], Emtiaz Hussai [54], Tanvir M [55] and Ahmed S Elkorany [56] by accuracy for both binary and multi-classification. The proposed SCS-Net model based on cosine similarity operation that can identify distinct features between covid-19 and other diseases is analyzed. To demonstrate the SCS-Net performance, it is compared with a few published experiments that mainly use CNN architecture. Our SCS-Net obtained an accuracy of 97.67 % (binary class) and 94.05 % (4 class), which is better than SOTA models for covid-19 classification. The following factors contribute to the SCS-Net model’s higher performance. SCS-net models use cosine similarity functions to learn the data projection. The SCS-Net model is more beneficial in model-trainable parameters and also learns non-linear transformations.

Fig. 5

Training and validation accuracy of SCS-Net model (a) Binary Class (b) Multiclass.

Fig. 6

Training and validation Loss of SCS-Net model (a) Binary Class (b) Multiclass.

Fig. 7

(a) Multi-class Classification (b) Binary Class Classification.

Moreover, compared to conventional CNN model, the SCS-Net model achieved good performance with substantially 10X-100X parameters. The SCS-Net architecture is also trained on an imbalanced dataset, but it proves to be a better model when counting the number of false positives among the classes. As seen in Table 5, model training parameters are significantly less for both binary and multiclass classification. It shows each label’s performance metrics on recall, precision, f1-score, overall accuracy, kappa score, and trainable parameter. The SCS layer proved worthy in extracting the distinct features between different classes and proved better in model training parameters.

The Covid-19, tuberculosis, pneumonia, and normal classes processed using the SCS layer’s filter are captured in the feature space. Visualize a feature map for a specific image to explain the observed features. In contrast to feature maps towards the model output, which capture more distinctive features, feature maps nearer the input are thought to identify fine-grained information. While the deeper layer networks learn more sophisticated features like pathological lesions, the first layer networks always learn basic features like edges, lines, patterns, and color. Later layers incorporate features from earlier layers to create new features. To analyze the feature maps of the proposed SCS Net model are shown in Fig. 8. Figure 8(a) shows the feature map of covid-19 class, Fig. 8(b) shows the feature maps of the pneumonia class, 8(c) shows the feature maps of the tuberculosis class, and finally, 8(d) shows the feature maps of normal class at the SCS layer. The SCS-Net feature map visualization demonstrates that the model can recognize features that can distinguish between the classes of covid-19, pneumonia, tuberculosis, and normal class.

Fig. 8

Feature map visualization of SCS layer.

Finally, our technique is cost-effective and time-efficient, and healthcare practitioners can quickly adopt it, enabling faster covid-19 screening processes and earlier isolation of patients. If a CXR image is available, real-time screening of covid-19 patients using SCS-Net techniques may be achievable with minimal human intervention. Furthermore, AI-based screens can be tailored to the level of understanding of the end user, and technicians may not be required to be trained in complex computational procedures. The following are the study’s drawbacks, which present the immediate potential for additional research.