Enhancing medical image analysis: A fusion of fully connected neural network classifier with CNN-VIT for improved retinal disease detection

Article type: Research Article

Authors: Mannanuddin, Khaja^a | Vimal, V.R.^b | Srinivas, Angalkuditi^c | Uma Mageswari, S.D.^d | Mahendran, G.^e | Ramya, J.^f | Kumar, Ashok^g | Das, Pranjal^h | Vidhya, R.G.^{i; *}

Affiliations: [a] School of CS and AI, SR University, Telangana, India | [b] Department of Computer Science and Engineering, Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Thandalam, Chennai, Tamil Nadu, India | [c] Department of Computer Science Applications, Koneru Lakshmaiah Education Foundation, Vedeswaram, Andhra Pradesh, India | [d] Department of Science and Humanities, RMK Engineering College, Kavaraipettai, Tiruvallur, Tamil Nadu, India | [e] Department of Mechanical Engineering, RMK Engineering College, Kavaraipettai, Tiruvallur, Tamil Nadu, India | [f] Department of Computer Science and Engineering, St. Joseph’s College of Engineering, Chennai, Tamil Nadu, India | [g] Department of Computer Science, Banasthali Vidyapith, Banasthali, Tonk, Rajasthan, India | [h] Department of Electronics & Telecommunication Engineering, Jorhat Institute of Science and Technology, Jorhat, Assam, India | [i] Department of Electronics & Communication Engineering, HKBK College of Engineering, Bangalore, India

Correspondence: [*] Corresponding author. R.G. Vidhya, Department of Electronics & Communication Engineering, HKBK College of Engineering, Bangalore, India. E-mail: [email protected].

Abstract: Diseases of the retina continue to be a leading cause of blindness and visual impairment around the world. In the field of medical image analysis, specifically retinal disease identification, deep learning techniques, such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), have showed remarkable potential. In this paper, we present a unique method for detecting retinal diseases by combining the advantages of the Inception-V3, ResNet-50, and Vision Transformer architectures into a single model called a Cascade CNN-ViT. The suggested Cascade CNN-ViT model extracts local features from retinal pictures by leveraging the spatial hierarchy learning capabilities of Inception-V3 and ResNet-50. The Vision Transformer takes these regional characteristics and uses self-attention mechanisms to pick up global context information and long-range interdependence. The model successfully combines fine-grained local information with semantically significant global contextual cues by merging the output representations from the CNNs and Vision Transformer. undertaking comprehensive experiments on a large and varied dataset of multimodal retinal pictures to evaluate the performance of the proposed technique. Cascade CNN-ViT model outperforms standalone CNNs and Vision Transformers, as shown by the experimental findings. The model is also resilient across all classes of retinal diseases and is able to successfully deal with the complications introduced by using multiple picture types. Overall, the power of cascading Inception-V3, ResNet-50, and Vision Transformer topologies for improved retinal illness diagnosis has been demonstrated. Potentially improving the management of retinal illnesses and preserving visual health, the proposed approach could have important consequences for early detection and timely intervention.

Keywords: Multimodal retinal images, deep learning, Inception-V3, vision transformer, cascade CNN-ViT

DOI: 10.3233/JIFS-235055

Journal: Journal of Intelligent & Fuzzy Systems, vol. 45, no. 6, pp. 12313-12328, 2023