Deep pyramidal residual networks with inception sub-structure in image classification

Article type: Research Article

Authors: Xu, Fei^{a; *} | Wang, Peng^a | Xu, Huimin^b

Affiliations: [a] Department of Applied Mathematics, Northeast Agricultural University, Harbin, P R China | [b] Public Teaching Department of Mathematics, Northeast Agricultural University, Harbin, P R China

Correspondence: [*] Corresponding author. Fei Xu. E-mail: [email protected].

Abstract:  Deep convolutional neural networks (DCNNs) have shown remarkable performance in image classification tasks in recent years. In the network structure of DPRN, as the network depth increases, the number of convolutional kernels also increases linearly or nonlinearly. On the one hand, in the DPRN block, the size of the receptive field is only 3 × 3, which results in insufficient network ability to extract feature map information of different filter sizes. On the other hand, the number of convolution kernels in the second 1x1 convolution will be multiplied by a coefficient relative to the first convolution, which can cause overfitting to some extent. In order to overcome these weaknesses, we introduce the inception-like structure on the basis of the DPRN network which is called by pyramid inceptional residual networks (PIRN). In addition, we also discuss the performance of PIRN network with squeeze and excitation (SE) mechanism and regularization term. Furthermore, some results in network performance are discussed when adding a stochastic depth networkto the PIRN model. Compared to DPRN, PIRN achieved better results on the CIFAR10, CIFAR100, and Mini-ImageNet datasets. In the case of using zero-padding, the multiplicative PIRN with SE mechanism achieves the best result of 95.01% on the CIFAR10 dataset. Meanwhile, on the CIFAR100 and Mini-ImageNet datasets, the additive PIRN network with a network depth of 92 achieves the best results of 76.06% and 65.86%, respectively. According to the experimental results, our method has achieved better accuray than that of DPRN with same network settings which demonstrate its effectiveness in generalization ability.

Keywords: Convolution neural network, Deep pyramidal residual network, Squeeze and excitation mechanism, Pyramidal inceptional residual network, L2 regularization

DOI: 10.3233/JIFS-230569

Journal: Journal of Intelligent & Fuzzy Systems, vol. 45, no. 4, pp. 5885-5906, 2023