Quantitative analysis of the generalization ability of deep feedforward neural networks

Article type: Research Article

Authors: Yang, Yanli^{; *} | Li, Chenxia

Affiliations: Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, Tiangong University, Tianjin, China

Correspondence: [*] Corresponding author. Yanli Yang, Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, Tiangong University, Tianjin, China. E-mail: [email protected].

Abstract: Generalization ability is known as an important performance index of artificial neural networks (ANNs). The generalization ability of an ANN usually refers to its ability to recognize untrained samples, but it lacks quantitative analysis. A method is designed by using frequency-domain signals to observe the generalization ability of deep feedforward neural networks (DFFNNs) which are popular ANN models. This method allows us to observe that the generalization ability of DFFNNs is limited to a small neighborhood around the trained samples. Then, the relationship between sample similarity and the DFFNN’s generalization performance is further analyzed. The analysis results show that the correlation coefficient between samples has a certain positive correlation with the DFFNN’s generalization performance. Based on this new understanding, an algorithm in which shadows of the trained samples are added into the training set is proposed to improve the generalization ability of DFFNNs. The proposed algorithm is tested with some simulated signals and some real-world data. The tests show that the proposed method can indeed improve the DFFNN’s generalization ability by only changing the training sample set.

Keywords: Deep feedforward neural network, deep learning, artificial neural network, generalization ability, correlation coefficient

DOI: 10.3233/JIFS-201679

Journal: Journal of Intelligent & Fuzzy Systems, vol. 40, no. 3, pp. 4867-4876, 2021