On shape of pattern error function, initializations and intrinsic dimensionality in ANN classifier design

Article type: Research Article

Authors: Raudys, Šarūnas

Affiliations: Institute of Mathematics and Informatics, 2600 Vilnius, Akademijos 8t.4, Lithuania

Abstract: An analytical equation for a generalization error of minimum empirical error classifier is derived for a case when true classes are spherically Gaussian. It is compared with the generalization error of a mean squared error classifier – a standard Fisher linear discriminant function. In a case of spherically distributed classes the generalization error depends on a distance between the classes and a number of training samples. It depends on an intrinsic dimensionality of a data only via initialization of a weight vector. If initialization is successful the dimensionality does not effect the generalization error. It is concluded advantageous conditions to use artificial neural nets are to classify patterns in a changing environment, when intrinsic dimensionality of the data is low or when the number of training sample vectors is really large.

Keywords: feed forward neural nets, training sample size, generalization, intrinsic dimensionality, initialization, insufficient learning

DOI: 10.3233/INF-1993-43-410

Journal: Informatica, vol. 4, no. 3-4, pp. 360-383, 1993