A Bayesian evolutionary approach to the design and learning of heterogeneous neural trees

Article type: Research Article

Authors: Zhang, Byoung-Tak

Affiliations: Biointelligence Laboratory, School of Computer Science and Engineering, Seoul National University, Seoul 151-742, South Korea. Tel.: +82 2 880 1833; Fax: +82 2 883 3595; E-mail: [email protected]

Abstract: Evolutionary algorithms have been successfully applied to the design and training of neural networks, such as in optimization of network architecture, learning connection weights, and selecting training data. While most of existing evolutionary methods are focused on one of these aspects, we present in this paper an integrated approach that employs evolutionary mechanisms for the optimization of these components simultaneously. This approach is especially effective when evolving irregular, not-strictly-layered networks of heterogeneous neurons with variable receptive fields. The core of our method is the neural tree representation scheme combined with the Bayesian evolutionary learning framework. The generality and flexibility of neural trees make it easy to express and modify complex neural architectures by means of standard crossover and mutation operators. The Bayesian evolutionary framework provides a theoretical foundation for finding compact neural networks using a small data set by principled exploitation of background knowledge available in the problem domain. Performance of the presented method is demonstrated on a suite of benchmark problems and compared with those of related methods.

DOI: 10.3233/ICA-2002-9105

Journal: Integrated Computer-Aided Engineering, vol. 9, no. 1, pp. 73-86, 2002