Backward Elimination Methods for Associative Memory Network Pruning

Article type: Research Article

Authors: Hong, Xia^a | Harris, Chris^b | Brown, Martin^c | Chen, Sheng^b

Affiliations: [a] Department of Cybernetics, University of Reading, Reading, UK | [b] Department of Electronics and Computer Science, University of Southampton, Southampton, UK | [c] Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, UK

Abstract: Three hybrid data based model construction/pruning formula are introduced by using backward elimination as automatic postprocessing approaches to improved model sparsity. Each of these approaches is based on a composite cost function between the model fit and one of three terms of A-/D-optimality / (parameter 1-norm in basis pursuit) that determines a pruning process. The A-/D-optimality based pruning formula contain some orthogonalisation between the pruned model and the deleted regressor. The basis pursuit cost function is derived as a simple formula without need for an orthogonalisation process. These different approaches to parsimonious data based modelling are applied to the same numerical examples in parallel to demonstrate their computational effectiveness.

Keywords: nonlinear modelling, backward elimination, forward regression, model sparsity, generalization

DOI: 10.3233/HIS-2004-11-211

Journal: International Journal of Hybrid Intelligent Systems, vol. 1, no. 1-2, pp. 90-98, 2004