Affiliations: Department of Mechanical Production, College of
Technological Studies, PAAET, Kuwait | Department of Mechanical Engineering, Brunel
University, Uxbridge, Middlesex, UB8 3PH, UK
Abstract: A new neural network approach for matrix computations is presented.
The idea is to construct a feed-forward neural network (FNN) and then train it
by matching a desired set of patterns. The solution of the problem is the
converged weight of the FNN. Accordingly, unlike the conventional FNN research
that concentrates on external properties (mappings) of the networks, this study
concentrates on the internal properties (weights) of the network. The present
network is linear and its weights are usually strongly constrained; hence, a
complicated overlapped network needs to be constructed. It should be noticed,
however, that the present approach depends highly on the training algorithm of
the FNN. Unfortunately, the available training methods such as the, the
original Back-propagation (BP) algorithm, encounter many deficiencies when
applied to matrix algebra problems, including slow convergence due to improper
choice of learning rates (LR). Thus, this study focused on the development of
new efficient and accurate FNN training methods. One improvement suggested to
alleviate the problem of LR choice is the use of a line search with steepest
descent method, namely, bracketing with golden section method. This provides an
optimal LR as training progresses. Another improvement proposed in this study
is the use of conjugate gradient (CG) methods to speed up the training process
of the neural network. The computational feasibility of these methods is
assessed on two matrix problems; namely, the LU-decomposition of both band and
square ill-conditioned unsymmetric matrices and the inversion of square
ill-conditioned unsymmetric matrices. In this paper, only the first one is
reported.
