Recursive hybrid decomposition with reduced pattern training

Article type: Research Article

Authors: Tan, Chin Hiong | Guan, Sheng-Uei | Ramanathan, Kiruthika^{; *} | Bao, Chunyu

Affiliations: Department of Electrical and Computer Engineering, National University of Singapore, 10 Kent Ridge Crescent, Singapore 119260

Correspondence: [*] Corresponding author. E-mail: [email protected]

Abstract: When neural networks are applied to large scale real-world classification problems, a major drawback is its inefficiency in utilizing network resources. A natural approach to overcome this drawback is to decompose the problem into several smaller sub-problems based on the “divide-and-conquer” methodology. This paper presents a hybrid method of task decomposition – OP-RPHP (Output Parallelism with Recursive Percentage-based Hybrid Pattern training). OP-RPHP employs a combination of both class decomposition and domain decomposition in its architecture thereby incorporating the advantages of both methods. OP-RPHP can be grown and trained in parallel on separate processing units to improve training time. To further improve the training time, a reduced pattern training algorithm is introduced. The reduction parameter p associated with the reduced pattern training algorithm is optimized to obtain maximum reduction in training time without compromising classification accuracy. Our approach is tested on four benchmark classification problems retrieved from the UCI repository of machine learning databases. The results show that OP-RPHP with reduced pattern training outperformed conventional OP and RPHP algorithms in both classification accuracy and training times.

Keywords: Task decomposition, domain decomposition, neural networks, parallelism, reduced pattern training, hybrid algorithm

DOI: 10.3233/HIS-2009-0085

Journal: International Journal of Hybrid Intelligent Systems, vol. 6, no. 3, pp. 135-146, 2009