Pair wise training for stacked convolutional autoencoders using small scale images

Issue title: Soft Computing and Intelligent Systems: Techniques and Applications

Guest editors: Sabu M. Thampi and El-Sayed M. El-Alfy

Article type: Research Article

Authors: Kumar, Aiswarya S. | Nair, Jyothisha J.^{; *}

Affiliations: Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India

Correspondence: [*] Corresponding author. Jyothisha J. Nair, Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, India. E-mail: [email protected].

Abstract: Deep neural networks have dramatically gained immense potential in almost every field like computer vision, natural language processing, biomedical informatics etc. Among these networks, autoencoders are popular in performing dimensionality reduction task, while learning a representation for an unlabeled dataset. A usual way of dealing with such networks is to pre-train them in a layer-wise fashion, and consequently fine-tune the whole stack in a supervised manner. In this paper, a pair-wise training strategy is proposed to determine optimum model parameters by reducing training time as well as the complexity of training a convolutional autoencoder without compromising its accuracy. The proposed approach works in a fully unsupervised manner and has been tested on datasets like MNIST, CIFAR10, and CIFAR100 and it shows that the training time has improved by an average of 25% on these three datasets.

Keywords: Deep learning, convolutional autoencoder, pair-wise training

DOI: 10.3233/JIFS-169910

Journal: Journal of Intelligent & Fuzzy Systems, vol. 36, no. 3, pp. 1987-1995, 2019