Intelligent document classification

Article type: Research Article

Authors: Calvo, Rafael A. | Ceccatto, H.A.

Affiliations: Instituto de Física Rosario (CONICET-UNR), 27 de Febrero 210bis, 2000 Rosario, Argentina. E-mail: [email protected]

Abstract: In this work we investigate some technical questions related to the application of neural networks in document classification. First, we discuss the effects of different averaging protocols for the χ2 statistic used to remove non-informative terms. This is an especially relevant issue for the neural network technique, which requires an aggressive dimensionality reduction to be feasible. Second, we estimate the importance of performance fluctuations due to inherent randomness in the training process of a neural network, a point not properly addressed in previous works. Finally, we compare the neural network results with those obtained using the best methods for this application. For this we optimize the network architecture by evaluating much larger nets than previously considered in similar studies in the literature.

Keywords: text classification, statistical learning, neural networks, knowledge management

DOI: 10.3233/IDA-2000-4503

Journal: Intelligent Data Analysis, vol. 4, no. 5, pp. 411-420, 2000