Affiliations: [a] Knowledge Discovery and Data Mining Lab, University of Ottawa, Ottawa, Canada | [b] Computer Engineering Department, Iran University of Science and Technology, Tehran, Iran
Correspondence:
[*]
Corresponding author: Mahdi Mohammadi, Knowledge Discovery and Data Mining Lab, 6143, 55 Laurier Ave, E., University of Ottawa, Ottawa, K1N 6N5, Canada. Tel.: +1 613 562 5800; Ext: 8833; E-mail:[email protected]
Abstract: Analysis of network traffic, financial
transactions, and mobile communications are examples of applications where
examining entire samples of a large dataset is computationally expensive,
and requires significant memory space. A common approach to address this
challenge is to reduce the number of samples without compromising the
accuracy of analyzing them. In this paper, we propose a new cluster-based
sample reduction method which is unsupervised, geometric, and density-based.
The original data is initially divided into clusters, and each cluster is
divided into ``portions'' defined as the areas between two concentric
circles. Then, using the proposed geometric-based formulas, the selection
value of each sample belonging to a specific portion is calculated. Samples
are then selected from the original data according to the corresponding
calculated selection value. The performance of the proposed method is
measured on various datasets and compared with several cluster-based and
density-based methods. We conduct various experiments on the NSL-KDD,
KDDCup99, and IUSTsip datasets, and evaluate the performance of the proposed
method by measuring the cluster validity indices, as well as the accuracy of
the classifier applied on the reduced data. We demonstrate that the reduced
dataset has similar sample scattering as that of the original dataset. We
also demonstrate that, while reducing the sample size of the input dataset
in half, the classification accuracy is not reduced significantly,
indicating that the proposed method selects the most relevant samples from
the original dataset.
Keywords: Sample reduction, clustering, classification, density-based, membership function
