Local Feature Selection with Dynamic Integration of Classifiers

Issue title: Intelligent Systems

Article type: Research Article

Authors: Puuronen, Seppo | Tsymbal, Alexey

Affiliations: Department of Computer Science and Information Systems, University of Jyväskylä, P.O. Box 35, FIN-40351 Jyväskylä, Finland (e-mail: [email protected] ; alexey{@}cs.jyu.fi)

Abstract: Multidimensional data is often feature space heterogeneous so that individual features have unequal importance in different sub areas of the feature space. This motivates to search for a technique that provides a strategic splitting of the instance space being able to identify the best subset of features for each instance to be classified. Our technique applies the wrapper approach where a classification algorithm is used as an evaluation function to differentiate between different feature subsets. In order to make the feature selection local, we apply the recent technique for dynamic integration of classifiers. This allows to determine which classifier and which feature subset should be used for each new instance. Decision trees are used to help to restrict the number of feature combinations analyzed. For each new instance we consider only those feature combinations that include the features present in the path taken by the new instance in the decision tree built on the whole feature set. We evaluate our technique on data sets from the UCI machine learning repository. In our experiments, we use the C4.5 algorithm as the learning algorithm for base classifiers and for the decision trees that guide the local feature selection. The experiments show some advantages of the local feature selection with dynamic integration of classifiers in comparison with the selection of one feature subset for the whole space.

Keywords: Feature selection, ensemble of classifiers, dynamic integration, data mining, machine learning

Journal: Fundamenta Informaticae, vol. 47, no. 1-2, pp. 91-117, 2001