Affiliations: [a] Department of Pharmacy, Kaohsiung Municipal Chinese Medical Hospital, Taiwan | [b] Department of Computer Science and Information Engineering, Hwa Hsia University of Technology, Taiwan | [c] Department of Information and Computer Engineering, Chung Yuan Christian University, Taiwan | [d] Department of Information Management, National Central University, Taiwan | [e] Department of Information Management, National Chung Cheng University, Taiwan
Correspondence:
[*]
Corresponding author: Chih-Fong Tsai, Department of Information Management, National Central University, Taiwan. Tel.: +886 3 422 7151; Fax: +886 3 4254604; E-mail:[email protected]
Abstract:
BACKGROUND: To collect medical datasets, it is usually the case
that a number of data samples contain some missing values. Performing the
data mining task over the incomplete datasets is a difficult problem. In
general, missing value imputation can be approached, which aims at providing
estimations for missing values by reasoning from the observed data.
Consequently, the effectiveness of missing value imputation is heavily
dependent on the observed data (or complete data) in the incomplete
datasets. OBJECTIVE: In this paper, the research objective is to perform
instance selection to filter out some noisy data (or outliers) from a given
(complete) dataset to see its effect on the final imputation result.
Specifically, four different processes of combining instance selection and
missing value imputation are proposed and compared in terms of data
classification. METHODS: Experiments are conducted based on 11 medical related
datasets containing categorical, numerical, and mixed attribute types of
data. In addition, missing values for each dataset are introduced into all
attributes (the missing data rates are 10%, 20%, 30%, 40%, and
50%). For instance selection and missing value imputation, the DROP3 and
k-nearest neighbor imputation methods are employed. On the other hand, the
support vector machine (SVM) classifier is used to assess the final
classification accuracy of the four different processes. RESULTS: The experimental results show that the second process by
performing instance selection first and imputation second allows the SVM
classifiers to outperform the other processes. CONCLUSIONS: For incomplete medical datasets containing some
missing values, it is necessary to perform missing value imputation. In this
paper, we demonstrate that instance selection can be used to filter out some
noisy data or outliers before the imputation process. In other words, the
observed data for missing value imputation may contain some noisy
information, which can degrade the quality of the imputation result as well
as the classification performance.
Keywords: Instance selection, missing value imputation, incomplete data, medical data mining
