Affiliations: [a]
College of Artificial Intelligence, Tianjin University of Science & Technology, Tianjin, China
| [b]
College of Sciences, Tianjin University of Science & Technology, Tianjin, China
Correspondence:
[*]
Corresponding author. Yufeng Li, Department of Data Science and Big Data, College of Artificial Intelligence, Tianjin University of Science & Technology, Tianjin, China. E-mail: [email protected].
Abstract: Many traditional clustering algorithms are incapable of processing mixed-type datasets in parallel, limiting their applications in big data. In this paper, we propose a CF tree clustering algorithm based on MapReduce to handle mixed-type datasets. Mapper phase and reducer phase are the two primary phases of MR-CF. In the mapper phase, the original CF tree algorithm is modified to collect intermediate CF entries, and in the reducer phase, k-prototypes is extended to cluster CF entries. To avoid the high costs associated with I/O overheads and data serialization, MR-CF loads a dataset from HDFS only once. We first analyze the time complexity, space complexity, and I/O complexity of MR-CF. We also compare it with sklearn BIRCH, Apache Mahout k-means, k-prototypes, and mrk-prototypes on several real-world datasets and synthetic datasets. Experiments on two mixed-type big datasets reveal that MR-CF reduces execution time by 45.4% and 61.3% when compared to k-prototypes, and it reduces execution time by 73.8% and 55.0% when compared to mrk-prototypes.
