Association-based similarity testing and its applications

Article type: Research Article

Authors: Li, Tao^{; *} | Ogihara, Mitsunori | Zhu, Shenghuo

Affiliations: Computer Science Department, University of Rochester, Rochester, NY 14627-0226, USA

Correspondence: [*] Corresponding author: Tel.: +1 585 275 8479; Fax: +1 585 273 4556; E-mail: [email protected]

Abstract: This paper proposes a new similarity measure between basket datasets based on associations. The new measure is calculated from support counts using a formula inspired by information entropy. Experiments on both real and synthetic datasets show the effectiveness of the measure. This paper then investigates the applications of the similarity measure. It first studies the problem of finding a mapping between categorical database attribute sets using similarity measures. A generic approach for identifying such a mapping is proposed. The approach is implemented based on the similarity measure proposed in the paper and its performance has been evaluated and validated. Moreover, this paper also explores the applications of using the similarity measure to mine distributed datasets.

Keywords: similarity measure, association, maximal frequent itemset, heterogeneous, distributed data mining

DOI: 10.3233/IDA-2003-7304

Journal: Intelligent Data Analysis, vol. 7, no. 3, pp. 209-232, 2003