An evaluation of Naive Bayes variants in content-based learning for spam filtering

Article type: Research Article

Authors: Seewald, Alexander K.

Affiliations: Seewald Solutions, A-1180 Vienna, Austria. Tel.: +43(664) 110 68 86; Fax: +43(1) 2533033 2764; E-mail: [email protected]

Abstract: We describe an in-depth analysis of spam-filtering performance of a simple Naive Bayes learner and two extended variants. A set of seven mailboxes comprising about 65,000 mails from seven different users, as well as a representative snapshot of 25,000 mails which were received over 18 weeks by a single user, were used for evaluation. Our main motivation was to test whether two extended variants of Naive Bayes learning, SA-Train and CRM114, were superior to simple Naive Bayes learning, represented by SpamBayes. Surprisingly, we found that the performance of these systems was remarkably similar and that the extended systems have significant weaknesses which are not apparent for the simpler Naive Bayes learner. The simpler Naive Bayes learner, SpamBayes, also offers the most stable performance in that it deteriorates least over time. Overall, SpamBayes should be preferred over the more complex variants.

Keywords: Empirical study, spam filtering, machine learning, Naive Bayes

DOI: 10.3233/IDA-2007-11505

Journal: Intelligent Data Analysis, vol. 11, no. 5, pp. 497-524, 2007