Affiliations: [a] Department of Computer Science, Michigan Technological University, Houghton, MI, USA | [b] Department of Computer Science, University of Crete, Iraklio, Crete, Greece | [c] Institute of Computer Science, Foundation for Research and Technology, Hellas, Greece | [d] Department of Mathematics, Vanderbilt University, Nashville, TN, USA | [e] Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA
Correspondence:
[*]
Corresponding author: Laura E. Brown, Department of Computer Science, Michigan Technological University, Houghton, MI 49931, USA. Tel.: +1 906 487 3472; Fax: +1 906 487 2283; E-mail: [email protected]
Note: [1] The research of this author was supported, in part, by the U.S. National Science Foundation under grants DMS-0808093
and DMS-0934630.
Abstract: Polynomial Support Vector Machine models of degree d are linear functions in a feature space of monomials of at most degree d. However, the actual representation is stored in the form of support vectors and Lagrange multipliers that is unsuitable for human understanding. An efficient, heuristic method for searching the feature space of a polynomial Support Vector Machine model for those features with the largest absolute weights is presented. The time complexity of this method is Θ(dms2+sdp), where m is the number of variables, d the degree of the kernel, s the number of support vectors, and p the number of features the algorithm is allowed to search. In contrast, the brute force approach of constructing all weights and then selecting the largest weights has complexity Θ(sdm+dd). The method is shown to be effective in identifying the top-weighted features on several simulated data sets, where the true weight vector is known. Additionally, the method is run on several high-dimensional, real world data sets where the features returned may be used to construct classifiers with classification performances similar to models built with all or subsets of variables returned by variable selection methods. This algorithm provides a new ability to understand, conceptualize, visualize, and communicate polynomial SVM models and has implications for feature construction, dimensionality reduction, and variable selection.
Keywords: Support Vector Machines, classification, variable selection
