iBoost: Boosting using an instance-based exponential weighting scheme

Article type: Research Article

Authors: Karmaker, Amitava | Yoon, Kihoon | Nguyen, Chau | Kwek, Stephen

Affiliations: University of Texas at San Antonio, San Antonio, TX 78249, USA. E-mail: [email protected]

Abstract: AdaBoost is a well-recognized ensemble method to improve prediction accuracy over the base learning algorithm. However, it is prone to overfitting the training instances [18]. Freund, Mansour and Schapire [5] established that using exponential weighting scheme in combining classifiers reduces the problem of overfitting. Also, Helmbold, Kwek and Pitt [7] showed in the prediction using a pool of experts framework an instance-based weighting scheme improves performance. Motivated by these results, we propose here an instance-based exponential weighting scheme in which the weights of the base classifiers are adjusted according to the test instance x. Here, a competency classifier ci is constructed for each base classifier hi to predict whether the base classifier’s guess of x’s label can be trusted and adjust the weight of hi accordingly. We show that this instance-based exponential weighting scheme enhances the performance of AdaBoost.

Keywords: Ensemble methods, multiple classifier system, boosting

DOI: 10.3233/HIS-2007-4404

Journal: International Journal of Hybrid Intelligent Systems, vol. 4, no. 4, pp. 243-254, 2007