Affiliations: [a] Ninah Consulting Ltd., New York, NY, USA | [b] Google Inc., New York, NY, USA | [c] MarketTools, Inc., Minneapolis, MN, USA | GfK Custom Research North America, 8401 Golden Valley Rd, Minneapolis, MN, USA
Correspondence:
[*]
Corresponding author: Dimitri Liakhovitski, 1675 Broadway, 8th Floor, New York, NY 10019, USA. Tel.: +1 917 538 1000; Fax: +1 212 479 9441; E-mail: [email protected].
Abstract: Decision makers in organizations frequently need to determine relative importance of multiple predictors of an important Dependent Variable. Using Multiple Regression for this purpose is often challenging when predictors are intercorrelated. In this paper we present the results of two Monte-Carlo studies comparing the effectiveness of two methods for determining relative importance of predictors under conditions of multicollinearity: Johnson’s Relative Weights (JRW) and Breiman’s Random Forests (RF). The following factors were systematically varied: number of predictors, correlations among predictors in population, regression model R2 in population, number of observations per predictor, reliability of measurement, and standard deviation of regression coefficients across predictors in population. To serve as a benchmark measure of the relative importance of predictors, General Dominance Weights (GDW) method was used (also known as Shapley Value of the predictors in a regression), which defines predictor importance as an average increase in R2 associated with that predictor across all possible regression submodels. Sample-based predictor importances determined by JRW and RF were compared to GDW importances in population. The implications of the results for practitioners are discussed.
Keywords: Predictor importance, relative importance, multicollinearity, Random Forests, Shapley Value Regression, general dominance weights, relative weights, regression
