Repeated undersampling in PrInDT (RePrInDT): Variation in undersampling and prediction, and ranking of predictors in ensembles

Abstract

In this paper, we extend our PrInDT method (Weihs & Buschfeld 2021a) towards undersampling with different percentages of the smaller and the larger classes (psmall and plarge), stratification of predictors, varying the prediction threshold, and measuring variable importance in ensembles. An application of these methods to a linguistic example suggests the following: 1. In undersampling, a careful selection of the percentages plarge and psmall is important for building models with high balanced accuracies; 2. Stratification of predictors does not majorly enhance balanced accuracies; 3. Lowering the prediction threshold for the smaller class turns out to be an alternative method to undersampling because it increases the likelihood of the smaller class being selected. Finally, we introduce a method for ranking predictor importance that allows for a straightforward interpretation of the results.