Machine learning struggles with imbalanced data. Although several mitigation approaches exist, their application depends on the extent of imbalance. To determine the latter, a protocol was developed. Across 428 synthetic and 70 real datasets, 8 imbalance measures were benchmarked and evaluated using multiple classifiers, metrics, and correlation coefficients. The coding environment, data preparation, and correlation and complexity analyses are described. These are complemented by procedures for an ablation study of the most efficient measure: SIMBA (status of imbalance). For complete details on the use and execution of this protocol, please refer to Pivin-Bachler et al. 1 • Procedure for critical review and evaluation of imbalance measures • Instructions for correlation analyses on synthetic and real datasets • Steps for running data complexity and ablation analyses • Guidance on how to use additional classifiers, evaluation metrics, and datasets Publisher’s note: Undertaking any experimental protocol requires adherence to local institutional guidelines for laboratory safety and ethics. Machine learning struggles with imbalanced data. Although several mitigation approaches exist, their application depends on the extent of imbalance. To determine the latter, a protocol was developed. Across 428 synthetic and 70 real datasets, 8 imbalance measures were benchmarked and evaluated using multiple classifiers, metrics, and correlation coefficients. The coding environment, data preparation, and correlation and complexity analyses are described. These are complemented by procedures for an ablation study of the most efficient measure: SIMBA (status of imbalance).
Pivin-Bachler et al. (Mon,) studied this question.