Beyond Balanced Accuracy: Balanced Matthews’ Correlation Coefficient
Sébastien Guesné, Stéphane Werner and Thierry Hanser
Lhasa Limited, Granary Wharf House, 2 Canal Wharf, Leeds, LS11 5PS, United Kingdom
Balanced accuracy is a metric to assess the global performance of binary quantitative structure-activity relationship (QSAR) classification models. In contrast to accuracy, balanced accuracy is independent on the positive prevalence of the test set that is used to validate a QSAR classifier. As such balanced accuracy is used to overcome the effect of imbalance test sets on accuracy.
Matthews’ correlation coefficient (MCC), an alternative global performance metric, is also known to mitigate the imbalanced of the test set. However, in contrast to the balanced accuracy, MCC remains dependant on the positive prevalence. Its value may be underestimated at extremely high or low positive prevalence. It may thus lead to incorrect interpretation of validation results when a change of and/or a different positive prevalence occurs in the test set(s) during the validation experiment.
The concept of balanced metrics beyond balanced accuracy is, as far as we are aware, not described in the chemoinformatic literature. Therefore, this presentation will first demonstrate that balanced accuracy is the performance metric accuracy which is calibrated to a test set with a positive prevalence of 50% (i.e., balanced test set). Secondly, this concept of balanced accuracy will be extended to MCC after showing its dependency on the positive prevalence. And finally, a tale of two MCCs will exemplify the use of this concept of balanced MCC versus MCC with a couple of use cases.
