Drug Representation and Scrambling Experiments Highlight Issues with Training and Evaluation of Drug-target Interaction Predictive Models
Antonio de la Vega de León, Mirko Torrisi and Alejandro Panjkovich
CITRE, Bristol Myers Squibb, C. Isaac Newton 4, 41092 Seville, Spain
Recently developed deep learning models that can predict drug-target interactions (DTI),
such as DeepConv-DTI or DeepAffinity, have shown good performance on DTI benchmark
datasets, thus generating great interest for drug-discovery applications. In this work, we re-
evaluated multiple models independently and investigated their performance when 1) different representations for drugs are used and, 2) drug or target information is scrambled.
Our initial results, when testing both classic fingerprints and novel deep learning based fingerprints, show that varying drug representation had a relatively small effect on model performance across the selected benchmark datasets. Furthermore, our experiments show that the models still achieve predictive performance when drug or target information is scrambled on published benchmark datasets. Overall, these results suggest that the performance of DTI models as evaluated previously may rely more on the structure of the benchmark dataset than on intrinsic DTI properties, highlighting the need for mitigation strategies while training the model.
