Elena Gelzinyte Abstract

Understanding and Interpreting How Neural Networks Identify Toxic Compounds

Elena Gelzinyte¹, Timothy EH Allen¹, Andrew J Wedlake¹, Jonathan M Goodman¹, Steve Gutsell², Paul J Russell²

¹Centre for Molecular Informatics, Department of Chemistry, University of Cambridge, Lensfield Road, Cambridge, United Kingdom, CB2 1EW
²Unilever Safety and Environmental Assurance Centre, Colworth Science Park, Sharnbrook, Bedfordshire MK44 1LQ, United Kingdom
Safety evaluation of molecules is crucial for the development of new chemicals in many industries. This is an expensive and time-consuming process. Adverse Outcome Pathways (AOPs)[1] are a new approach to risk assessment that link Molecular Initiating Events (MIEs)[2], the initial ligand-receptor interactions, to the corresponding toxic response at organ or organism level. Large amounts of experimental data have been collected about MIEs, and they provide good targets for computational modelling approaches. Artificial neural networks have been shown to provide highly reliable toxicity predictions, outperforming other algorithms[3]. However, for these methodologies to be widely accepted by toxicology experts, the models must be as transparent as possible. In silico MIE prediction tools have been developed employing neural network algorithms to be used in risk assessment. In addition, Layer-wise Relevance Propagation[4] and Network Activation Similarity methods have been used to make the machine learning model more transparent.

The MIEs for investigation were selected from a collection of important human pharmacological targets with adverse effects, highlighted by Bowes[5]. Open source data from ChEMBL (www.ebi.ac.uk/chembl) and ToxCast (www.epa.gov/chemical-research/toxicity-forecasting) databases were used, providing balanced numbers of active and inactive chemicals for constructing an artificial neural network classifier. Once trained, the neural network takes the compound’s chemical structure as an input and predicts binary activity towards the receptor in question. Evaluation by cross-validation shows good performance with most models achieving an accuracy above 80%, with some exceeding 90%. While the high accuracy is a good indication of model performance, for their general acceptance in a field such as toxicology, there must be more transparency on how the predictions are reached.

Layer-wise Relevance Propagation highlights the input molecule’s substructures, critical to its identification as toxic. By considering the output score and network’s internal parameters, the input features are ranked according to their importance to the classifier’s prediction. The input features correspond to a binary ECPF6 fingerprint and, using RDKit (www.rdkit.org), are traced back to the corresponding chemical substructures in the molecule. The molecular substructures are outputted and ranked according to their importance to the neural network’s prediction. The explanations have been evaluated for the compounds in our dataset and compared with other approaches, such as structural alerts. By relating the model prediction to the chemical structure of the molecule, this method provides the user with a more intuitive understanding of the chemistry behind this predictive technique.

Network Activation Similarity employs neural network’s parameters, determined upon training. Once trained, these weights and biases modify values from the input fingerprint to provide an activity prediction. As a result, the fingerprint of each chemical produces a distinct activation profile within the network. When a new molecule is analysed, the network will provide a prediction and calculate its profile. This is then compared to ones generated by the training set compounds to highlight which entries are ‘perceived’ similarly by the neural network. Thus, activation similarities support a read-across style of risk assessment by rationalising predictions and providing insight into the cases where the model fails.

These in silico approaches not only predict MIEs, but also provide explanations on how the predictions have been reached. By supplementing conventional toxicity evaluations with transparent and high performing computational approaches we can improve the efficiency of AOP based risk assessment and better understand these important molecular interactions.