Machine learning programs have been largely successful in predicting the structure or sequence of proteins, but we still do not know what biophysical characteristics are learned by the model to inform these predictions. With the aim of simulating the accuracy of computational physics based models, message passing neural networks (MPNNs) use spatial data to construct feature vector representations of proteins which can then be trained to predict functionality. These models represent proteins as a set of nodes and edges where information about each is first embedded as vectors and is then shared across them via messages. ProteinMPNN uses the 3D coordinates of backbone atoms to build a latent dimension which is then decoded to determine sequences of amino acids likely to fold into the original backbone conformation. This research aims to use sparse autoencoders (SAEs) to expand the latent dimension and correlate feature dimensions to true protein features with the aim of understanding what makes ProteinMPNN effective at predicting sequences. We find that several features within a sparse latent space are well correlated with protein features and that SAEs can be used to increase interpretability within MPNNs and explain their behavior.
Mukkavilli et al. (Sun,) studied this question.