Abstract We have evaluated the prediction accuracy of three different tools, deep-learning-based AlphaFold2, AlphaFold3, and large language model-based ESMFold, utilizing the experimentally derived structures deposited in the Protein Data Bank between 2022 and 2024, excluding those entries with close homologs in the structures released prior to 2022. Based on the criteria of sequence identity lower than 40% and query coverage 70%, 1666 monomeric and 994 dimeric proteins were selected as challenging targets for benchmarking. Our analysis showed that AlphaFold2 and AlphaFold3 correctly predicted 88% of monomeric structures and 77% of dimeric proteins. On the other hand, ESMFold accurately predicted 76% of the monomeric proteins and 41% of the dimeric proteins. Since most incorrect predictions involved nuclear magnetic resonance structures, benchmarking on X-ray and cryo-electron microscopy structures showed that the prediction accuracy of AlphaFold and ESMFold was 95% and 83%, respectively, for monomeric proteins. Overall, these findings demonstrate significant differences in the prediction accuracy of these machine learning (ML)-based tools for monomeric and dimeric proteins, highlighting the advantages and limitations of these tools. Finally, to facilitate easy access to benchmarking data, we developed ProModEv (Protein Model Evaluation portal), an interactive web portal for systematic analysis of these benchmarking results, and it is available at http://pdbi.nii.ac.in/ProModEv/.
Mahtha et al. (Tue,) studied this question.