Comparison of the discriminative ability of artificial neural networks and GRACE score for prediction of mortality in acute coronary syndromes

Abstract Introduction Artificial neural networks are predictive algorithms with higher discriminative ability than models derived from conventional logistic regression. Aim To compare the discriminative ability of an algorithm developed by an artificial neural network and the GRACE score for the prediction of mortality in acute coronary syndromes (ACS). Material/Methods :A database with consecutively included patients of acute coronary syndromes from a Community Hospital was analyzed. 57 variables were analyzed (50 cofactors and 7 covariates): demographic data, cardiovascular risk factors, cardiovascular and non-cardiovascular history, pre-admission medication, pre-admission procedures, vital signs at admission, admission Killip class , electrocardiographic patterns, peak levels of serum troponin T, serum creatinine, blood glucose, white blood cell count at admission, left ventricular function determined by echocardiography, invasive procedures and clinical events during hospitalization. Neural network models were trained with different methods (multilayer perceptron or radial basis), neural architectures and activation functions to select the best performing algorithm. The sample was subdivided into a 70% neural network training subpopulation and another 30% for validation. The areas under the ROC curve of the neural network and the GRACE score were compared using De Long's test. Calibration or goodness-of-fit analysis was performed by Hosmer Lemeshow test. Sensitivity analysis was performed to identify the variables with the highest independent normalized importance for the model. The SPSS 23.0 Statistics program was used for statistical analysis and neural network modeling. The end point of the analysis was hospital mortality. Results A total of 1818 ACS patients were analyzed. The best performing neural network model was the multilayer perceptron, with an architecture composed of a hidden layer with 4 neurons, tangent hyperbolic activation function of the in the hidden layer neurons and softmax of those in the output layer. The area under the ROC curve of the resulting algorithm for death prediction was 0.98 (95% CI 0.97-0.99). The GRACE score had an area under the ROC curve of 0.88 (95% CI 0.77-0.90). The comparison between both areas by De Long test showed significant differences: p0.001. Calibration or goodness of fit showed better correlation between observed and calculated events (lower dispersion or residual values) with the neural networks chi-square 1.9 p= 0.78 with respect to the GRACE chi-square score of 4.9, p 0.06. The variables with the highest independent normalized importance in the artificial neural network model were serum creatinine (100%) and troponin T elevation (82%), Conclusions According to the present analysis, both the discriminatory capacity of hospital mortality as well as the calibration or goodness of fit of the artificial neural networks are significantly superior to the GRACE score.