Abstract Cerebellar mutism syndrome (pCMS) is a severe complication experienced by up to 40% of children undergoing resection of a primary posterior fossa tumour. Accurate prediction of pCMS risk could improve perioperative patient/parental counselling and surgical decision making. This multinational, multi-reader retrospective study aimed to evaluate the generalisation performance of two previously reported artificial intelligence (AI) models (Rotterdam and GOSH-Colorado-Stanford) for the preoperative prediction of pCMS. To this end, 120 children with primary posterior fossa tumours were enrolled across four participating centres (30 children per centre). Neuroimaging was reviewed by 11 readers blinded to clinical outcome and the risk of each patient developing pCMS predicted with and without AI decision support. Model accuracy and the agreement of predictions between investigators were evaluated. Across all 120 patients and 11 readers, the accuracy of clinical pCMS predictions without AI decision support was 72.7% which increased significantly in the second round of assessment with AI decision support from both the Rotterdam model (accuracy 76.7%; P0.001) and the GOSH-Colorado-Stanford model (accuracy 80.3%; P=0.008). Upon direct comparison, the GOSH-Colorado-Stanford model outperformed the Rotterdam model (P=0.012), demonstrating a higher sensitivity (88% vs. 80%) and positive predictive value (82% vs. 76%). AI decision support was also found to significantly standardise the prediction of pCMS across study readers (Fleiss’ κ = 0.021-0.396 without vs. 0.248-0.643 with AI decision support). Finally, AI decision support was reported as helpful by expert clinical readers in a significant number of cases (64%; P0.001) and was found to increase reader predictive accuracy independent of experience. Collectively, these results demonstrate that AI decision support provides more accurate, reproducible, and standardised pCMS risk predictions compared to clinical opinion alone, irrespective of investigator experience, across a large, international real-world dataset. Consensus Posterior Fossa Society practice recommendations on model use will be presented.
Sidpra et al. (Fri,) studied this question.