• Validation design shapes predictive performance and geological interpretation. • LOCO validation provides a conservative test of cross-cluster transferability. • A compact predictor core remains stable under stricter validation. • Recurrent predictors indicate regionally transferable permissive factors. • Random, spatially blocked, and LOCO schemes offer complementary perspectives. Machine learning-based mineral prospectivity mapping (MPM) has become an important framework for regional mineral exploration, but objective model evaluation remains difficult because mineral occurrences are strongly spatially clustered and the data are highly imbalanced. Under these conditions, conventional validation schemes may reward local analogue matching rather than more transferable geological relationships. Using a continental-scale dataset for Canadian magmatic Ni (±Cu ± Co ± PGE) sulphide systems, this study compares random cross-validation, spatially blocked cross-validation, and convex hull-based leave-one-cluster-out (LOCO) validation to examine how validation design influences predictive performance, prospectivity patterns, and predictor interpretation. Rather than treating validation only as a benchmarking tool, we use it as an interpretive framework for assessing model behavior under progressively stricter spatial and cluster-level separation. Apparent predictive performance declines as validation becomes more conservative, with LOCO validation providing the most demanding test among the schemes considered here. At the same time, the three validation regimes yield broadly similar first-order prospectivity patterns at continental scale, whereas clearer differences emerge in the allocation of the highest-priority hotspot cells. Feature-stability analysis identifies a compact cross-regime predictor core, including geological age, the geology feature family, magnetic horizontal gradient magnitude, lake and stream sediment Ni, and Moho depth. These recurrent predictors are most plausibly interpreted as regionally transferable first-order permissive factors, whereas several secondary predictors, particularly seismic-velocity and radiometric variables, show stronger dependence on validation design and are therefore better viewed as more context-sensitive evidential layers. Overall, the results show that validation strategy fundamentally shapes both predictive assessment and geological interpretation in prospectivity mapping of magmatic Ni sulphide systems, and that random CV, spatially blocked CV, and LOCO validation are best viewed as complementary rather than interchangeable evaluation perspectives.
Oh et al. (Fri,) studied this question.