Raw material criticality assessments are essential tools for policy-makers, industry stakeholders, and researchers. A decade ago, two articles critiqued the compliance of criticality studies with risk theory, concluding that raw material criticality assessments were fundamentally flawed in several ways. Here, we systematically review 36 peer-reviewed criticality assessments published between 2017 and 2024, examining whether they have corrected previously identified methodological shortcomings. We show that most studies rely on unjustified aggregation methods, some of which may yield statistically “worse-than-useless” results, and consistently overlook key aspects of quantitative risk theory. Empirical validation of indicators is rare, revealing continued reliance on indicators without demonstrated causal links. Nearly half of the studies citing the two articles criticizing the field from the point of view of risk theory misinterpret or overlook the core arguments of these critiques. We complement this review by identifying methodological advancements from adjacent disciplines, including quantitative risk assessments of natural hazards and empirically grounded causal modeling. Drawing on these advancements, we outline two tracks for progress: improving indicator-based assessments through empirical testing, and developing quantitative risk analyses at facility or company resolution. Given the influence of criticality assessments, weak or subjective methods can misallocate public funds. Where methodological robustness cannot be demonstrated, publishing lists of “critical” materials is unjustified.
Andrieu et al. (Tue,) studied this question.