Background: Meta-analyses conventionally report pooled effect sizes, confidence intervals, and p-values. These measures address statistical significance and precision, but not how far the results are from a "no benefit" (nb) point, as measured on a standardized robustness scale. The Neutrality Boundary Framework provides a robust metric, nb, that ranges from 0 to 1 for individual studies, but its use in meta-analysis has not been formally described. A nb of 0 indicates that results are at the neutrality boundary, where no relationship between treatment and outcome is present; a nb of 1 indicates that results show a strong relationship between treatment and outcome. Methodology: A sample of published clinical trials was selected for evaluation. The distribution of nb across all trials was summarized, and the correlation between nb and p-values was examined. Results: Across 161 trials, the median nb was 0.147 (range 0.000 to 0.902). Using standardized robustness bands derived from prior validation work based on simulation and trial data, 35.4% of trials showed weak robustness (nb less than 0.075), 24.8% moderate robustness (nb between 0.075 and 0.227), and 39.8% strong robustness (nb greater than or equal to 0.227). Trials with binary 2-by-2 outcomes tended to have lower nb values than trials with continuous outcomes. The robust metric nb was moderately correlated with p-values (r = 0.35, p less than 0.001). Although a moderate level of correlation was present, p-values explained only about 12% of the variation in nb (r-squared = 0.12), indicating that nb captures substantial additional information about distance from neutrality that p-values alone do not provide. Conclusion: The distribution of trial-level nb values provides a simple way to describe robustness across a collection of studies and offers a cross-design view of how far the overall evidence base lies from therapeutic neutrality. Incorporating robust measures alongside p-values in routine evidence synthesis may provide a more complete picture of the strength and reliability of the evidence.
Thomas F. Heston (Thu,) studied this question.