Uncertainty quantification in hierarchical healthcare data presents a fundamental methodological challenge. Existing approaches face fundamental trade-offs. Conformal prediction offers coverage guarantees but struggles to adapt to instance-level difficulty in clusters, while Bayesian methods quantify uncertainty but lose reliability when models are misspecified. We propose a hybrid framework for hierarchical random forests that addresses the trade-off between coverage and precision by combining group-aware conformal calibration with Bayesian posterior uncertainties to yield adaptive prediction intervals with near-nominal empirical coverage. We assess length-of-stay predictions on 61538 patients from 3793 hospitals using 5-fold cross-validation. The hybrid approach achieved near-target coverage (94.3 % ± 0.5 pp vs 95 % nominal) with adaptive interval width reallocation across uncertainty strata (21 % narrower for low-uncertainty cases, 6 % wider for high-uncertainty cases) comparable to standard conformal prediction (95.0 % ± 0.2 pp) which produces uniform-width intervals. Post-hoc Bayesian calibration alone severely under-covered (14.1 %) under our Gaussian specification, highlighting the necessity of conformal adjustment. Performance remained stable across hospitals and folds, with grouped cross-validation (hospital holdout) confirming generalization to unseen institutions. The pooled calibration approach achieves near-nominal empirical coverage at scale, supporting evidence-based resource allocation and risk-stratified patient management without site-specific recalibration.
Shahbazi et al. (Thu,) studied this question.