There is limited guidance on the use of causal forests in moderately sized randomized controlled trials (RCTs). This study aimed to apply and evaluate a causal forest to estimate heterogeneous treatment effects of a healthy food subsidy program using data from a moderately sized RCT. Using data from a RCT of the British Columbia Farmers’ Market Nutrition Coupon Program (FMNCP) (n=263), a causal forest analysis examined heterogeneous treatment effects on Healthy Eating Index-2015 scores (HEI-2015; 0-100) post-intervention. Treatment effect heterogeneity was assessed in three ways: 1) using the best linear prediction test to measure covariance between predicted and true treatment effects, 2) comparisons of high versus low treatment effect groups, and 3) estimating rank-weighted average treatment effects on a targeting operator characteristics (TOC) curve and estimating the area under the TOC (AUTOC). A simulation-based power analysis examined the sample size at which the causal forest could detect a 5-point difference in HEI-2015 scores between participants with high versus low educational attainment. The best linear prediction test showed poor calibration and did not detect heterogeneous treatment effects (β 0.44, p=0.27). Heterogeneity was not detected when comparing groups with high versus low treatment effects (-2.25, 95% CI -13.53, 9.03). The TOC curve was flat with wide confidence intervals, and the AUTOC was not significant (-1.02 (p=0.77)). Based on the simulation-based power analysis, 1,050 participants were required to achieve 80% power, whereas our sample provided 40% power. A causal forest did not detect heterogeneous treatment effects of the FMNCP on the diet quality of adults with low incomes. Simulation results indicated that the trial was underpowered, underscoring the need for larger trials. Nevertheless, with careful application and evaluation, causal forests remain a useful tool to explore heterogeneous treatment effects in moderately sized trials when the sample size is adequate. Clinical trials registration number NCT03952338 https://clinicaltrials.gov/ct2/home
Aktary et al. (Wed,) studied this question.