Evaluating “Effectiveness” under the Policy on Results

Abstract Evaluation units provide essential information for administrators of public programs. Canada's 2016 Policy on Results emphasizes evaluating “effectiveness,” with a focus on outcomes. However, it is unclear if evaluation units use this or more implementation‐oriented definitions in practice, which has implications for the information evaluations will provide. This study examines 370 evaluation reports published by the largest federal departments to determine how evaluation functions are approaching effectiveness, using a combination of text mining and manual thematic analysis. Since 2014, evaluation reports have increasingly adopted implementation‐oriented approaches, with variations across departments. Explanations for these patterns are discussed, along with policy implications.