Impact of an Ophthalmic Knowledge Assessment Program (OKAP) Pre-Test: A Pilot Study at Two Institutions

Background: A formal assessment of knowledge prior to the Ophthalmic Knowledge Assessment Program (OKAP) may help ophthalmology residents identify areas of weakness and organize their study plans. Purpose: To assess the utility of an OKAP pre-test as a means of helping students prepare for the OKAP. Methods: A 91-question OKAP pre-test was formulated from the American Academy of Ophthalmology self-assessment study bank and administered in January to 28 residents at two academic institutions. Residents were given the results of their January test scores with a breakdown by subtest before the OKAP. Overall and subtest scores from the January pretest and the OKAP were standardized and compared. Based on the January pretest scores, we identified the subtests of weaker scores for each resident (≥18% points below their own average). We compared whether the change in score for that subtest (January pretest vs. OKAP) differed between those who performed poorly on the pretest subtest and those who did not. We used a linear mixed effects model accounting for variability by residents and subtests. Results: For the total test score, there was a positive association between January test scores and OKAP scores (r = 0.72; 95% CI, 0.48-0.86; p 0.25). The average score increase for subtests in which residents performed poorly was 1.32 standard deviations higher than the subtests in which they did not perform poorly (SE = 0.14, p < 0.001). Conclusions: The January test scores were predictive of residents’ performance on the OKAP. Additionally, January test scores may have helped guide OKAP studying, as residents who performed poorly on subtests in January improved their OKAP scores on those subtests more than residents who did not score poorly on the same subtest. The January test may help residents and faculty identify poor performance on particular subtests, allowing for more attention to those sections before the OKAP. Future studies could replicate our work on a larger scale to enhance generalizability