Moving beyond projects: a logic model evaluation of an established translational simulation program

Advancing beyond a sole focus on individuals’ and team’s clinical skills training, translational simulation uses simulation-based methodologies to diagnose healthcare system gaps and inform targeted quality and safety improvements. While promising conceptually, few reports exist on how to build sustainable, system-level translational simulation programs in hospital settings. We applied a logic model evaluation framework to assess four-years of activity in Unity Health Toronto’s established translational simulation program. Researchers not directly involved in the translational simulation program interviewed internal team members and program partners who had utilized the program. The interviews focused on the processes associated with project selection, implementation, and outcome collection. The research team used an iterative, team-based consensus approach to develop a fulsome logic model to richly describe the observed impacts and challenges of the program. Our analysis reflected experiences of 10 internal team members, and 14 program partners who contributed to 11 completed projects. We found that early collaboration during project intake was critical, but sometimes slowed perceived progress. Program members described a regular need to establish a shared conceptual understanding and clear roles at the outset with each program partner. Mutual adaptability between the simulation team and partners was reported as a defining factor for successful project implementation. Program outputs primarily consisted of simulation team generated reports, which generally met partner expectations; however, several partners desired more detailed, actionable analyses. Mid- to long-term outcomes varied based on project goals, including changes to clinical workflows, policy revisions, improved team performance, and enhanced patient outcomes. Grounded in refined claims for each logic model element, this evaluation provides transferable principles and data-driven recommendations for implementing and sustaining translational simulation programs. Our findings informed local program refinements while also producing strategies that can be applied broadly to translational simulation programs: 1) Flexible engagement tailored to partner needs and context 2) Clear definitions of simulation modalities, milestones, roles, objectives, and deliverables, 3) Structured communication throughout all project phases, and 4) Consistent follow-up to ensure translation of insights into clear returns on investment, and sustained system change.