Measuring Simulation Participant Playing Strategies

Background Improving awareness of Technical Debt (TD) has been demonstrated in prior work using the serious game TDSIM. Assessing how effectively TDSIM supports epistemic change is essential for its continued development. Examining participant play strategies offers a way to quantify a serious game’s impact on shifting player epistemic states, while also revealing opportunities for refinement. Yet the literature on serious game player behavior remains largely silent on strategy assessment, a gap our methodology directly addresses. Method The serious game TD-SIM was developed in prior work using real-world players. We analyze prior participant game data for actions taken. Combinations of actions define the concept of agent archetypes. Bayesian analysis is used to calculate probabilities of real-world play conditioned on agent archetypes. Principal Component Analysis (PCA) is used to classify conditional probabilities to which K-Means clustering is applied. Clusters of probability data are the strategies players of TD-SIM use, referred to as player archetypes. Results Our method identifies four player archetypes, characterizing playing strategies as Gaming , Managers , Learning , or Testing . Player archetypes inform and quantify behaviors of players, providing insight into improvement opportunities. Conclusion Measuring player archetypes is complex, but not impossible. We evaluate player archetypes of TD-SIM quantifying design improvements. Identifying and modelling participant playing strategies is a novel enhancement to general serious game evaluation. The benefit of our approach is that it allows us to characterize a player’s behavior in a quantifiable manner, which we can measure against intended purpose.