Benchmark Recall versus Deployed Consistency: An Empirical Comparison of Memory Strategies for Local Small-Model NPCs

This preprint studies memory management for local small-model NPCs under fixed prompt budgets. We compare three representative strategies: sliding-window recency, embedding-based retrieval, and a query-independent compression method called LogMem. The evaluation uses 10 REALTALK conversations as the main testbed and one LoCoMo conversation as a sparse-dialogue control, under fixed total budgets of 1K, 2K, and 4K tokens per answer call. On benchmark-style factual recall, embedding-based retrieval achieves the highest mean judge score at every tested budget. At the same time, the paper argues that benchmark recall and deployed NPC consistency are different targets, and that future game-facing evaluations should measure both. The practical motivation for this work comes from the design and deployment of memory systems for persistent characters in the Veridia interactive narrative platform. Project website: https://veridia.games Files included:- PDF manuscript- LaTeX/source submission package