Contemporary interpretability research in large language models operates predominantly through post-hoc analysis: probing trained models, tracing activations, and reverse-engineering internal representations after training is complete. This paper proposes a fundamentally different approach. Rather than forensic analysis of trained systems, we introduce the concept of training-time observability through embedded dynamic markers, a framework in which structured, behaviorally inert signals are embedded within training data prior to model training, enabling researchers to trace how specific signals propagate, transform, and persist within a model's internal representations. The central contribution is a formal design framework for such markers, which we term Steganographic Provenance Markers (SPMs). A valid SPM must satisfy four properties: non-dominance over the primary learning signal, persistence across transformation layers, recoverability through post-hoc probing, and orthogonality with respect to core semantic features. We further require that SPMs occupy the null space of the training objective, present in the distributional structure of data and carrying no gradient signal, such that the model neither optimizes toward nor away from them.
Amal Nair (2026) studied this question.