This paper presents a self-contained rate–distortion theorem for a simple but practically relevant class of product quantizers under a covariance domination condition that arises naturally when delta encoding is applied to temporally correlated embedding sequences. We consider coordinate-wise uniform scalar quantization in an eigen-basis of the input covariance and prove that the mean squared error (MSE) is upper-bounded by a constant times the covariance trace. When the covariance of a delta representation is dominated by that of the original representation in the Loewner order, this immediately implies a worst-case MSE advantage for delta encoding under a fixed quantization configuration. Although the core result is fully abstract, the theorem is motivated by, and designed for, the Mnemosyne Project, an edge-oriented large language model (LLM) infrastructure. Part III of Mnemosyne focuses on compressing activation vectors via Soft-ZCA whitening and product quantization (PQ). In this context, our theorem supplies the mathematical backbone for Chapter 3.2.1 (Theorem 5.1‑R), which claims that inserting a delta-encoding stage before whitening+PQ can strictly improve MSE-type distortion on a large subset of tokens, while respecting updated security requirements that no longer treat delta encoding as a cryptographic entropy source. The present version is purely theoretical: we provide complete proofs of the scalar and vector quantization lemmas, the Loewner-order trace monotonicity, and the coordinate-wise PQ trace bound. We state LLM-specific claims as conditional on an empirically testable covariance domination assumption. We also discuss how recent empirical work on residual-based KV compression, angle-based KV quantization, and semantic/context compression for LLMs makes this assumption highly plausible, suggesting that subsequent experiments on Mnemosyne’s embedding streams are likely to match the theoretical predictions.
Bo Jun Han (2026) studied this question.