HD-PUD: Holographic Encoding and Positional Unbinding as an Algebraic Alternative to Sub-Word Tokenization

Sub-word tokenization (BPE, WordPiece, SentencePiece) imposes a learned, corpusdependent fragmentation of text. The mapping between characters and token identifiers is not algebraically structured: similar morphemes may map to unrelated identifiers, and adding a single new word to the vocabulary may require retraining or shifting embedding indices. We describe HD-PUD (HyperDimensional Positional Unbinding Decoder), a tokenizer-free encoding scheme in which each sub-word segment is represented as a single hyperdimensional vector v ∈ Rd obtained by holographic binding of character embeddings with deterministic sign-vector positional keys. Decoding is performed by element-wise unbinding against an absolute-position key followed by a single matrix-vector product against the character codebook, recovering individual characters. The construction yields three properties by algebraic guarantee, without learning: (i) positional reversibility—every character at every position can be recovered up to an interference term that decreases as 1/√d; (ii) compositional locality—segments sharing prefixes, suffixes or internal n-grams have analytically predictable cosine similarity; (iii) drift-free vocabulary extension—adding a new character to the alphabet does not modify any previously computed segment representation. We give a complete mathematical specification, a NumPy reference implementation, and CPU-only experiments that empirically verify the algebraic properties on commodity hardware. We do not provide end-to-end language modeling benchmarks; this work is framed as a theoretical proposal. We outline falsifiable predictions, the minimal experimental conditions under which they should be tested, and the qualitative behavior expected when HD-PUD is plugged into an existing transformer pipeline.