SHEAF: Spectral-Heavy-tail Eigen-Anchored Folding with Fisher-Calibrated Dynamic Precision

SHEAF is a post-training compression scheme for dense layers in deep neural networks. We split each weight matrix into two parts: a small set of heavy-tailed singular components kept at full precision (the anchor), and the remaining bulk, which we approximate with a Monarch factorization (the fold). Activations are quantized with a per-block bit-width derived from local entropy, subject to a Lipschitz bound that limits how much the layer's output can deviate. On the FFN, QKV, and projection layers of Transformers, and on the FC and 1×1-convolutional layers of vision models, this gives a 3–4× reduction in both memory and FLOPs. The drop in output KL divergence is bounded analytically through the Fisher information metric, with the bound controlled by a single truncation parameter — provided the layer's spectrum is heavy-tailed, which is the regime LLM and ViT weights typically fall into under the Heavy-Tailed Self-Regularization framework. SHEAF is near-lossless, not lossless. It does not help with embeddings (flat spectra), with very small attention projections, or with layers that have high Fisher curvature.