The Information Density Limit: Why Adding GPUs Can Make Training Slower

There exists a density where adding accelerators makes synchronized training slower. At the IDL crossover, the fabric stops acting like many independent links and behaves like one coupled noisy medium; Shannon capacity globalizes, ECC and retries ratchet, and effective payload drops through NVLink‑class tiers 810→720→630→540 GB/s. This paper formalizes the crossover and shows why speedup curves get an early knee and then reverse --- Large-scale AI training systems are conventionally modeled as parallel computers limited only by compute, memory, power, and latency. In practice, performance plateaus and efficiency cliffs appear much earlier than these primitive constraints predict. This paper shows that the missing constraint is the Information Density Limit (IDL): the critical density at which the interconnect fabric transitions from many independent high-bandwidth channels into a single coupled noisy medium governed by shared environmental noise (EMI, thermal drift, and vibration). At this crossover density D*, Shannon capacity globalizes across the system rather than remaining link-local. Error-correction overhead and retransmission probability then grow faster than any nominal bandwidth gain, driving effective payload through discrete degradation tiers (e. g. , 810 → 720 → 630 → 540 GB/s in NVLink-class fabrics) and ultimately reversing the direction of strong scaling: adding accelerators increases wall-clock step time (dTₛtep/dD > 0). The IDL is not a seventh primitive wall but a multiplicative amplifier that warps the Compute-Efficiency Frontier inward, pulling the Information/Throughput axis down while simultaneously tightening the Power, Heat, Parallelism, and Transmission boundaries through closed feedback loops. This mechanism provides a unified physical explanation for the observed early knees in speedup curves, rising joules per useful gradient, and the accelerating architectural divergence between dense, deterministic training islands and sparser, latency-predictable inference fabrics. The work is purely theoretical and conceptual. It introduces the IDL primitives, derives the crossover condition, formalizes the feedback dynamics, and offers engineering guidelines for delaying (but not eliminating) the regime change. It deliberately omits empirical validation, hardware diagnostics, or predictive simulations, serving instead as a foundational structural lens for future modeling, simulation, and design of hyperscale GPU fabrics.