Enhancing Extended-Context Dependency Resolution via Controlled Epistemic Noise Injection

Foundation models frequently encounter mid-sequence attention degradation when processing extended contexts, a limitation that hampers distant signal retrieval. To mitigate this structural deficit, we introduce a scalable, synthetic data curriculum designed to fortify long-range information routing mechanisms. Our methodology employs a progressive context-scaling paradigm. Initially, we synthesize high-fidelity, localized reasoning trajectories utilizing a dual-phase hypothesis verification protocol, ensuring that the generated outputs are causally dependent on specific reference priors rather than intrinsic model weights. Subsequently, we project these localized tasks into ultra-long sequence regimes through the uniform, stochastic injection of epistemically irrelevant distractors. This controlled modulation of the signal-to-noise ratio forces the underlying network to develop robust heuristics for signal isolation, non-local dependency modeling, and position-invariant feature extraction. Empirical observations indicate that integrating this synthesized curriculum significantly elevates the upper bound of extended-context retrieval capabilities. Furthermore, we observe a non-trivial transfer effect: exposure to dense, adversarial contexts yields measurable generalization improvements in localized, logically rigorous reasoning domains. This work validates that task-aware data co-design, specifically through progressive sequence augmentation and adversarial noise integration, is a highly effective strategy for unlocking advanced reasoning in massive-context regimes.