Neural networks suffer from operational amnesia: they process each input independently, without recalling which computational patterns proved effective in similar contexts. We introduce experiential neural architecture selection (ExNAS), a system that performs real-time, fine-grained structural adaptation during inference by leveraging a lightweight experiential memory that updates online as the system processes inputs. Unlike static pruning (which commits to fixed configurations at calibration) and input-adaptive methods (which train frozen gating networks), ExNAS maintains a non-parametric memory bank that records how computation succeeded and updates during inference when predictions fail. This enables learning from routing mistakes within the same inference session—a capability fundamentally impossible in prior approaches. We provide theoretical analysis showing that cosine similarity in fingerprint space bounds output divergence (justifying memory-based routing), that the sentinel energy ratio is bounded by rectified linear unit (ReLU) activation density (justifying its use as difficulty proxy), and that exponential recency weighting is the maximum-entropy decay under unknown distribution shift. We evaluate ExNAS across convolutional neural networks (CNNs) (CIFAR-10, CIFAR-100), video processing, and Transformers (Qwen2-0.5B). On CIFAR-100 with recurring inputs—the deployment scenario ExNAS targets—ExNAS achieves 94% accuracy on “trap zone” inputs where DynaBERT-style entropy routing achieves only 38%, demonstrating a +56 percentage point advantage on the slim model’s systematic blind spots. Code is publicly released under MIT license.
José María Lancho Rodríguez (Wed,) studied this question.