Abstract: We present ThermoRG, a phenomenological thermodynamic framework for neural architecture scaling. The central objects are: (1) the effective dimension D₄₅₅ of each weight matrix, (2) the topological correlation Jₓ₎₎ measuring information-flow quality, (3) the variance fluctuation characterizing training dynamics, and (4) the scaling exponent governing how fast loss decreases with width. We first validate the core predictions of ThermoRG using controlled micro-experiments on plain convolutional networks with different normalization layers. We establish a two-channel decomposition of the asymptotic error floor that explains Simpson's paradox in architecture correlations. Based on this physical equation of state, we introduce ThermoRG-AL, a calibration-guided architecture search framework. After a one-time dataset calibration, it pre-screens architectures semi-analytically at zero cost (1ms per architecture). On CIFAR-10, ThermoRG-AL achieves comparable or better results than random search, and critically, its predictions are independently cross-validated by SynFlow, a gradient-based zero-cost method, with both converging on the identical optimal architecture. Scope of this Release (Version 1. 0 - Proof of Concept): This initial preprint serves as a foundational proof-of-concept for the ThermoRG theoretical framework. The quantitative validation of the scaling laws, the () cooling theory, and the architecture search algorithm (ThermoRG-AL) are currently strictly calibrated and validated on the CIFAR-10 dataset using plain ConvNet building blocks. While observational Jₓ₎₎ analyses for ResNet and DenseNet are discussed, large-scale validation across more complex architectural motifs and datasets (e. g. , ImageNet, ViT) remains active future work
Xiaonan Liu (Thu,) studied this question.