Measuring Neural Network Similarity

Deep neural networks often achieve similar performance despite substantial architectural differences, while small structural changes can cause large functional divergences. This highlights the need for a principled framework to quantify structural similarity (properties of the computational graph independent of training), functional similarity (defined by output behavior or internal representations), and their interaction. Drawing inspiration from biological concepts such as homology, analogy, and convergent evolution, I argue that neural architectures can be systematically compared using measures grounded in graph topology, motif composition, and representational alignment. Such measures would potentially enable the identification of architectural families, robust design patterns, and fragile “prone-to-break” configurations. I call for a unified strategy to measure neural network similarity to better understand architectural lineage, inform model design, and guide future neural architecture search.