PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
March 31, 20260 citationsOpen Access

Topological Gaps exist before learning and are amplified by regularization

View Full Paper
RRRégis RIGAUD

Key Points

  • This research aims to determine if missing training categories create detectable topological signatures in neural network representations.
  • Compared PCA, deterministic autoencoder (AE), and variational autoencoder (VAE) on MNIST and Fashion-MNIST.
  • Analyzed data under five ablation conditions to assess the impact of missing categories.
  • Conducted ghost centroid analysis and beta sweep across KL weights to evaluate signal strength.
  • Topological gaps are detectable within raw PCA projections in 2 out of 5 conditions.
  • Nonlinear encoding without regularization reduces the detection of topological gaps in AE.
  • KL regularization amplifies the detection of topological signatures in VAE across all conditions.

Abstract

We investigate whether missing training categories create detectable topological signatures in learned representations. Comparing PCA, a deterministic autoencoder (AE), and a variational autoencoder (VAE) on MNIST and Fashion-MNIST under five ablation conditions, we observe three regimes. This is an interpretive framework, not a formal derivation. First, topological gaps are already detectable in raw PCA projections (2/5 conditions). Second, nonlinear encoding without regularization reduces this signal (AE: 1/5). Third, KL regularization amplifies it in this setting (VAE: 5/5 means above null). A ghost centroid analysis suggests a mechanism consistent with directed aspiration (r = -0. 28, p < 10^-5, in VAE only). A beta sweep across six KL weights produces a dose-response curve. Five classical geometric metrics fail to predict signal strength. A replication on Fashion-MNIST confirms that the phenomenon generalizes beyond handwritten digits (3/5 conditions above null). A random ablation control indicates that approximately 30–50% of the signal is specific to categorical removal. A reconstruction error baseline confirms that MSE and topological detection capture orthogonal aspects of the gap. Our results suggest that variational regularization renders visible and amplifies certain topological signatures of missing categories.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Régis RIGAUD (2026) studied this question.

synapsesocial.com/papers/69cb650ee6a8c024954b9148https://doi.org/10.5281/zenodo.19309873
Ask AI
Helpful
Bookmark
Share
View Full Paper

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Missing Attractors: An Energy-Landscape View of Detectable Topological Scars2026
  2. 2Mathematical reduction of KL divergence to L2 regularization in the loss function of a variational autoencoder2026
  3. 3Analysis of Variational Sparse Autoencoders2025
  4. 4Topological degree as a discrete diagnostic for disentanglement, with applications to the $Δ$VAE2024
  5. 5Ensuring Topological Data-Structure Preservation under Autoencoder Compression Due to Latent Space Regularization in Gauss–Legendre Nodes2024 · 2 citations