Training deep neural networks on imperfect data is prone to training anomalies, including mislabeled samples, occlusions, distribution shifts, and stealthy backdoor triggers, that distort optimization signals and undermine robustness. This thesis presents a model-agnostic framework that detects and mitigates such anomalies in gradient space by monitoring deltas (gradients of the loss with respect to pre-activations) during training. The approach learns the manifold of normal training dynamics using lightweight, sparse autoencoders applied layer-wise and flags deviations via reconstruction error with an adaptive percentile threshold to track non-stationarity. A theoretical analysis shows that both label and input induced perturbations produce separable shifts in delta space across depth, justifying reconstruction-based detection. Practically, detection at deeper layers, especially the penultimate layer, offers strong separability while remaining architecture agnostic. The framework supports an online approach, integrates into standard training loops without prior knowledge of anomaly types, and enables selective filtering to stabilize optimization. Experiments across diverse architectures and benchmarks indicate improved robustness to heterogeneous anomaly sources with modest computational overhead, while preserving clean accuracy. Beyond performance gains, the method provides interpretable diagnostics of learning behavior, revealing where and how anomalies propagate through the network. The work concludes that monitoring and acting on gradient-space signals offers a practical route to training-aware robustness and outlines extensions to selfsupervised and reinforcement learning.
Βασίλειος Ν. Μουστακίδης (Wed,) studied this question.