BatchNorm Layers have an Outsized Effect on Adversarial Robustness

Key Points

• Adversarial robustness is influenced by how BatchNorm layers are optimized, showing variations across different strategies.
• Alterations in the optimization techniques for BatchNorm layers yield distinct changes in clean accuracy when confronted with adversarial examples.
• The approach includes a partial adversarial pre-training methodology for investigating BatchNorm layers within models.
• Understanding the effect of BatchNorm layers on adversarial robustness could inform better model design choices for enhanced security.