What question did this study set out to answer?

The study aims to evaluate how PCA and Elastic Net influence the Double Descent phenomenon in polynomial regression models.

March 13, 2026Open Access

PCA and Elastic Net VS Double Descent

Key Points

The study aims to evaluate how PCA and Elastic Net influence the Double Descent phenomenon in polynomial regression models.
Analyzed a synthetic nonlinear dataset with degrees up to 129.
Compared the effects of PCA and Elastic Net on model stability.
Examined how PCA removes low-variance components versus Elastic Net’s coefficient smoothing.
PCA effectively stabilizes models by eliminating components that lead to error spikes.
Elastic Net provides smoothing but does not prevent error spikes as effectively as PCA.
Insights gained can help stabilize high-dimensional models in overparameterized regimes.

Abstract

This study investigates the impact of Principal Component Analysis (PCA) and Elastic Net regularization on the "Double Descent" phenomenon in polynomial regression. Using a synthetic nonlinear dataset and degrees up to 129, we analyze how PCA stabilizes models by removing low-variance components that cause error spikes, compared to Elastic Net’s approach of coefficient smoothing. Our findings provide insights into stabilizing high-dimensional models in overparameterized regime The implementation code and experiments are available at: https://github.com/mouradgad1/Statistical-Learning-Analysis-Double-Descent-and-PCA

PCA and Elastic Net VS Double Descent

Key Points

Abstract

Cite This Study