A hyperparameter optimization framework for transformer-based time series forecasting using evolutionary algorithms

Time series datasets often exhibit complex and diverse characteristics, including seasonal fluctuations, long-term trends, and irregular variations, which makes it difficult for a single, fixed machine learning model to achieve consistently accurate forecasts across different domains. To address this challenge, we introduce a neuroevolutionary framework for time series forecasting that integrates Evolutionary Algorithms with a Transformer Encoder designed for time series forecasting. This hybrid approach leverages the Transformer’s ability to capture short- and long-term dependencies while using evolutionary search to adjust the model’s hyperparameters to the unique dynamics of each dataset. In this work, we adopt an encoder-only Transformer architecture as a design choice motivated by computational efficiency and the nature of time series forecasting tasks, where the prediction does not require a full encoder–decoder structure. Its optimization is guided by evolutionary algorithms, namely Genetic Algorithm and one of the Estimation of Distribution Algorithms; the Population-Based Incremental Learning. The effectiveness of the approach is validated by benchmarking against ten publicly available univariate time series datasets that cover various patterns and structures. The results demonstrate the notable performance of the proposed model in terms of forecast precision and robustness, highlighting its ability to generalize across various time series scenarios.