Hyperparameter Optimization Techniques for Enhanced Machine Learning Energy Forecasting:A Comparative Analysis

Abstract

Advanced machine learning (ML) models are essential for power system forecasting, yet their performance critically depends on architecture structure and parameter definition. Manual parameter tuning is time-consuming and forecasting errors can significantly impact utilities economically, making ML model optimization vital. This paper presents a comparative analysis of optimization techniques for tuning ML models across diverse energy data sources (photovoltaic (PV), mains, and battery energy storage systems (BESS)) and varying dataset sizes. Evaluation with real-world data on a Deep Neural Network (DNN) for 1-second ahead predictions revealed that Bayesian and meta-learning approaches consistently deliver superior performance with lower computational time. Grid search showed unexpected strength with smaller datasets, while random search and Population-Based Training (PBT) performed well with extensive data but degraded with small datasets. The Bayesian multi objective approach performed comparably to standard Bayesian optimization but with increased computational demands. Results revealed that all models showed 10-15% lower performance with mains data compared to PV, while BESS data yielded results approximately 3% below PV performance. The significant variance across data sources underscores the importance of tailoring optimization strategies to each energy data type’s inherent characteristics, including temporal volatility patterns, noise profiles, and feature correlations. Therefore, effective hyperparameter tuning must consider both computational constraints and the fundamental stochastic properties of the underlying energy systems