AI-Driven Load Forecasting and Optimization in Smart Grids using Cloud-Based Analytics

Abstract

Smart grids combine digital communications, real-time sensing, and cloud-based computing to facilitate two-way electricity and information flows, generating large amounts of data produced by smart meters, IoT sensors, and distributed energy resources. With the increase in renewable energy penetration, and nonlinear demand patterns, accurate short-term load prediction is now required to maintain grid stability, day-ahead scheduling, and demand response activation. Conventional statistical models are not enough to capture such dynamics, but the state of the research in AI has been mainly based on forecast accuracy in the laboratory with no attention to deployment, monitoring, or the operational connection between forecasts and grid decisions. This thesis explores the role of cloud-based AI models in improving the accuracy of short-term load forecasting and operational optimization in smart grids. The empirical basis is based on the Finnish national electricity consumption data of the Fingrid open data platform and weather covariates of Open-Meteo. Three AI models, Long Short-Term Memory networks, Prophet, and XGBoost are contrasted with ARIMA and a weekly persistence baseline under identical conditions, and evaluated based on RMSE, MAE, MAPE, and R square, which are all validated using Diebold-Mariano significance tests. On the Amazon Web Services, a six-stage MLOps pipeline is designed and implemented, which addresses data ingestion, preprocessing, training, validation, inference, and monitoring. XGBoost engineered lag, calendar and weather features have the best accuracy by all measures. The statistical baselines are surpassed by LSTM, whereas Prophet is viable under appropriate settings. The analysis of feature importance reveals that autoregressive structure prevails in short-horizon forecasting. At least twelve months of training data are determined to be operationally acceptable. The demand response scenarios are forecasted to allow automatic identification of the peak hours and low-load windows to be used as the decision support in the grid. The participant in the expert interview confirmed that 15-minute interval forecasting is the primary operational standard at the national transmission level, making short-term load forecasting the most operationally significant forecasting horizon in day-to-day grid management. The study concludes that, as a system-design problem, cloud-based AI forecasting can provide good load prediction accuracy and operative demand response indicators when using publicly available data and open-source tooling.