Accurate residential load forecasting is a key to achieve grid stability and efficient energy management. However, it becomes challenging due to the non-linear and seasonally fluctuating energy usage of domestic users. Existing statistical and machine learning-based forecasting models struggle to produce accurate forecasts due to dynamic and stochastic user behaviors for energy usage. On the other hand, pairwise ensemble methods can achieve higher forecasting accuracy in short-term load forecasting, but are not scalable and face generalization issues that often lead to overfitting and complexity in managing multivariate data. To address these limitations, we propose to integrate LightGBM, XGBoost and CatBoost models to systematically address the limitations of existing ensemble-based forecasting models. This integration aims to leverage the strengths of each ensemble method, where LightGBM handles generalization across multiple sites, XGBoost avoids overfitting the model, and CatBoost effectively manages categorical features. We implement our proposed model using a real-world, publicly available dataset for 13 residential locations in North America and Europe. The proposed model outperforms other state-of-the-art algorithms with the lowest root mean squared logarithmic error (RMSLE) values of 0.1898, while the coefficient of determination (R²) calculated from the data is 0.9745. Other evaluation measures such as root mean square error (RMSE), coefficient of variation of the root mean square error (CVRMSE) and mean bias error (MBE) also support the proposed approach regarding the efficiency of the model.Finally, we also perform an ablution study to show predictive efficacy of incremental model integration.

Demand response (DR) is increasingly recognized as a critical flexibility resource for modernizing power systems, enabling the large-scale integration of renewable energy and enhancing grid stability. While the field of general electricity load forecasting is supported by numerous systematic reviews, the specific subfield of DR potential forecasting has received comparatively less synthesized attention. This gap leaves a fragmented understanding of modeling techniques, practical implementation challenges, and future research problems for a function that is essential for market participation. To address this, this paper presents a PRISMA-2020-compliant systematic review of 172 studies to comprehensively analyze the state-of-the-art in DR potential estimation. We categorize and evaluate the evolution of forecasting methodologies, from foundational statistical models to advanced AI architectures. Furthermore, the study identifies key technological enablers and systematically maps the persistent technical, regulatory, and behavioral barriers that impede widespread DR deployment. Our analysis demonstrates a clear trend towards hybrid and ensemble models, which outperform standalone approaches by integrating the strengths of diverse techniques to capture complex, nonlinear consumer dynamics. The findings underscore that while technologies like Advanced Metering Infrastructure (AMI) and the Internet of Things (IoT) are critical enablers, the gap between theoretical potential and realized flexibility is primarily dictated by non-technical factors, including inaccurate baseline methodologies, restrictive market designs, and low consumer engagement. This synthesis brings much-needed structure to a fragmented research area, evaluating the current state of forecasting methods and identifying the critical research directions required to improve the operational effectiveness of DR programs.

Accurately valuing demand-side flexibility hinges on two tightly coupled tasks: reconstructing the customer baseline load (CBL) that would have occurred in the absence of demand-response (DR) events and, from that baseline, forecasting the flexibility of load that can be curtailed. Existing studies usually treat these problems in isolation and often depend on appliance-level datasets that are rarely available at scale. This paper proposes DualDRNet, a unified deep learning framework that converts one-dimensional smart-meter load data into frequency-aware 2-D tensors, then captures intra-period and inter-period dynamics through a CNN–Transformer vision backbone. CBL estimation is cast as a masked-imputation task, where DR intervals are treated as missing values and the model learns the missing patterns mimicking CBL. DR Potential forecasting is addressed with the same network, trained autoregressively for multi-horizon prediction. The model is evaluated on the Low Carbon London dataset comprising half-hourly readings from 10% households exposed to 77 peak shaving and 81 valley filling DR events. DualDRNet lowers CBL root-mean-square error (RMSE) to $0.715~kW$ , mean absolute percentage error (MAPE) to 6.9%, outperforming both averaging and deep learning based CBL estimation baselines. For aggregated DR Potential forecasting, it sustains accurate performance from five- to nine-day horizons, delivering lowest RMSE and MAPE relative to state-of-the-art temporal convolutional, hierarchical interpolation and Transformer variants. By coupling baseline reconstruction and flexibility forecasting in a single architecture with near-identical hyper-parameters, DualDRNet offers load aggregators a practical, end-to-end solution for day-ahead market bidding without the need for intrusive sub-metering. Future work will extend the approach to mixed commercial–industrial portfolios and explore parameter-efficient CNN-Transformer variants suitable for edge deployment where computational requirements are more stringent.

Automated demand response in residential sectors is critical for grid stability, but centralized control strategies fail to address the unique energy profiles of individual households. This paper introduces a decentralized control framework using multi-agent deep reinforcement learning. We assign an independent Soft Actor–Critic (SAC) agent to each building’s battery energy storage system (BESS), enabling it to learn a control policy tailored to local conditions while responding to shared grid signals. Evaluated in a high-fidelity simulation environment of CityLearn using real-world data, our multi-agent system demonstrated a reduction of approximately 50% in both electricity costs and carbon emissions. Crucially, this decentralized approach considerably outperformed all benchmarks, including a rule-based controller, tabular Q-learning, and even a centralized single-agent SAC controller. At the district level, learned policies flatten the net load profile, lowering daily peaks by 16% and ramping by 26%, and improve the load factor. The resulting dispatch patterns are interpretable and consistent with operator objectives such as peak shaving and valley filling. These findings indicate that decentralized reinforcement learning can translate local optimization into system-level benefits and offers a scalable pathway for aggregators and utilities to operationalize the flexibility of residential storage at scale.

Demand response is shifting towards continuous coordination of flexible demand, storage, and distributed generation across buildings and prosumer communities. Multi-agent reinforcement learning has gained attention because it can support decentralized execution under partial observability while still learning coordinated behavior through centralized training. This systematic review follows PRISMA 2020 guidance and synthesizes $n=70$ peer-reviewed studies published in the 2021 to 2025 window, covering building clusters, grid-aware district coordination, program-level aggregation, industrial demand response, and transactive energy mechanisms. The results show that the dominant evaluation context is grid-responsive building clusters, with growing reliance on benchmark environments that standardize interfaces and encourage reproducible multi-KPI reporting. Across the methods, centralized training with decentralized execution is the prevailing pattern, often combined with attention-based critics or value factorization to handle heterogeneity and global rewards. Reward design and constraint handling emerge as primary determinants of stability, since objectives mix cost, peak, ramp, comfort, and emissions, while rebound and synchronized behavior are recurring risks. A descriptive and cross-variable quantitative synthesis is also provided, showing that publication activity increased from three studies (4.3%) in 2021 to 28 studies (40.0%) in 2025, with the strongest concentration in 2024–2025. Quantitatively, grid-responsive building clusters accounted for 26 of 70 studies (37.1%), actor–critic methods for 24 studies (34.3%), CityLearn for 16 studies (22.9%), and cost-based evaluation was reported in 64 studies (91.4%), whereas robustness testing appeared in only 16 studies (22.9%). Across the reviewed studies, peak reduction was reported in 55 (78.6%) studies, whereas robustness testing appeared in only 16 studies (22.9%) and transferability or deployment realism in 11 (15.7%), indicating that evaluation remains much stronger for operational performance than for real-world generalization.