Traffic signal control plays a pivotal role in intelligent transportation systems, directly affecting urban mobility, congestion mitigation, and environmental sustainability. As traffic networks become more dynamic and complex, traditional strategies such as fixed-time and actuated control increasingly fall short in addressing real-time variability. In response, adaptive signal control—powered predominantly by reinforcement learning—has emerged as a promising data-driven solution for optimizing signal operations in evolving traffic environments. The current review presents a comprehensive analysis of high-impact reinforcement-learning-based traffic signal control methods, evaluating their contributions across numerous key dimensions: methodology type, multi-agent architectures, reward design, performance evaluation, baseline comparison, network scale, practical applicability, and simulation platforms. Through a systematic examination of the most influential studies, the review identifies dominant trends, unresolved challenges, and strategic directions for future research. The findings underscore the transformative potential of RL in enabling intelligent, responsive, and sustainable traffic management systems, marking a significant shift toward next-generation urban mobility solutions.

With the rise in electricity, gas and oil prices and the persistently high levels of carbon emissions, there is an increasing demand for effective energy management in energy systems, including electrical grids. Recent literature exhibits large potential for optimizing the behavior of such systems towards energy performance, reducing peak loads and exploiting environmentally friendly ways for energy production. However, the primary challenge relies on the optimization of such systems, which introduces significant complexities since they present quite dynamic behavior. Such cyberphysical frameworks usually integrate multiple interconnected components such as power plants, transmission lines, distribution networks and various types of energy-storage systems, while the behavior of these components is affected by various external factors such as user individual requirements, weather conditions, energy demand and market prices. Consequently, traditional optimal control approaches—such as Rule-Based Control (RBC)—prove inadequate to deal with the diverse dynamics which define the behavior of such complicated frameworks. Moreover, even sophisticated techniques—such as Model Predictive Control (MPC)—showcase model-related limitations that hinder the applicability of an optimal control scheme. To this end, AI model-free techniques such as Reinforcement Learning (RL) offer a fruitful potential for embedding efficient optimal control in cases of energy systems. Recent studies present promising results in various fields of engineering, indicating that RL frameworks may prove the key element for delivering efficient optimal control in smart buildings, electric vehicle charging and smart grid applications. The current paper provides a comprehensive review of RL implementations in energy systems frameworks—such as Renewable Energy Sources (RESs), Building Energy-Management Systems (BEMSs) and Electric Vehicle Charging Stations (EVCSs)—illustrating the benefits and the opportunities of such approaches. The work examines more than 80 highly cited papers focusing on recent RL research applications—between 2015 and 2023—and analyzes the model-free RL potential as regards the energy systems’ control optimization in the future.

The integration of renewable energy systems into modern buildings is essential for enhancing energy efficiency, reducing carbon footprints, and advancing intelligent energy management. However, optimizing RES operations within building energy management systems introduces significant complexity, requiring advanced control strategies. One significant branch of modern control algorithms concerns reinforcement learning, a data-driven strategy capable of dynamically managing renewable energy sources and other energy subsystems under uncertainty and real-time constraints. The current review systematically examines RL-based control strategies applied in BEMS frameworks integrating RES technologies between 2015 and 2025, classifying them by algorithmic approach and evaluating the role of multi-agent and hybrid methods in improving real-time adaptability and occupant comfort. Following a thorough explanation of a rigorous selection process—which targeted the most impactful peer-reviewed publications from the last decade, the paper presents the mathematical concepts of RL and multi-agent RL, along with detailed summaries and summary tables of the integrated works to facilitate quick reference to key findings. For evaluation, the paper examines and outlines the different attributes in the field considering the following: methodologies of RL; agent types; value-action networks; reward functions; baseline control approaches; RES types; BEMS types; and building typologies. Grounded on the findings presented in the evaluation section, the paper offers a structured synthesis of emerging research trends and future directions, identifying the strengths and limitations of RL in energy management.

The integration of advanced control strategies into building energy management systems (BEMS) is essential for achieving energy efficiency and sustaining thermal comfort. Model predictive control (MPC) has gained significant traction as a model-based approach capable of optimizing control actions by predicting future system behavior under dynamic conditions. The current review offers an in-depth analysis of MPC, combining its core theoretical foundations with a broad survey of impactful applications in buildings, for extracting key breakthroughs and trends that have defined the field over the past decade. Emphasis is placed on multiverse MPC configurations and their application across various BEMS frameworks integrating HVACs, energy storage, renewable energy, domestic hot water, electric vehicle charging, and lighting systems. A detailed evaluation of MPC key attributes is then conducted, based on essential aspects of MPC, such as algorithms, optimization solvers, baselines, performance indexes, and building types, as well as simulation tools that support system modeling and real-time validation. The study concludes by outlining key research trends and proposing future directions, with a strong emphasis on addressing real-world deployment challenges and advancing scalable, interoperable solutions on smart building ecosystems. According to the evaluation, MPC research is shifting from simple white-box setups to gray- and black-box models paired with metaheuristic or hybrid solvers, leveraging machine learning for forecasting and multi-objective optimization, but still lacking robustness, benchmarks, and real-world validation. Consequently, next-generation MPC is anticipated to evolve into adaptive, hybrid, and multi-agent frameworks that integrate forecasting and control, embed occupant behavior, enable grid-interactive flexibility, and support lightweight, explainable deployment in real building environments.

The efficient control of HVAC devices in building structures is mandatory for achieving energy savings and comfort. To balance these objectives efficiently, it is essential to incorporate adequate advanced control strategies to adapt to varying environmental conditions and occupant preferences. Model-free control approaches for building HVAC systems have gained significant interest due to their flexibility and ability to adapt to complex, dynamic systems without relying on explicit mathematical models. The current review presents the recent advancements in HVAC control, with an emphasis on reinforcement learning, artificial neural networks, fuzzy logic control, and their hybrid integration with other model-free algorithms. The main focus of this study is a literature review of the most notable research from 2015 to 2023, highlighting the most highly cited applications and their contributions to the field. After analyzing the concept of each work according to its control strategy, a detailed evaluation across different thematic areas is conducted. To this end, the prevalence of methodologies, utilization of different HVAC equipment, and diverse testbed features, such as building zoning and utilization, are further discussed considering the entire body of work to identify different patterns and trends in the field of model-free HVAC control. Last but not least, based on a detailed evaluation of the research in the field, the current work provides future directions for model-free HVAC control considering different aspects and thematic areas.