Saeed Shamaghdari

Saeed Shamaghdari

Auto-generated profile, based on publicly available information

Saeed Shamaghdari

Electrical Engineering Department, Iran University of Science and Technology (IUST), Tehran, Iran

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 December 2022 in IEEE Transactions on Automation Science and Engineering
Verified
Journal Article
Closed access
SRSajad Roshanravan
Sajad Roshanravan
SSSaeed Shamaghdari
Saeed Shamaghdari
This paper investigates the optimal fault-tolerant tracking control (FTTC) problem for unknown affine nonlinear continuous-time systems with process and actuator faults in the framework of reinforcement learning (RL). The proposed novel active FTTC scheme is based on adaptive optimal control theory. In this way, the FTTC problem is formulated as an optimal regulation problem for the augmented system, which consists of the controlled system and the reference trajectory. To solve the Hamilton-Jacobi-Bellman (HJB) equation of the augmented system, an identifier-critic-based online RL strategy is employed with a dual neural network (NN) approximation structure. Initially, in order to remove the requirement of prior knowledge of the system dynamics, an adaptive NN identifier is designed. The forgetting factor in the proposed identifier update law is variable and a function of the filtered state estimation error and filtered state error. Choosing this variable forgetting factor increases the convergence speed and decreases the estimation error of identifier NN weights compared to the constant one while maintaining its robustness. When a fault occurs, the system continues to operate under the former FTTC until the fault is detected. Meanwhile, the optimal FTTC design in the RL framework requires the initial admissible control condition. In order to make it possible to initiate the FTTC learning process from the former FTTC, we employed a stabilizing term in the critical update rule. The Uniformly Ultimately Boundedness (UUB) of identifier and critic NN weight errors and, as a result, the convergence of the control input to the neighborhood of the optimal solution are all proved by Lyapunov theory. In the proposed method, changes in the values of faults are detected by comparing the HJB error to a predefined threshold. Finally, the simulation results are given to validate the effectiveness of the developed method. Note to Practitioners —long-time operations and the influence of external perturbations often make the faults inevitable for many practical engineering systems which can lead to unpredictable behaviors and catastrophic impacts. In general, the faults are naturally uncertain in time, value, and pattern, that is, it is unknown when, how much, and which system components fail. Therefore, the control system must be able to tolerate an extensive set of component faults. The design of optimal model-free FTTC strategies in an adaptive manner is challenging in nonlinear systems. The proposed method is suitable for a large class of nonlinear systems with input-affine form, and guarantees the system stability in the presence of process and actuator faults.

11 November 2024 in IEEE Access
Verified
Journal Article
Open access
PMParisa Mohammadi
Parisa Mohammadi
RDRazieh Darshi
Razieh Darshi
SSSaeed Shamaghdari
Saeed Shamaghdari
PSPierluigi Siano
Pierluigi Siano
The depletion of fossil fuel reserves and the urgent need to cut greenhouse gas emissions are driving a significant shift in the global energy sector. This transformation necessitates advanced energy management strategies in microgrids to integrate renewable energy sources efficiently. Traditional methods, including classical, metaheuristic, model predictive, stochastic, and robust control techniques, have been extensively studied. However, these approaches often struggle with slow performance and high computational demands, making them less effective for real-time applications due to the need for frequent re-optimization. In contrast, reinforcement learning, a branch of machine learning, excels by continuously learning and optimizing through real-time interactions This approach offers greater flexibility and adaptability in complex and dynamic environments. This paper provides a comparative review of various control strategies for microgrid energy management, examining their strengths and limitations. We place a particular focus on reinforcement learning algorithms and their application in microgrids. This paper conducts a detailed analysis of studies on reinforcement learning-based energy management systems, evaluating their potential to enhance energy efficiency, optimize usage, and contribute to the sustainability of modern energy infrastructures. This review aims to enhance understanding in this field and offer insights for future research on smart energy management systems.

23 February 2023 in IET Generation, Transmission & Distribution
Verified
Journal Article
Open access
RDRazieh Darshi
Razieh Darshi
SSSaeed Shamaghdari
Saeed Shamaghdari
AJAliakbar Jalali
Aliakbar Jalali
HAHamidreza Arasteh
Hamidreza Arasteh
This paper presents a novel fully decentralized and intelligent energy management system (EMS) for a smart microgrid based on reinforcement learning (RL) strategy. The purpose of the proposed EMS is to maximize the benefit of all microgrid entities comprising customers and distributed energy resources (DERs). Due to unpredictable features of renewable energy sources and variability of consumers’ demands, designing the microgrid EMS is a complicated task. To overcome this issue, the multi‐agent hour‐ahead energy management problem is modelled as a finite Markov decision process. The microgrid entities are considered as intelligent agents. The optimal policy of agents is obtained through a newly developed framework of the model‐free Q‐learning algorithm to maximize the benefit of all renewable and non‐renewable energy resources and battery energy storage system. The degradation model of the battery is considered to reduce the number of battery replacements. To ensure customers’ comfort, customers’ expenses are decreased without demand curtailment via introducing two types of load shifting techniques. The microgrid operation is analysed under four scenarios comprising no‐learning, generator‐learning, customer‐learning, and whole‐learning. the performance of the proposed algorithm is compared to the Monte Carlo method and simulation results on the real power‐grid dataset show the superiority of the algorithm.

14 June 2024 in IEEE Transactions on Automation Science and Engineering
Verified
Journal Article
Closed access
FGFatemeh Mahdavi Golmisheh
Fatemeh Mahdavi Golmisheh
SSSaeed Shamaghdari
Saeed Shamaghdari
This paper introduces a novel approach to solving robust optimal formation control problems in heterogeneous multi-agent systems (MASs) with disturbances. Our approach frames this problem as an adversarial graphical apprentice game problem, using the game concept and an inverse reinforcement learning (IRL) algorithm. This work innovates the derivation of unknown reward functions through demonstrations to achieve the desired formation. In addition, we define learner, estimator, and expert MASs as separate entities. This aim is achieved by introducing a model-based IRL algorithm with three stages of learning. After determining the optimal control of estimators in the first stage, the optimal control of learners is found in the second stage by using them. In the third stage, the reward functions of the estimator and learner MASs are updated in the inverse optimal control (IOC) stage simultaneously. Following that, we present an algorithm for model-free IRL that does not require knowledge of learner agents’ dynamics and can reconstruct reward functions after observing online trajectories. As subproblems, both proposed IRL algorithms address optimal control and IOC. We analyze these approaches for stability and convergence and demonstrate that state reward weights are non-unique. Simulation results show the effectiveness of the introduced algorithms for a group of unmanned aerial vehicles (UAVs). Note to Practitioners —Designing an optimal controller is challenging in MAS, especially where we only have access to the optimal reference path. The challenge is compounded if we do not know the agents’ dynamics and the appropriate reward function needed to achieve the desired path. This article proposes two IRL algorithms with the motivation to solve this problem. This study considers the optimal reference path for agents as expert MAS. Also, the MAS, we want to follow the desired path with the minor error, is represented as a learner MAS. The proposed approach is suitable for a large class of linear MAS since we have modeled the learner MAS with disturbance. Therefore, this method suits tracking, path-planning, and optimal formation problems in MASs. It requires minimal information but yields impressive results.

2 May 2017 in Aircraft Engineering and Aerospace Technology
Verified
Journal Article
Closed access
SSSeyed Hamed Seyedipour
Seyed Hamed Seyedipour
MJMohsen Fathi Jegarkandi
Mohsen Fathi Jegarkandi
SSSaeed Shamaghdari
Saeed Shamaghdari
Purpose: The purpose of this paper is to design an adaptive nonlinear controller for a nonlinear system of integrated guidance and control. Design/methodology/approach: A nonlinear integrated guidance and control approach is applied to a homing, tail-controlled air vehicle. Adaptive backstepping controller technique is used to deal with the problem, and the Lyapanov theory is used in the stability analysis of the nonlinear system. A nonlinear model of normal force coefficient is obtained from an existing nonlinear model of lift coefficient which was validated by open loop response. The simulation was performed in the pitch plane to prove the benefits of the proposed scheme; however, it can be readily extended to all the three axes. Findings: Monte Carlo simulations indicate that using nonlinear adaptive backstepping formulation meaningfully improves the performance of the system, while it ensures stability of a nonlinear system. Practical implications: The proposed method could be used to obtain better performance of hit to kill accuracy without the expense of control effort. Originality/value: A nonlinear adaptive backstepping controller for nonlinear aerodynamic air vehicle is designed and guaranteed to be stable which is a novel-based approach to the integrated guidance and control. This method makes noticeable performance improvement, and it can be used with hit to kill accuracy.

14 March 2025 in IEEE Transactions on Cybernetics
Verified
Journal Article
Closed access
FGFatemeh Mahdavi Golmisheh
Fatemeh Mahdavi Golmisheh
SSSaeed Shamaghdari
Saeed Shamaghdari
This article presents novel data-driven inverse reinforcement learning (IRL) algorithms to optimally address heterogeneous formation control problems in the presence of disturbances. We propose expert-estimator-learner multiagent systems (MASs) as independent systems with similar interaction graphs. First, a model-based IRL algorithm is introduced for the estimator MAS to determine its optimal control and reward functions. Using the estimator IRL algorithm results, a robust algorithm for model-free IRL is presented to reconstruct the learner MAS’s optimal control and reward functions without knowing the learners’ dynamics. Therefore, estimator MAS aims to estimate experts’ desired formation and learner MAS wants to track the estimators’ trajectories optimally. As a final step, data-driven implementations of these proposed IRL algorithms are presented. Consequently, this research contributes to identifying unknown reward functions and optimal controls by conducting demonstrations. Our analysis shows that the stability and convergence of MASs are thoroughly ensured. The effectiveness of the given algorithms is demonstrated via simulation results.