A microgrid is a small-scale version of a centralized power grid that generates, distributes and regulates electricity flow to local entities using distributed generation and the main grid. Distributed energy storage systems can be used to mitigate adverse effects of intermittent renewable sources in a microgrid in which operators dynamically adjust electricity procurement and storage decisions in response to randomly-evolving demand, renewable supply and pricing information. We formulate a multistage stochastic programming (SP) model whose objective is to minimize the expected total energy costs incurred within a microgrid over a finite planning horizon. The model prescribes the amount of energy to procure, store and discharge in each decision stage of the horizon. However, for even a moderate number of stages, the model is computationally intractable; therefore, we customize the stochastic dual dynamic programming (SDDP) algorithm to obtain high-quality approximate solutions. Computation times and optimization gaps are significantly reduced by implementing a dynamic cut selection procedure and a lower bound improvement scheme within the SDDP framework. An extensive computational study reveals significant cost savings as compared to myopic and non-storage policies, as well as policies obtained using a two-stage SP model. The study also demonstrates the scalability of our solution procedure.
1 January 2014 in Transportation Research Part C: Emerging Technologies
Cyber-physical systems are vulnerable to cyber attacks that can produce serious physical consequences. Our previous work showed how hypergame analysis, an extension of game theory for situations with information asymmetries and player misperceptions, could be applied to control systems subject to deception-based attacks. Here, we build on that research to study a repeated, stochastic context. We consider detection of an attacker attempting to manipulate the control system while remaining undetected. We discuss different monitoring approaches that can be used to do this and define a learning scheme for the defender. In our numerical experiments, we find that the attacker impact and time to detection depend strongly on the cost incurred by the defender in removing an attacker from the system. We also show that the defender learning scheme enforces a strong tradeoff, for the attacker, between remaining undetected, and having an impact on the system.
30 November 2010 in Journal of Intelligent Manufacturing
Adversary emulation is an offensive exercise that provides a comprehensive assessment of a system’s resilience against cyber attacks. However, adversary emulation is typically a manual process, making it costly and hard to deploy in cyber-physical systems (CPS) with complex dynamics, vulnerabilities, and operational uncertainties. In this paper, we develop an automated, domain-aware approach to adversary emulation for CPS. We formulate a Markov Decision Process (MDP) model to determine an optimal attack sequence over a hybrid attack graph with cyber (discrete) and physical (continuous) components and related physical dynamics. We apply model-based and model-free reinforcement learning (RL) methods to solve the discrete-continuous MDP in a tractable fashion. As a baseline, we also develop a greedy attack algorithm and compare it with the RL procedures. We summarize our findings through a numerical study on sensor deception attacks in buildings to compare the performance and solution quality of the proposed algorithms.