28 December 2022 in Sustainable Energy, Grids and Networks
1 June 2023 in IEEE Transactions on Intelligent Transportation Systems
We develop a new algorithm, based on approximate dynamic programming, to efficiently schedule the charging processes of a large number of electric vehicles (EVs) with flexibility, which allows the EV charging service provider to charge the EVs between their minimum and target state of charge (SoC) within a time window. We assume that the charging demands have known distributions and the charging process of EVs can be scheduled preemptively. We model the scheduling problem as a dynamic program with state-dependent constraints and solve it utilizing the fitted value iteration. We propose a multi-stage decoupled algorithm to reduce the dimensions in the fitted value iteration to reduce the computation complexity, and provide sufficient conditions for achieving optimality.
2 November 2021 in IEEE Transactions on Control of Network Systems
This article devises a novel fair pricing policy for the on-demand transportation system under asymmetrical demand patterns. We assume that the city comprises multiple nodes with directed edges connecting the nodes. At each node, passengers arrive according to a Poisson distribution. Based on the arrival processes of the passengers and the operational cost of moving a vehicle from one node to another, we devise a fair pricing scheme for the passengers using the Shapley value. We show that at certain nodes, as the arrival rate or the valuation distribution of the passengers increases, the fair pricing policy increases up to a threshold, and then decreases after a different threshold. Our numerical simulations illustrate the theoretical findings.
29 December 2022 in IEEE Transactions on Information Forensics and Security
A bus-off attack is a denial-of-service (DoS) attack which exploits error handling in the controller area network (CAN) to induce an honest node to disconnect itself from the CAN bus. This paper develops a stochastic transmission policy as a countermeasure for the controller-transmitter pair against the bus-off attack. We model this as a non-zero-sum linear-quadratic-Gaussian game between the controller-transmitter pair and the attacker. We derive Nash equilibria of the game for two different information structures of the attacker. We show that the attacker has a dominant attack strategy under both information structures. Under the dominant attack strategy, we show that the optimal control policy is linear in the system state. We further identify a necessary and a sufficient conditions on the transmission policy to have bounded average cost. The theoretical results are complemented by a detailed case study of a bus-off attack on a vehicular adaptive cruise control model.
24 June 2021 in IEEE Control Systems Letters
In this letter, we establish the convergence of fitted value iteration and fitted Q-value iteration for continuous-state continuous-action Markov decision problems (MDPs) with state-dependent action sets. We further extend the algorithm and the convergence result to the case of monotone MDPs.
4 February 2025 in IEEE Open Journal of Control Systems
We consider risk-sensitive Markov decision processes (MDPs), where the MDP model is influenced by a parameter which takes values in a compact metric space. These situations arise when the underlying dynamics of the system depend on parameters that drifts over time. For example, mass of a vehicle depends on the number of passengers in the vehicle, which may change from one trip to another. Similarly, the energy demand of a building depends on the local weather, which changes every hour of the day. We identify sufficient conditions under which small perturbations in the model parameters lead to small changes in the optimal value function and optimal policy. This is achieved by establishing the continuity of the value function with respect to the parameters. A direct consequence of this result is that an optimal policy under a specific parameter remains near-optimal if the parameter is perturbed slightly. Implications of the results for data-driven decision-making, decision-making with preference uncertainty, and systems with changing noise distributions are discussed.