Due to the vulnerability of wireless network control systems (WNCSs), data loss on unreliable channels severely degrades the remote state estimation performance in autonomous driving and connected vehicles. While redundant channels enhance successful transmission probability, they may lead to excessive consumption of limited vehicular network resources. To address this trade-off, this paper investigates optimal scheduling of redundant transmission in remote state estimation over dynamic vehicular links, modeled as Gilbert-Elliott channels. To minimize the long-term average estimation error in the remote estimation process, the redundant transmission scheduling problem is formulated as a Markov decision process (MDP), and deep reinforcement learning (DRL) is employed to solve the infinite-horizon reward optimization problem. To verify the stability of the established MDP model, a sufficient condition guaranteeing the boundedness of the average estimation error is derived. Furthermore, the existence of the optimal deterministic and stationary policy is established based on Bellman's optimality equation. Furthermore, considering the limitations of conventional reinforcement learning methods in processing high-dimensional vehicular communication data, the dueling double deep Q-network (D3QN) algorithm is developed to mitigate learning instability and value overestimation, enabling rapid convergence to the approximate optimal scheduling policy. In conclusion, the validity of the proposed method is corroborated through comprehensive comparisons with the periodic policy, greedy holding times policy, and the deep Q-network algorithm.

The proton exchange membrane fuel cell (PEMFC) is a promising power source, but the short lifespan and high maintenance cost restrict its development and widespread application. Performance degradation prediction is an effective technique to extend the lifespan and reduce the maintenance cost of PEMFC. This paper proposed a novel hybrid method for the performance degradation prediction of PEMFC. Firstly, considering the randomness of PEMFC degradation, a Wiener process model is established to describe the degradation of the aging factor. Secondly, the unscented Kalman filter algorithm is used to estimate the degradation state of the aging factor from monitoring voltage. Then, in order to predict the degradation state of PEMFC, the transformer structure is used to capture the data characteristics and fluctuations of the aging factor. To quantify the uncertainty of the predicted results, we also add the Monte Carlo dropout technology to the transformer to obtain the confidence interval of the predicted result. Finally, the effectiveness and superiority of the proposed method are verified on the experimental datasets.

This paper addressed the optimal policy selection problem of attacker and sensor in cyber-physical systems (CPSs) under denial of service (DoS) attacks. Since the sensor and the attacker have opposite goals, a two-player zero-sum game is introduced to describe the game between the sensor and the attacker, and the Nash equilibrium strategies are studied to obtain the optimal actions. In order to effectively evaluate and quantify the gains, a reinforcement learning algorithm is proposed to dynamically adjust the corresponding strategies. Furthermore, security state estimation is introduced to evaluate the impact of offensive and defensive strategies on CPSs. In the algorithm, the $\epsilon$-greedy policy is improved to make optimal choices based on sufficient learning, achieving a balance of exploration and exploitation. It is worth noting that the channel reliability factor is considered in order to study CPSs with multiple reasons for packet loss. The reinforcement learning algorithm is designed in two scenarios: reliable channel (that is, the reason for packet loss is only DoS attacks) and unreliable channel (the reason for packet loss is not entirely from DoS attacks). The simulation results of the two scenarios show that the proposed reinforcement learning algorithm can quickly converge to the Nash equilibrium policies of both sides, proving the availability and effectiveness of the algorithm.

This paper proposes a reinforcement learning (RL) algorithm for the security problem of state estimation of cyber-physical system (CPS) under denial-of-service (DoS) attacks. The security of CPS will inevitably decline when faced with malicious cyber attacks. In order to analyze the impact of cyber attacks on CPS performance, a Kalman filter, as an adaptive state estimation technology, is combined with an RL method to evaluate the issue of system security, where estimation performance is adopted as an evaluation criterion. Then, the transition of estimation error covariance under a DoS attack is described as a Markov decision process, and the RL algorithm could be applied to resolve the optimal countermeasures. Meanwhile, the interactive combat between defender and attacker could be regarded as a two-player zero-sum game, where the Nash equilibrium policy exists but needs to be solved. Considering the energy constraints, the action selection of both sides will be restricted by setting certain cost functions. The proposed RL approach is designed from three different perspectives, including the defender, the attacker and the interactive game of two opposite sides. In addition, the framework of Q-learning and state–action–reward–state–action (SARSA) methods are investigated separately in this paper to analyze the influence of different RL algorithms. The results show that both algorithms obtain the corresponding optimal policy and the Nash equilibrium policy of the zero-sum interactive game. Through comparative analysis of two algorithms, it is verified that the differences between Q-Learning and SARSA could be applied effectively into the secure state estimation in CPS.

This paper is concerned with the state fusion estimation problem for a class of stochastic hybrid systems with asynchronous multi-sensors and multiple packet dropouts. The stochastic packet dropouts over communication channels from sensors to the fusion center are formulated as independent Bernoulli sequences. As one of the most cost-effective estimation approach for hybrid systems, the interactive multiple model framework is adopted, where an input mixing step is introduced at the beginning of each filtering cycle. The asynchronous sensor measurements collected at the fusion center are first aligned to the fusion time, and then fused to update the mode-matched filters based on the quasi-recursive form of linear minimum mean square error estimation, where correlations induced by the synchronization process are carefully calculated and the stochastic packet dropouts are taken into account. Finally, the overall estimate is obtained by further fusing the mode-matched estimates with their updated posterior mode probabilities accordingly. The feasibility and effectiveness of the proposed algorithm is illustrated by a numerical simulation.