29 August 2022 in IEEE Transactions on Industrial Informatics
Intelligent connected vehicle is rapidly growing with the 5-G technology; the diversity of functional interfaces has significantly expanded the avenues of attack, making automotive controller area network (CAN) more vulnerable to cyberthreats. Automotive CAN network attacks are a direct threat to traffic safety, and in this study, we explore the use of intrusion detection techniques for mitigating cyberattacks. However, most automotive CAN network intrusion detection technologies are not capable of defending against sophisticated attacks, making it extremely challenging for detecting intrusions in practice. In this article, we propose a novel time interval conditional entropy method for detecting intrusions in automotive CAN networks. The time interval conditional entropy intrusion detection method is not susceptible to interference and is capable of detecting a variety of attacks. In our experiments, the conditional entropy values of regular communication messages are collected and utilized to distinguish and detect the attacks. The time interval conditional entropy detection method is implemented and evaluated in our controller area net-work bus (CAN-BUS) network platform. The experiments show that our method has higher detection accuracy and is easier to deploy compared to existing automotive CAN network intrusion detection methods.
22 March 2016 in IEEE Transactions on Computational Intelligence and AI in Games
We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular RTS game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.
17 July 2024 in IEEE Access
Multi-Agent Reinforcement Learning (MARL) is extensively utilized for addressing intricate tasks that involve cooperation and competition among agents in Multi-Agent Systems (MAS). However, learning such tasks from scratch is challenging and often unfeasible, especially for MASs with a large number of agents. Hence, leveraging knowledge from prior experiences can effectively expedite the MARL learning process. Prior work has shown that we successfully facilitated transfer learning for MARL by consolidating various state spaces into fixed-size inputs, enabling a single unified deep-learning policy applicable to several scenarios within the StarCraft Multi-Agent Challenge (SMAC) environment. In this study, we expand SMAC to Multi-Player enabled SMAC (MP-SMAC) by enabling the dynamic selection of training opponents and introducing a co-evolving MARL framework, which creates a co-evolutionary arena where multiple policies learn simultaneously. Our arena comprised the simultaneous training of multiple policies in diverse scenarios, pitting them against both static AI opponents and their peers within MP-SMAC. Furthermore, we integrate co-evolution with curriculum transfer learning into Co-MACTRL framework, enabling our MARL policies to systematically acquire knowledge and skills across predetermined scenarios organized by varying difficulty levels, including evolving opponents. The results revealed significant enhancements in MARL learning performance, demonstrating the advantage of leveraging the co-evolving opponents and maneuvering skills obtained from different scenarios. Additionally, the Co-MACTRL learners consistently attained high performance across a range of SMAC scenarios, showcasing the robustness and generalizability of Co-MACTRL.
1 June 2023 in IEEE Access
Multi-agent reinforcement learning (MARL) is a challenging branch of reinforcement learning that requires cooperation of interactive learning agents to achieve individual objectives as well as shared team objectives. Existing MARL algorithms generally use either centralized global state representation or decentralized local observation to perform training and execution. In this paper, we introduce a novel MARL learning paradigm, centralized training with semi-centralized execution (CTSCE), and present a new MARL algorithm for addressing multi-agent problems: Semi-Centralized Multi-Agent Imitation Reinforcement Learning (SC-MAIRL). The semi-centralized approach aggregated with agents’ spatial and temporal information serves as a joint knowledge base to facilitate a learning agent to discover team objectives and make fine-grained decisions. We also utilize a pre-trained performant teacher policy to guide an untrained model towards positive game states as a form of imitation learning, significantly increasing the agent’s learning speed. In addition, for encouraging agents to learn both offensive and defensive behaviors and smoothing the high-dimensional learning curve, we present a new set of reward-shaping functions to further improve SC-MAIRL’s learning performance. Our approach is evaluated using one of the most challenging scenarios within the StarCraft Multi-Agent Challenge environment, and the results show that SC-MAIRL outperforms the state-of-the-art MARL algorithm MAPPO in several metrics and allows our agents to learn and employ novel, complex macro strategies more effectively.